PROBLEM ANALYSIS:
Signal that lost -$32: ADX 14.8, VOL 2.29x → scored 70-90 (PASSED)
Signal that won +3%: ADX 15.7, VOL 1.18x → scored 45-65 (got BLOCKED before fix)
Key insight: High volume during choppy conditions (ADX < 16) indicates
whipsaw/trap, not genuine breakout. Our volume bonus (+15 pts for >1.5x)
was rewarding flip-flop signals instead of real moves.
FIX:
Add anti-chop filter in volume scoring:
- If ADX < 16 AND volume > 1.5x → -15 points (whipsaw trap)
- Overrides the normal +15 bonus for high volume
- Protects against false signals during consolidation
IMPACT ON RECENT SIGNALS:
1. 00:40 SHORT (ADX 17.2, VOL 0.98): 55→75 ✅ Still passes
2. 00:55 LONG (ADX 15, VOL 0.47): 35→55 ❌ Still blocked (correct, weak vol)
3. 01:05 SHORT (ADX 14.8, VOL 2.29): 70→60 ⚠️ Now flagged as whipsaw trap
4. 01:10 LONG (ADX 15.7, VOL 1.18): 45→65 ✅ Catches the +3% runup
Result: Loser signal now barely passes (60) with warning flag,
winner signal passes cleanly (65). Better risk/reward profile.
- Fix external closure P&L using tp1Hit flag instead of currentSize
- Add direction change detection to prevent false TP1 on signal flips
- Signal flips now recorded with accurate P&L as 'manual' exits
- Add retry logic with exponential backoff for Solana RPC rate limits
- Create /api/trading/cancel-orders endpoint for manual cleanup
- Improves data integrity for win/loss statistics
- Add market data cache service (5min expiry) for storing TradingView metrics
- Create /api/trading/market-data webhook endpoint for continuous data updates
- Add /api/analytics/reentry-check endpoint for validating manual trades
- Update execute endpoint to auto-cache metrics from incoming signals
- Enhance Telegram bot with pre-execution analytics validation
- Support --force flag to override analytics blocks
- Use fresh ADX/ATR/RSI data when available, fallback to historical
- Apply performance modifiers: -20 for losing streaks, +10 for winning
- Minimum re-entry score 55 (vs 60 for new signals)
- Fail-open design: proceeds if analytics unavailable
- Show data freshness and source in Telegram responses
- Add comprehensive setup guide in docs/guides/REENTRY_ANALYTICS_QUICKSTART.md
Phase 1 implementation for smart manual trade validation.
- Add ATR-based dynamic TP2 scaling from 0.7% to 3.0% based on volatility
- New config options: useAtrBasedTargets, atrMultiplierForTp2, minTp2Percent, maxTp2Percent
- Enhanced settings UI with ATR controls and updated risk calculator
- Fix external closure P&L calculation using unrealized P&L instead of volatile current price
- Update execute and test endpoints to use calculateDynamicTp2() function
- Maintain 25% runner system for capturing extended moves (4-5% targets)
- Add environment variables for ATR-based configuration
- Better P&L accuracy for manual position closures
- Fix P&L calculation in Position Manager to use actual entry vs exit price instead of SDK's potentially incorrect realizedPnL
- Calculate actual profit percentage and apply to closed position size for accurate dollar amounts
- Update database record for last trade from incorrect 6.58 to actual .66 P&L
- Update .github/copilot-instructions.md to reflect TP2-as-runner system changes
- Document 25% runner system (5x larger than old 5%) with ATR-based trailing
- Add critical P&L calculation pattern to common pitfalls section
- Mark Phase 5 complete in development roadmap
CHANGE: TP2 now activates trailing stop on full 25% remaining instead
of closing 80% and leaving 5% runner.
Benefits:
- 5x larger runner (25% vs 5%) = 25 vs 05 on 100 position
- Eliminates Drift minimum size issues completely
- Simplifies logic - no more canUseRunner() viability checks
- Better R:R on extended moves
New flow:
- TP1 (+0.4%): Close 75%, keep 25%
- TP2 (+0.7%): Skip close, activate trailing stop on full 25%
- Runner: 25% with ATR-based trailing (0.25-0.9%)
Config change: takeProfit2SizePercent: 80 → 0
Position Manager: Remove canUseRunner logic, activate trailing at TP2 hit
PROBLEM: Runner never activated because Drift force-closes positions below
minimum size. TP2 would close 80% leaving 5% runner (~$105), but Drift
automatically closed the entire position.
SOLUTION:
1. Created runner-calculator.ts with canUseRunner() to check if remaining
size would be above Drift minimums BEFORE executing TP2 close
2. If runner not viable: Skip TP2 close entirely, activate trailing stop
on full 25% remaining (from TP1)
3. If runner viable: Execute TP2 as normal, activate trailing on 5%
Benefits:
- Runner system will now actually work for viable position sizes
- Positions that are too small won't try to force-close below minimums
- Better logs showing why runner did/didn't activate
- Trailing stop works on larger % if runner not viable (better R:R)
Example: $2100 position → $525 after TP1 → $105 runner = VIABLE
$4 ETH position → $1 after TP1 → $0.20 runner = NOT VIABLE
Runner will trail with ATR-based dynamic % (0.25-0.9%) below peak price.
- Shorts/longs at < 15% range require ADX > 18 AND volume > 1.2x
- OR RSI < 35 for shorts, RSI > 60 for longs
- Increased penalty from -10 to -15 when conditions not met
- Changed threshold from < 5% to < 15% to catch more edge cases
Test results:
- Big loser (01:35): ADX 16.1, price 9.3% → Score 60 (was 90) → BLOCKED
- Today's signal (10:05): ADX 17.3, price 0.9% → Score 55 (was 85) → BLOCKED
Rationale: False breakdowns in choppy ranges (ADX < 18) cause losses.
Tradeoff: May block some profitable breakdowns, but prevents chop losses.
- Allow shorts at range bottom (<5%) with volume >1.2x OR RSI <40
- Allow longs at range bottom with volume >1.2x OR RSI >60
- Reduce ADX penalty from -15 to -5 when strong volume (>1.2x) present
- Reduce price position penalties from -15 to -10 (less harsh)
- Volume compensation recognizes breakdowns start before ADX strengthens
Test case (blocked signal that would have profited):
- OLD: ATR 0.32, ADX 17.3, RSI 32.5, Vol 1.27x, Price 0.9% → Score 45 (blocked)
- NEW: Same metrics → Score 85 (executes)
Rationale: Breakdowns continue lower, volume confirms conviction, ADX lags price action
- Position Manager was calculating P&L using tracked size instead of actual on-chain size
- Example: Tracked 100, actual 0.04 SOL () = -99.63% false loss instead of -0.32%
- Fixed external closure detection to use position.size * currentPrice as lastKnownSize
- Manually corrected phantom trade P&L from -092.25 to /bin/bash
- Total P&L corrected: -013.92 → +8.33 (accurate)
- Prevents all future phantom/mismatch trades from wildly incorrect P&L
Modified:
- lib/trading/position-manager.ts lines 421-445 (external closure P&L calculation)
**Feature: Position Scaling**
Allows adding to existing profitable positions when high-quality signals confirm trend strength.
**Configuration (config/trading.ts):**
- enablePositionScaling: false (disabled by default - enable after testing)
- minScaleQualityScore: 75 (higher bar than initial 60)
- minProfitForScale: 0.4% (must be at/past TP1)
- maxScaleMultiplier: 2.0 (max 200% of original size)
- scaleSizePercent: 50% (add 50% of original position)
- minAdxIncrease: 5 (ADX must strengthen)
- maxPricePositionForScale: 70% (don't chase resistance)
**Validation Logic (check-risk endpoint):**
Same-direction signal triggers scaling check if enabled:
1. Quality score ≥75 (stronger than initial entry)
2. Position profitable ≥0.4% (at/past TP1)
3. ADX increased ≥5 points (trend strengthening)
4. Price position <70% (not near resistance)
5. Total size <2x original (risk management)
6. Returns 'allowed: true, reason: Position scaling' if all pass
**Execution (execute endpoint):**
- Opens additional position at scale size (50% of original)
- Updates ActiveTrade: timesScaled, totalScaleAdded, currentSize
- Tracks originalAdx from first entry for comparison
- Returns 'action: scaled' with scale details
**ActiveTrade Interface:**
Added fields:
- originalAdx?: number (for scaling validation)
- timesScaled?: number (track scaling count)
- totalScaleAdded?: number (total USD added)
**Example Scenario:**
1. LONG SOL at $176 (quality: 45, ADX: 13.4) - weak but entered
2. Price hits $176.70 (+0.4%) - at TP1
3. New LONG signal (quality: 78, ADX: 19) - strong confirmation
4. Scaling validation: ✅ Quality 78 ✅ Profit +0.4% ✅ ADX +5.6 ✅ Price 68%
5. Adds 50% more position at $176.70
6. Total position: 150% of original size
**Conservative Design:**
- Disabled by default (requires manual enabling)
- Only scales INTO profitable positions (never averaging down)
- Requires significant quality improvement (75 vs 60)
- Requires trend confirmation (ADX increase)
- Hard cap at 2x original size
- Won't chase near resistance levels
**Next Steps:**
1. Enable in settings: ENABLE_POSITION_SCALING=true
2. Test with small positions first
3. Monitor data: do scaled positions outperform?
4. Adjust thresholds based on results
**Safety:**
- All existing duplicate prevention logic intact
- Flip logic unchanged (still requires quality check)
- Position Manager tracks scaling state
- Can be toggled on/off without code changes
**Root Causes:**
1. Auto-flip logic could create phantom trades if close failed
2. Position size mismatches (0.01 SOL vs 11.92 SOL expected) not caught
3. Multiple trades for same symbol+direction in database
**Preventive Measures:**
1. **Startup Validation (lib/startup/init-position-manager.ts)**
- Validates all open trades against Drift positions on startup
- Auto-closes phantom trades with <50% expected size
- Logs size mismatches for manual review
- Prevents Position Manager from tracking ghost positions
2. **Duplicate Position Prevention (app/api/trading/execute/route.ts)**
- Blocks opening same-direction position on same symbol
- Returns 400 error if duplicate detected
- Only allows auto-flip (opposite direction close + open)
3. **Runtime Phantom Detection (lib/trading/position-manager.ts)**
- Checks position size every 2s monitoring cycle
- Auto-closes if size ratio <50% (extreme mismatch)
- Logs as 'manual' exit with AUTO_CLEANUP tx
- Removes from monitoring immediately
4. **Quality Score Fix (app/api/trading/check-risk/route.ts)**
- Hardcoded minScore=60 (removed non-existent config reference)
**Prevention Summary:**
- ✅ Startup validation catches historical phantoms
- ✅ Duplicate check prevents new phantoms
- ✅ Runtime detection catches size mismatches <30s after they occur
- ✅ All three layers work together for defense-in-depth
Issue: User had LONG (phantom) + SHORT (undersized 0.01 SOL vs 11.92 expected)
Fix: Both detected and closed, bot now clean with 0 active trades
- Add SymbolSettings interface with enabled/positionSize/leverage fields
- Implement per-symbol ENV variables (SOLANA_*, ETHEREUM_*)
- Add SOL and ETH sections to settings UI with enable/disable toggles
- Add symbol-specific test buttons (SOL LONG/SHORT, ETH LONG/SHORT)
- Update execute and test endpoints to check symbol enabled status
- Add real-time risk/reward calculator per symbol
- Rename 'Position Sizing' to 'Global Fallback' for clarity
- Fix position manager P&L calculation for externally closed positions
- Fix zero P&L bug affecting 12 historical trades
- Add SQL scripts for recalculating historical P&L data
- Move archive TypeScript files to .archive to fix build
Defaults:
- SOL: 10 base × 10x leverage = 100 notional (profit trading)
- ETH: base × 1x leverage = notional (data collection)
- Global: 10 × 10x for BTC and other symbols
Configuration priority: Per-symbol ENV > Market config > Global ENV > Defaults
CRITICAL BUG: Position Manager was using current price to determine exit reason,
but on-chain orders filled at a DIFFERENT price in the past!
Example: LONG entry $184.55, TP1 filled at $184.66, but when Position Manager
checked later (price dropped), it saw currentPrice < TP1 and defaulted to 'SL'
Result: Profitable trades incorrectly labeled as SL exits in database
Fix:
- Use trade.tp1Hit and trade.tp2Hit flags to determine exit reason
- If no TP flags set, use realized P&L to distinguish:
- Profit >0.5% = TP1 filled
- Negative P&L = SL filled
- Remove duplicate P&L calculation
This ensures exit reasons match actual on-chain order fills
Added Maximum Favorable/Adverse Excursion tracking:
- Track maxFavorableExcursion: best profit % reached during trade
- Track maxAdverseExcursion: worst loss % reached during trade
- Track maxFavorablePrice and maxAdversePrice
- Update every price check (2s interval)
- Save to database on trade exit for optimization analysis
Benefits:
- Identify if TP levels are too conservative (MFE consistently higher)
- Determine if SL is too tight (MAE < SL but trade recovers)
- Optimize runner size based on how often MFE >> TP2
- Data-driven exit strategy tuning after collecting 10-20 trades
Display in monitoring logs: Shows MFE/MAE % every 20 seconds
CRITICAL: After TP1 closes 75%, the on-chain stop loss orders were NOT being updated
- Position Manager was tracking new SL price internally but not updating Drift orders
- Old SL orders (e.g., $181.69) remained active even after TP1 at $185.28
- This prevented the 'move SL to breakeven after TP1' logic from working
Fix:
- After TP1 hits, cancel ALL old orders on-chain
- Place new SL orders at updated price (breakeven + configured %)
- Place remaining TP2 order for the 25% runner position
- Maintains dual-stop system if enabled
Result: SL will now actually move up on Drift UI after TP1 fires
- Convert closePosition.closedSize (base asset) to USD when updating trade.currentSize
- Fix conversion when position.size detected from Drift: set currentSize = position.size * currentPrice
- Prevent trade.currentSize from being reduced to tiny values due to unit mismatch
- Position Manager now detects TP1/TP2 fills by monitoring position size reductions instead of entry price mismatches
- When position size reduces by ~75%, marks TP1 as filled and updates currentSize
- When position size reduces by ~95%, marks TP2 as filled and activates trailing stop for 5% runner
- Entry price mismatch check now skipped after TP fills (Drift shows weighted average entry price after partial closes)
- Fixes bug where runners were incorrectly closed after TP1/TP2 fired on-chain
- Adds grace period for new trades (<30s) to avoid false positives during blockchain propagation delays
- This unblocks Phase 1 data collection for signal quality optimization (need 10+ trades with MAE/MFE data)
- Add qualityScore to ExecuteTradeResponse interface and response object
- Update analytics page to always show Signal Quality card (N/A if unavailable)
- Fix n8n workflow to pass context metrics and qualityScore to execute endpoint
- Fix timezone in Telegram notifications (Europe/Berlin)
- Fix symbol normalization in /api/trading/close endpoint
- Update Drift ETH-PERP minimum order size (0.002 ETH not 0.01)
- Add transaction confirmation to closePosition() to prevent phantom closes
- Add 30-second grace period for new trades in Position Manager
- Fix execution order: database save before Position Manager.addTrade()
- Update copilot instructions with transaction confirmation pattern
Bug: Position Manager was comparing ANY position on the symbol to the trade being
tracked, without verifying entry price match. When a new position opened, it would
think the old tracked trade 'closed externally' and cancel ALL orders - including
the new position's exit orders.
Fix: Added entry price verification (0.5% tolerance). If position entry price doesn't
match the tracked trade, mark the old trade as 'lost tracking' and remove from
monitoring WITHOUT cancelling orders (they belong to the new position).
This prevents the catastrophic scenario where exit orders are repeatedly cancelled,
leaving positions unprotected.
Bug: MAE/MFE was tracked in memory during trades but not saved to database on exit
Cause: updateTradeExit() wasn't receiving or saving MAE/MFE parameters
Changes:
- Added MAE/MFE fields to UpdateTradeExitParams interface
- Modified updateTradeExit() to save maxFavorableExcursion, maxAdverseExcursion, maxFavorablePrice, maxAdversePrice
- Updated both updateTradeExit() calls in Position Manager to pass MAE/MFE values
- Enhanced exit logging to show final MAE/MFE percentages
Impact: Future trades will now properly save MAE/MFE data for analytics
Note: Past 2 trades (from before this fix) don't have MAE/MFE saved
- Detect on-chain TP2 fills in size mismatch logic and set tp2Hit flag
- Position size thresholds: <30% = TP1, <10% = TP2 (prevents runner from being closed)
- Ensures runner (5-20%) trails properly instead of being market-closed immediately
- Remove saveTradeState() call from addTrade() to avoid P2025 error
- Add initialization check in checkTradeConditions() to skip when Drift not ready
- Silence 'not initialized' errors during startup (expected behavior)
- Trade state is now saved only by API endpoint after DB record created
- Save currentSize before it becomes 0 in external closure detection
- Use sizeBeforeClosure for P&L calculation instead of trade.currentSize
- Prevents /bin/bash.00 P&L for TP2 exits when position closes externally
- Ensures win/loss analytics counts TP trades correctly
**Root Cause:** Position Manager didn't detect when on-chain TP/SL orders closed positions externally, causing endless error loops and stale position data.
**Issues Fixed:**
1. Position Manager now checks if on-chain position still exists before attempting to close
2. Detects external closures (by on-chain orders) and updates database accordingly
3. Determines likely exit reason based on price vs TP/SL levels
4. Automatically cancels leftover orders when position detected as closed
5. Analytics now properly shows stopped-out trades
**Technical Changes:**
- Added position existence check at start of checkTradeConditions()
- Calls DriftService.getPosition() to verify on-chain state
- Updates database with exitPrice, exitReason, realizedPnL when external closure detected
- Removes trade from monitoring after external closure
- Handles size mismatches for partial closes (TP1 hit externally)
**Database Fix:**
- Manually closed orphaned trade (stopped out 9 hours ago but still marked 'open')
- Calculated and set realizedPnL = -$12.00 for stopped-out SHORT position
- Analytics now shows 3 total trades instead of missing the SL exit
**Testing:**
- Bot starts cleanly with no error loops
- Position monitoring active with 0 trades (as expected)
- Analytics correctly shows stopped-out trade in statistics
- Added /close Telegram command for full position closure
- Updated /reduce to accept 10-100% (was 10-90%)
- Implemented auto-flip logic: automatically closes opposite position when signal reverses
- Fixed risk check to allow opposite direction trades (signal flips)
- Enhanced Position Manager to cancel orders when removing trades
- Added startup initialization for Position Manager (restores trades on restart)
- Fixed analytics to show stopped-out trades (manual DB update for orphaned trade)
- Updated reduce endpoint to route 100% closes through closePosition for proper cleanup
- All position closures now guarantee TP/SL order cancellation on Drift
- Implemented trailing stop logic in Position Manager for remaining position after TP2
- Added new ActiveTrade fields: tp2Hit, trailingStopActive, peakPrice
- New config settings: useTrailingStop, trailingStopPercent, trailingStopActivation
- Added trailing stop UI section in settings page with explanations
- Fixed env file parsing regex to support numbers in variable names (A-Z0-9_)
- Settings now persist correctly across container restarts
- Added back arrow navigation on settings page
- Updated all API endpoints and test files with new fields
- Trailing stop activates when runner reaches configured profit level
- SL trails below peak price by configurable percentage
- Add Position Manager state persistence to survive restarts
- Auto-restore open trades from database on startup
- Save state after TP1, SL adjustments, profit locks
- Persist to configSnapshot JSON field
- Add automatic order cancellation
- Cancel all TP/SL orders when position fully closed
- New cancelAllOrders() function in drift/orders.ts
- Prevents orphaned orders after manual closes
- Improve stop loss management
- Move SL to +0.35% after TP1 (was +0.15%)
- Gives more breathing room for retracements
- Still locks in half of TP1 profit
- Add database sync when Position Manager closes trades
- Auto-update Trade record with exit data
- Save P&L, exit reason, hold time
- Fix analytics showing stale data
- Add trade state management functions
- updateTradeState() for Position Manager persistence
- getOpenTrades() for startup restoration
- getInitializedPositionManager() for async init
- Create n8n database analytics workflows
- Daily report workflow (automated at midnight)
- Pattern analysis (hourly/daily performance)
- Stop loss effectiveness analysis
- Database analytics query workflow
- Complete setup guide (N8N_DATABASE_SETUP.md)
Features:
- Autonomous trading system with Drift Protocol on Solana
- Real-time position monitoring with Pyth price feeds
- Dynamic stop-loss and take-profit management
- n8n workflow integration for TradingView signals
- Beautiful web UI for settings management
- REST API for trade execution and monitoring
- Next.js 15 with standalone output mode
- TypeScript with strict typing
- Docker containerization with multi-stage builds
- PostgreSQL database for trade history
- Singleton pattern for Drift client connection pooling
- BN.js for BigNumber handling (Drift SDK requirement)
- Configurable stop-loss and take-profit levels
- Breakeven trigger and profit locking
- Daily loss limits and trade cooldowns
- Slippage tolerance controls
- DRY_RUN mode for safe testing
- Real-time risk calculator
- Interactive sliders for all parameters
- Live preview of trade outcomes
- Position sizing and leverage controls
- Beautiful gradient design with Tailwind CSS
- POST /api/trading/execute - Execute trades
- POST /api/trading/close - Close positions
- GET /api/trading/positions - Monitor active trades
- GET /api/trading/check-risk - Validate trade signals
- GET /api/settings - View configuration
- POST /api/settings - Update configuration
- Fixed Borsh serialization errors (simplified order params)
- Resolved RPC rate limiting with singleton pattern
- Fixed BigInt vs BN type mismatches
- Corrected order execution flow
- Improved position state management
- Complete setup guides
- Docker deployment instructions
- n8n workflow configuration
- API reference documentation
- Risk management guidelines
- Runs on port 3001 (external), 3000 (internal)
- Uses Helius RPC for optimal performance
- Production-ready with error handling
- Health monitoring and logging
