- Auto-close phantom positions immediately via market order
- Return HTTP 200 (not 500) to allow n8n workflow continuation
- Save phantom trades to database with full P&L tracking
- Exit reason: 'manual' category for phantom auto-closes
- Protects user during unavailable hours (sleeping, no phone)
- Add Docker build best practices to instructions (background + tail)
- Document phantom system as Critical Component #1
- Add Common Pitfall #30: Phantom notification workflow
Why auto-close:
- User can't always respond to phantom alerts
- Unmonitored position = unlimited risk exposure
- Better to exit with small loss/gain than leave exposed
- Re-entry possible if setup actually good
Files changed:
- app/api/trading/execute/route.ts: Auto-close logic
- .github/copilot-instructions.md: Documentation + build pattern
Root Cause:
- Execute endpoint saved to database AFTER adding to Position Manager
- Database save failures were silently caught and ignored
- API returned success even when DB save failed
- Container restarts lost in-memory Position Manager state
- Result: Unprotected positions with no TP/SL monitoring
Fixes Applied:
1. Database-First Pattern (app/api/trading/execute/route.ts):
- MOVED createTrade() BEFORE positionManager.addTrade()
- If database save fails, return HTTP 500 with critical error
- Error message: 'CLOSE POSITION MANUALLY IMMEDIATELY'
- Position Manager only tracks database-persisted trades
- Ensures container restarts can restore all positions
2. Transaction Timeout (lib/drift/orders.ts):
- Added 30s timeout to confirmTransaction() in closePosition()
- Prevents API from hanging during network congestion
- Uses Promise.race() pattern for timeout enforcement
3. Telegram Error Messages (telegram_command_bot.py):
- Parse JSON for ALL responses (not just 200 OK)
- Extract detailed error messages from 'message' field
- Shows critical warnings to user immediately
- Fail-open: proceeds if analytics check fails
4. Position Manager (lib/trading/position-manager.ts):
- Move lastPrice update to TOP of monitoring loop
- Ensures /status endpoint always shows current price
Verification:
- Test trade cmhxj8qxl0000od076m21l58z executed successfully
- Database save completed BEFORE Position Manager tracking
- SL triggered correctly at -$4.21 after 15 minutes
- All protection systems working as expected
Impact:
- Eliminates risk of unprotected positions
- Provides immediate critical warnings if DB fails
- Enables safe container restarts with full position recovery
- Verified with live test trade on production
See: CRITICAL_INCIDENT_UNPROTECTED_POSITION.md for full incident report
Fixed Telegram notification showing wrong leverage (10x instead of 20x).
Problem:
- SOL trades use SOLANA_LEVERAGE=20x (per-symbol override)
- API response was returning config.leverage (global default 10x)
- n8n workflow displayed incorrect leverage value
Changes:
- Line 345: Use 'leverage' variable (from getPositionSizeForSymbol)
- Line 448: ActiveTrade uses actual leverage
- Line 522: ExecuteTradeResponse uses actual leverage
- Line 557: Database createTrade() uses actual leverage
Now notifications correctly show 20x for SOL trades.
Added indicatorVersion field to track which TradingView indicator version
generated each signal (v5, v6, etc.)
Changes:
- Updated ExecuteTradeRequest interface to include indicatorVersion field
- Added indicatorVersion to both createTrade() calls with default 'v5' fallback
- Field already exists in Prisma schema (indicatorVersion String?)
- Defaults to 'v5' for backward compatibility with old alerts
This enables comparison of indicator performance:
- v5: Original Money Line indicator
- v6: Improved version with 100-bar price position filter
Works alongside existing signalQualityVersion (v4) which tracks backend
scoring algorithm changes. Two separate version fields:
1. indicatorVersion = TradingView Pine Script version (v5/v6)
2. signalQualityVersion = Backend scoring logic version (v4)
Frontend can now filter/compare trades by indicator version in analytics.
- Add BlockedSignal table with 25 fields for comprehensive signal analysis
- Track all blocked signals with metrics (ATR, ADX, RSI, volume, price position)
- Store quality scores, block reasons, and detailed breakdowns
- Include future fields for automated price analysis (priceAfter1/5/15/30Min)
- Restore signalQualityVersion field to Trade table
Database changes:
- New table: BlockedSignal with indexes on symbol, createdAt, score, blockReason
- Fixed schema drift from manual changes
API changes:
- Modified check-risk endpoint to save blocked signals automatically
- Fixed hasContextMetrics variable scope (moved to line 209)
- Save blocks for: quality score too low, cooldown period, hourly limit
- Use config.minSignalQualityScore instead of hardcoded 60
Database helpers:
- Added createBlockedSignal() function with try/catch safety
- Added getRecentBlockedSignals(limit) for queries
- Added getBlockedSignalsForAnalysis(olderThanMinutes) for automation
Documentation:
- Created BLOCKED_SIGNALS_TRACKING.md with SQL queries and analysis workflow
- Created SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md with 5-phase plan
- Documented data-first approach: collect 10-20 signals before optimization
Rationale:
Only 2 historical trades scored 60-64 (insufficient sample size for threshold decision).
Building data collection infrastructure before making premature optimizations.
Phase 1 (current): Collect blocked signals for 1-2 weeks
Phase 2 (next): Analyze patterns and make data-driven threshold decision
Phase 3-5 (future): Automation and ML optimization
BUG FOUND:
Line 558: tp2SizePercent: config.takeProfit2SizePercent || 100
When config.takeProfit2SizePercent = 0 (TP2-as-runner system), JavaScript's ||
operator treats 0 as falsy and falls back to 100, causing TP2 to close 100%
of remaining position instead of activating trailing stop.
IMPACT:
- On-chain orders placed correctly (line 481 uses ?? correctly)
- Position Manager reads from DB and expects TP2 to close position
- Result: User sees TWO take-profit orders instead of runner system
FIX:
Changed both tp1SizePercent and tp2SizePercent to use ?? operator:
- tp1SizePercent: config.takeProfit1SizePercent ?? 75
- tp2SizePercent: config.takeProfit2SizePercent ?? 0
This allows 0 value to be saved correctly for TP2-as-runner system.
VERIFICATION NEEDED:
Current open SHORT position in database has tp2SizePercent=100 from before
this fix. Next trade will use correct runner system.
- New /api/trading/sync-positions endpoint (no auth)
- Fetches actual Drift positions and compares with Position Manager
- Removes stale tracking, adds missing positions with calculated TP/SL
- Settings UI: Orange 'Sync Positions' button added
- CLI script: scripts/sync-positions.sh for terminal access
- Full documentation in docs/guides/POSITION_SYNC_GUIDE.md
- Quick reference: POSITION_SYNC_QUICK_REF.md
- Updated AI instructions with pitfall #23
Problem solved: Manual Telegram trades with partial fills can cause
Position Manager to lose tracking, leaving positions without software-
based stop loss protection. This feature restores dual-layer protection.
Note: Docker build not picking up route yet (cache issue), needs investigation
- Add usePercentageSize flag to SymbolSettings and TradingConfig
- Add calculateActualPositionSize() and getActualPositionSizeForSymbol() helpers
- Update execute and test endpoints to calculate position size from free collateral
- Add SOLANA_USE_PERCENTAGE_SIZE, ETHEREUM_USE_PERCENTAGE_SIZE, USE_PERCENTAGE_SIZE env vars
- Configure SOL to use 100% of portfolio (auto-adjusts to available balance)
- Fix TypeScript errors: replace fillNotionalUSD with actualSizeUSD
- Remove signalQualityVersion and fullyClosed references (not in interfaces)
- Add comprehensive documentation in PERCENTAGE_SIZING_FEATURE.md
Benefits:
- Prevents insufficient collateral errors by using available balance
- Auto-scales positions as account grows/shrinks
- Maintains risk proportional to capital
- Flexible per-symbol configuration (SOL percentage, ETH fixed)
- Fix external closure P&L using tp1Hit flag instead of currentSize
- Add direction change detection to prevent false TP1 on signal flips
- Signal flips now recorded with accurate P&L as 'manual' exits
- Add retry logic with exponential backoff for Solana RPC rate limits
- Create /api/trading/cancel-orders endpoint for manual cleanup
- Improves data integrity for win/loss statistics
- Change tp2SizePercent fallback from || 100 to ?? 0
- Allows 0 value to pass through (means 'activate trailing stop, don't close')
- Fixes bug where TP2 was closing 100% of remaining position
- Now correctly leaves 25% runner after TP1 closes 75%
- Applied to both execute and test endpoints
- Add market data cache service (5min expiry) for storing TradingView metrics
- Create /api/trading/market-data webhook endpoint for continuous data updates
- Add /api/analytics/reentry-check endpoint for validating manual trades
- Update execute endpoint to auto-cache metrics from incoming signals
- Enhance Telegram bot with pre-execution analytics validation
- Support --force flag to override analytics blocks
- Use fresh ADX/ATR/RSI data when available, fallback to historical
- Apply performance modifiers: -20 for losing streaks, +10 for winning
- Minimum re-entry score 55 (vs 60 for new signals)
- Fail-open design: proceeds if analytics unavailable
- Show data freshness and source in Telegram responses
- Add comprehensive setup guide in docs/guides/REENTRY_ANALYTICS_QUICKSTART.md
Phase 1 implementation for smart manual trade validation.
- Add ATR-based dynamic TP2 scaling from 0.7% to 3.0% based on volatility
- New config options: useAtrBasedTargets, atrMultiplierForTp2, minTp2Percent, maxTp2Percent
- Enhanced settings UI with ATR controls and updated risk calculator
- Fix external closure P&L calculation using unrealized P&L instead of volatile current price
- Update execute and test endpoints to use calculateDynamicTp2() function
- Maintain 25% runner system for capturing extended moves (4-5% targets)
- Add environment variables for ATR-based configuration
- Better P&L accuracy for manual position closures
- Changed numeric fields from typed as number to 'any' in raw query results
- Properly convert Prisma Decimal/BigInt types to JavaScript numbers
- Fixes TypeError: e.totalPnL.toFixed is not a function
- All numeric values (totalPnL, avgPnL, avgADX, etc.) now converted with Number()
Issue: Prisma returns Decimal objects from aggregation queries which don't have
toFixed() method. Frontend expects plain numbers for .toFixed(2) formatting.
- Added signalQualityVersion field to Trade model
- Tracks which scoring logic version was used for each trade
- v1: Original logic (price position < 5% threshold)
- v2: Added volume compensation for low ADX
- v3: CURRENT - Stricter logic requiring ADX > 18 for extreme positions (< 15%)
This enables future analysis to:
- Compare performance between logic versions
- Filter trades by scoring algorithm
- Data-driven improvements based on clean datasets
All new trades will be marked as v3. Old trades remain null/v1 for comparison.
- Detect position size mismatches (>50% variance) after opening
- Save phantom trades to database with expectedSizeUSD, actualSizeUSD, phantomReason
- Return error from execute endpoint to prevent Position Manager tracking
- Add comprehensive documentation of phantom trade issue and solution
- Enable data collection for pattern analysis and future optimization
Fixes oracle price lag issue during volatile markets where transactions
confirm but positions don't actually open at expected size.
- Remove trade from Position Manager BEFORE closing Drift position (prevents race condition)
- Explicitly save closure to database with proper P&L calculation
- Mark flipped positions as 'manual' exit reason
- Increase delay from 1s to 2s for better on-chain confirmation
- Preserve MAE/MFE data in closure records
Fixes issue where SHORT signal would close LONG but not properly track the new SHORT position.
Database now correctly records both old position closure and new position opening.
**UI Updates (settings page):**
Added new '📈 Position Scaling' section with:
- Enable/disable toggle (defaults to OFF)
- Min quality score slider (60-90, default 75)
- Min profit to scale (0-2%, default 0.4%)
- Scale size percent (10-100%, default 50%)
- Max position multiplier (1-3x, default 2.0x)
- Min ADX increase (0-15, default 5)
- Max price position for scale (50-90%, default 70%)
**Visual Feedback:**
- Purple-themed section with warning banner
- Real-time risk calculator showing:
* Original position size (SOL example)
* Scale addition amount
* Total after 1 scale
* Maximum possible position size
- Dynamic descriptions explain each parameter
- Warning: 'DISABLED by default' with red indicator
**API Updates:**
Extended /api/settings GET/POST to handle 7 new fields:
- ENABLE_POSITION_SCALING
- MIN_SCALE_QUALITY_SCORE
- MIN_PROFIT_FOR_SCALE
- MAX_SCALE_MULTIPLIER
- SCALE_SIZE_PERCENT
- MIN_ADX_INCREASE
- MAX_PRICE_POSITION_FOR_SCALE
**User Flow:**
1. Navigate to http://localhost:3001/settings
2. Scroll to '📈 Position Scaling' section
3. Toggle 'Enable Position Scaling' to 1
4. Adjust thresholds (defaults are conservative)
5. See live calculation of scaling impact
6. Click 'Save Settings'
7. Click 'Restart Bot' to apply
**Safety:**
- Feature OFF by default (requires explicit opt-in)
- Warning banner explains scaling behavior
- Risk calculator shows maximum exposure
- Conservative defaults prevent aggressive scaling
- All parameters adjustable via sliders
**Example:**
With defaults (SOL $210×10x = $2100):
- Scale adds: $1050 (50% of $2100)
- Total after 1 scale: $3150
- Max position (2x): $4200
User can now enable and configure position scaling without touching .env file!
**Feature: Position Scaling**
Allows adding to existing profitable positions when high-quality signals confirm trend strength.
**Configuration (config/trading.ts):**
- enablePositionScaling: false (disabled by default - enable after testing)
- minScaleQualityScore: 75 (higher bar than initial 60)
- minProfitForScale: 0.4% (must be at/past TP1)
- maxScaleMultiplier: 2.0 (max 200% of original size)
- scaleSizePercent: 50% (add 50% of original position)
- minAdxIncrease: 5 (ADX must strengthen)
- maxPricePositionForScale: 70% (don't chase resistance)
**Validation Logic (check-risk endpoint):**
Same-direction signal triggers scaling check if enabled:
1. Quality score ≥75 (stronger than initial entry)
2. Position profitable ≥0.4% (at/past TP1)
3. ADX increased ≥5 points (trend strengthening)
4. Price position <70% (not near resistance)
5. Total size <2x original (risk management)
6. Returns 'allowed: true, reason: Position scaling' if all pass
**Execution (execute endpoint):**
- Opens additional position at scale size (50% of original)
- Updates ActiveTrade: timesScaled, totalScaleAdded, currentSize
- Tracks originalAdx from first entry for comparison
- Returns 'action: scaled' with scale details
**ActiveTrade Interface:**
Added fields:
- originalAdx?: number (for scaling validation)
- timesScaled?: number (track scaling count)
- totalScaleAdded?: number (total USD added)
**Example Scenario:**
1. LONG SOL at $176 (quality: 45, ADX: 13.4) - weak but entered
2. Price hits $176.70 (+0.4%) - at TP1
3. New LONG signal (quality: 78, ADX: 19) - strong confirmation
4. Scaling validation: ✅ Quality 78 ✅ Profit +0.4% ✅ ADX +5.6 ✅ Price 68%
5. Adds 50% more position at $176.70
6. Total position: 150% of original size
**Conservative Design:**
- Disabled by default (requires manual enabling)
- Only scales INTO profitable positions (never averaging down)
- Requires significant quality improvement (75 vs 60)
- Requires trend confirmation (ADX increase)
- Hard cap at 2x original size
- Won't chase near resistance levels
**Next Steps:**
1. Enable in settings: ENABLE_POSITION_SCALING=true
2. Test with small positions first
3. Monitor data: do scaled positions outperform?
4. Adjust thresholds based on results
**Safety:**
- All existing duplicate prevention logic intact
- Flip logic unchanged (still requires quality check)
- Position Manager tracks scaling state
- Can be toggled on/off without code changes
**Problem:**
- Signal flips (SHORT→LONG or LONG→SHORT) were auto-approved
- Bypassed signal quality scoring, cooldown, drawdown checks
- User wanted flips ONLY if new signal has strong quality (score ≥60)
**Solution:**
- Removed early return for opposite-direction signals in check-risk
- Flips now go through FULL validation: quality score, cooldown, limits
- Execute endpoint still handles flip logic (close opposite + open new)
**New Flow:**
1. n8n sends flip signal → check-risk endpoint
2. Detects potential flip, logs 'checking quality score'
3. Continues to quality checks (not early return)
4. If score ≥60 AND all checks pass → execute handles flip
5. If score <60 → BLOCKS flip with 'Signal quality too low'
**Result:**
Flips now require signal strength, not just direction change
**Root Causes:**
1. Auto-flip logic could create phantom trades if close failed
2. Position size mismatches (0.01 SOL vs 11.92 SOL expected) not caught
3. Multiple trades for same symbol+direction in database
**Preventive Measures:**
1. **Startup Validation (lib/startup/init-position-manager.ts)**
- Validates all open trades against Drift positions on startup
- Auto-closes phantom trades with <50% expected size
- Logs size mismatches for manual review
- Prevents Position Manager from tracking ghost positions
2. **Duplicate Position Prevention (app/api/trading/execute/route.ts)**
- Blocks opening same-direction position on same symbol
- Returns 400 error if duplicate detected
- Only allows auto-flip (opposite direction close + open)
3. **Runtime Phantom Detection (lib/trading/position-manager.ts)**
- Checks position size every 2s monitoring cycle
- Auto-closes if size ratio <50% (extreme mismatch)
- Logs as 'manual' exit with AUTO_CLEANUP tx
- Removes from monitoring immediately
4. **Quality Score Fix (app/api/trading/check-risk/route.ts)**
- Hardcoded minScore=60 (removed non-existent config reference)
**Prevention Summary:**
- ✅ Startup validation catches historical phantoms
- ✅ Duplicate check prevents new phantoms
- ✅ Runtime detection catches size mismatches <30s after they occur
- ✅ All three layers work together for defense-in-depth
Issue: User had LONG (phantom) + SHORT (undersized 0.01 SOL vs 11.92 expected)
Fix: Both detected and closed, bot now clean with 0 active trades
- Add SymbolSettings interface with enabled/positionSize/leverage fields
- Implement per-symbol ENV variables (SOLANA_*, ETHEREUM_*)
- Add SOL and ETH sections to settings UI with enable/disable toggles
- Add symbol-specific test buttons (SOL LONG/SHORT, ETH LONG/SHORT)
- Update execute and test endpoints to check symbol enabled status
- Add real-time risk/reward calculator per symbol
- Rename 'Position Sizing' to 'Global Fallback' for clarity
- Fix position manager P&L calculation for externally closed positions
- Fix zero P&L bug affecting 12 historical trades
- Add SQL scripts for recalculating historical P&L data
- Move archive TypeScript files to .archive to fix build
Defaults:
- SOL: 10 base × 10x leverage = 100 notional (profit trading)
- ETH: base × 1x leverage = notional (data collection)
- Global: 10 × 10x for BTC and other symbols
Configuration priority: Per-symbol ENV > Market config > Global ENV > Defaults
CRITICAL: Cooldown was global across ALL symbols, causing missed opportunities
Example: ETH trade at 10:00 blocked SOL trade at 10:04 (5min cooldown)
Changes:
- Added getLastTradeTimeForSymbol() function to query last trade per symbol
- Updated check-risk endpoint to use symbol-specific cooldown
- Each coin (SOL/ETH/BTC) now has independent cooldown timer
- Cooldown message shows symbol: 'Must wait X min before next SOL-PERP trade'
Result: Can trade ETH and SOL simultaneously without interference
Example: ETH LONG at 10:00, SOL SHORT at 10:01 = both allowed
- Lower ATR threshold from 0.6% to 0.15% (allows low volatility breakouts)
- Increase volume bonus: +15 for very strong volume (1.5x+), was +10 for 1.2x+
- Add volume breakout logic: High volume (1.4x+) at 95%+ range gets +5 instead of -15 penalty
- Add volume compensation: +10 bonus when volume >1.8x and ATR <0.6%
- Example: SOL signal with 0.18% ATR, 1.74x volume at 95.6% range now scores 70/100 (PASS) instead of 25/100 (BLOCK)
- This signal moved +0.97% and would have hit TP1 (+1.5%) - proves quality scoring was too conservative
- Changes apply globally to all symbols (SOL, ETH, BTC) using same scoring algorithm
- Add qualityScore to ExecuteTradeResponse interface and response object
- Update analytics page to always show Signal Quality card (N/A if unavailable)
- Fix n8n workflow to pass context metrics and qualityScore to execute endpoint
- Fix timezone in Telegram notifications (Europe/Berlin)
- Fix symbol normalization in /api/trading/close endpoint
- Update Drift ETH-PERP minimum order size (0.002 ETH not 0.01)
- Add transaction confirmation to closePosition() to prevent phantom closes
- Add 30-second grace period for new trades in Position Manager
- Fix execution order: database save before Position Manager.addTrade()
- Update copilot instructions with transaction confirmation pattern
- New button in analytics page to clear orphaned trades
- API endpoint /api/trading/clear-manual-closes
- Intelligently checks Drift positions before deleting
- Only removes trades with no matching position or mismatched entry price
- Safe operation: keeps trades on error (false positives better than deletions)
- User-friendly confirmation dialog
- Added minQualityScore to TradingConfig (default: 60)
- Updated settings UI with slider control (0-100, step 5)
- Updated check-risk endpoint to use config value
- Made scoreSignalQuality function accept minScore parameter
- Updated API to read/write MIN_QUALITY_SCORE env variable
- Allows users to adjust quality threshold from settings page
- Extended MarketConfig with optional positionSize and leverage fields
- Configured ETH-PERP at @ 1x leverage for minimal-risk data collection
- Created getPositionSizeForSymbol() helper function in config/trading.ts
- Integrated symbol-specific sizing into execute endpoint
- Added comprehensive guide in docs/guides/SYMBOL_SPECIFIC_SIZING.md
Purpose: Enable ETH trading for faster signal quality data collection
while preserving SOL's profit-generation sizing (0 @ 10x)
Next: Create ETH alert in TradingView and restart bot
- Add signalQualityScore field to Trade model (0-100)
- Calculate quality score in execute endpoint using same logic as check-risk
- Save score with every trade for correlation analysis
- Create database migration for new field
- Enables future analysis: score vs win rate, P&L, etc.
This allows data-driven decisions on dynamic position sizing
- Add getLastTrade() function to database service
- Create /api/analytics/last-trade endpoint
- Display last trade with full details on analytics page
- Show entry/exit prices, P&L, position size, targets
- Visual indicators for trade direction and exit reason
- Helps quickly diagnose where trades went (TP1, TP2, or SL)
- Created /api/analytics/tp-sl-optimization endpoint
- Analyzes historical trades using MAE/MFE data
- Calculates optimal TP1/TP2/SL levels based on percentiles
- Provides win rate, profit factor, and hit rate analysis
- Shows money left on table (MFE - realized P&L)
- Projects impact of optimal levels on future performance
Analytics calculated:
- MAE analysis: avg, median, percentiles, worst
- MFE analysis: avg, median, percentiles, best
- Current level performance: TP1/TP2/SL hit rates
- Optimal recommendations: TP1=50% of avg MFE, TP2=80%, SL=70% of avg MAE
- Projected improvements: win rate change, profit factor, total P&L
Requires 10+ closed trades with MAE/MFE data to generate recommendations
Test script: scripts/test-analytics.sh
Next: Phase 4 (visual dashboard) or wait for trades with MAE/MFE data
- Added getFundingRate() method to DriftService
- Capture expectedEntryPrice from oracle before order execution
- Capture fundingRateAtEntry from Drift Protocol
- Save market context fields to database (expectedEntryPrice, fundingRateAtEntry)
- Calculate entry slippage percentage in createTrade()
- Fixed template literal syntax errors in execute endpoint
Database fields populated:
- expectedEntryPrice: Oracle price before order
- entrySlippagePct: Calculated from entrySlippage
- fundingRateAtEntry: Current funding rate from Drift
Next: Phase 3 (analytics API) or test market context on next trade
