Database changes:
- Added indicatorVersion field to Trade table
- Added indicatorVersion field to BlockedSignal table
- Tracks which Pine Script version (v5, v6, etc.) generated each signal
Pine Script changes:
- v6 now includes '| IND:v6' in alert messages
- Enables differentiation between v5 and v6 signals in database
Documentation:
- Created INDICATOR_VERSION_TRACKING.md with full implementation guide
- Includes n8n workflow update instructions
- Includes SQL analysis queries for v5 vs v6 comparison
- Includes rollback plan if needed
Next steps (manual):
1. Update n8n workflow Parse Signal Enhanced node to extract IND field
2. Update n8n HTTP requests to pass indicatorVersion
3. Update API endpoints to accept and save indicatorVersion
4. Rebuild Docker container
Benefits:
- Compare v5 vs v6 Pine Script effectiveness
- Track which version generated winning/losing trades
- Validate that v6 price position filter reduces blocked signals
- Data-driven decisions on Pine Script improvements
New v6 improvements:
- Fixed price position calculation: 100-bar range (was 20-bar)
- Added price position filter: prevents chasing extremes (85% max for longs, 15% min for shorts)
- Added volume filter: optional range check (0.7-3.0x average)
- Added RSI momentum filter: optional directional confirmation
- All new filters toggleable with sensible defaults
Key changes:
- Price position filter ENABLED by default (prevents flip-flop losses)
- Volume and RSI filters DISABLED by default (test incrementally)
- Aligns TradingView filtering with bot's 5-metric scoring system
- Reduces signals sent to bot that would be blocked anyway
Rationale:
Database analysis showed range extreme entries (9-94%) caused flip-flop losses.
V6 filters these at source instead of blocking in bot after webhook call.
Testing approach:
1. Phase 1: Price position filter only (test 5-10 signals)
2. Phase 2: Add volume filter if needed
3. Phase 3: Add RSI filter as last resort
Added SQL Analysis Queries section with:
- Phase 1 monitoring queries (count, score distribution, recent signals)
- Phase 2 comparison queries (blocked vs executed trades)
- Pattern analysis queries (range extremes, ADX distribution)
Benefits:
- AI agents have immediate access to standard queries
- Consistent analysis approach each time
- No need to context-switch to separate docs
- Quick reference for common investigations
Includes usage pattern guidance and reference to full docs.
- Added BlockedSignal to database models list
- Updated signalQualityVersion to v4 (current)
- Added blocked signals tracking functions to database section
- Updated check-risk endpoint description
- Added Signal Quality Optimization Roadmap reference
- Documented blocked signals analysis workflow
- Added reference to BLOCKED_SIGNALS_TRACKING.md
This ensures AI agents understand the new data collection system.
- Add BlockedSignal table with 25 fields for comprehensive signal analysis
- Track all blocked signals with metrics (ATR, ADX, RSI, volume, price position)
- Store quality scores, block reasons, and detailed breakdowns
- Include future fields for automated price analysis (priceAfter1/5/15/30Min)
- Restore signalQualityVersion field to Trade table
Database changes:
- New table: BlockedSignal with indexes on symbol, createdAt, score, blockReason
- Fixed schema drift from manual changes
API changes:
- Modified check-risk endpoint to save blocked signals automatically
- Fixed hasContextMetrics variable scope (moved to line 209)
- Save blocks for: quality score too low, cooldown period, hourly limit
- Use config.minSignalQualityScore instead of hardcoded 60
Database helpers:
- Added createBlockedSignal() function with try/catch safety
- Added getRecentBlockedSignals(limit) for queries
- Added getBlockedSignalsForAnalysis(olderThanMinutes) for automation
Documentation:
- Created BLOCKED_SIGNALS_TRACKING.md with SQL queries and analysis workflow
- Created SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md with 5-phase plan
- Documented data-first approach: collect 10-20 signals before optimization
Rationale:
Only 2 historical trades scored 60-64 (insufficient sample size for threshold decision).
Building data collection infrastructure before making premature optimizations.
Phase 1 (current): Collect blocked signals for 1-2 weeks
Phase 2 (next): Analyze patterns and make data-driven threshold decision
Phase 3-5 (future): Automation and ML optimization
BUG FOUND:
Line 558: tp2SizePercent: config.takeProfit2SizePercent || 100
When config.takeProfit2SizePercent = 0 (TP2-as-runner system), JavaScript's ||
operator treats 0 as falsy and falls back to 100, causing TP2 to close 100%
of remaining position instead of activating trailing stop.
IMPACT:
- On-chain orders placed correctly (line 481 uses ?? correctly)
- Position Manager reads from DB and expects TP2 to close position
- Result: User sees TWO take-profit orders instead of runner system
FIX:
Changed both tp1SizePercent and tp2SizePercent to use ?? operator:
- tp1SizePercent: config.takeProfit1SizePercent ?? 75
- tp2SizePercent: config.takeProfit2SizePercent ?? 0
This allows 0 value to be saved correctly for TP2-as-runner system.
VERIFICATION NEEDED:
Current open SHORT position in database has tp2SizePercent=100 from before
this fix. Next trade will use correct runner system.
- New /api/trading/sync-positions endpoint (no auth)
- Fetches actual Drift positions and compares with Position Manager
- Removes stale tracking, adds missing positions with calculated TP/SL
- Settings UI: Orange 'Sync Positions' button added
- CLI script: scripts/sync-positions.sh for terminal access
- Full documentation in docs/guides/POSITION_SYNC_GUIDE.md
- Quick reference: POSITION_SYNC_QUICK_REF.md
- Updated AI instructions with pitfall #23
Problem solved: Manual Telegram trades with partial fills can cause
Position Manager to lose tracking, leaving positions without software-
based stop loss protection. This feature restores dual-layer protection.
Note: Docker build not picking up route yet (cache issue), needs investigation
- Add SOLANA_USE_PERCENTAGE_SIZE and ETHEREUM_USE_PERCENTAGE_SIZE to TradingSettings interface
- Make SOL/ETH Position Size labels dynamic based on percentage mode
- Adjust max value (100 for %, 10000 for USD) based on mode
- Update descriptions to match mode (% of collateral vs fixed capital)
- Add usePercentageSize flag to SymbolSettings and TradingConfig
- Add calculateActualPositionSize() and getActualPositionSizeForSymbol() helpers
- Update execute and test endpoints to calculate position size from free collateral
- Add SOLANA_USE_PERCENTAGE_SIZE, ETHEREUM_USE_PERCENTAGE_SIZE, USE_PERCENTAGE_SIZE env vars
- Configure SOL to use 100% of portfolio (auto-adjusts to available balance)
- Fix TypeScript errors: replace fillNotionalUSD with actualSizeUSD
- Remove signalQualityVersion and fullyClosed references (not in interfaces)
- Add comprehensive documentation in PERCENTAGE_SIZING_FEATURE.md
Benefits:
- Prevents insufficient collateral errors by using available balance
- Auto-scales positions as account grows/shrinks
- Maintains risk proportional to capital
- Flexible per-symbol configuration (SOL percentage, ETH fixed)
- Updated Signal Quality System to reflect MIN_SIGNAL_QUALITY_SCORE is configurable (default: 65)
- Added critical pitfall #7: Never use hardcoded config values in endpoints
- Emphasized settings page can modify minSignalQualityScore dynamically
- Renumbered remaining pitfalls for clarity
PROBLEM ANALYSIS:
Signal that lost -$32: ADX 14.8, VOL 2.29x → scored 70-90 (PASSED)
Signal that won +3%: ADX 15.7, VOL 1.18x → scored 45-65 (got BLOCKED before fix)
Key insight: High volume during choppy conditions (ADX < 16) indicates
whipsaw/trap, not genuine breakout. Our volume bonus (+15 pts for >1.5x)
was rewarding flip-flop signals instead of real moves.
FIX:
Add anti-chop filter in volume scoring:
- If ADX < 16 AND volume > 1.5x → -15 points (whipsaw trap)
- Overrides the normal +15 bonus for high volume
- Protects against false signals during consolidation
IMPACT ON RECENT SIGNALS:
1. 00:40 SHORT (ADX 17.2, VOL 0.98): 55→75 ✅ Still passes
2. 00:55 LONG (ADX 15, VOL 0.47): 35→55 ❌ Still blocked (correct, weak vol)
3. 01:05 SHORT (ADX 14.8, VOL 2.29): 70→60 ⚠️ Now flagged as whipsaw trap
4. 01:10 LONG (ADX 15.7, VOL 1.18): 45→65 ✅ Catches the +3% runup
Result: Loser signal now barely passes (60) with warning flag,
winner signal passes cleanly (65). Better risk/reward profile.
- Fix external closure P&L using tp1Hit flag instead of currentSize
- Add direction change detection to prevent false TP1 on signal flips
- Signal flips now recorded with accurate P&L as 'manual' exits
- Add retry logic with exponential backoff for Solana RPC rate limits
- Create /api/trading/cancel-orders endpoint for manual cleanup
- Improves data integrity for win/loss statistics
- Change tp2SizePercent fallback from || 100 to ?? 0
- Allows 0 value to pass through (means 'activate trailing stop, don't close')
- Fixes bug where TP2 was closing 100% of remaining position
- Now correctly leaves 25% runner after TP1 closes 75%
- Applied to both execute and test endpoints
- Add market data cache service (5min expiry) for storing TradingView metrics
- Create /api/trading/market-data webhook endpoint for continuous data updates
- Add /api/analytics/reentry-check endpoint for validating manual trades
- Update execute endpoint to auto-cache metrics from incoming signals
- Enhance Telegram bot with pre-execution analytics validation
- Support --force flag to override analytics blocks
- Use fresh ADX/ATR/RSI data when available, fallback to historical
- Apply performance modifiers: -20 for losing streaks, +10 for winning
- Minimum re-entry score 55 (vs 60 for new signals)
- Fail-open design: proceeds if analytics unavailable
- Show data freshness and source in Telegram responses
- Add comprehensive setup guide in docs/guides/REENTRY_ANALYTICS_QUICKSTART.md
Phase 1 implementation for smart manual trade validation.
- Add ATR-based dynamic TP2 scaling from 0.7% to 3.0% based on volatility
- New config options: useAtrBasedTargets, atrMultiplierForTp2, minTp2Percent, maxTp2Percent
- Enhanced settings UI with ATR controls and updated risk calculator
- Fix external closure P&L calculation using unrealized P&L instead of volatile current price
- Update execute and test endpoints to use calculateDynamicTp2() function
- Maintain 25% runner system for capturing extended moves (4-5% targets)
- Add environment variables for ATR-based configuration
- Better P&L accuracy for manual position closures
- Fix P&L calculation in Position Manager to use actual entry vs exit price instead of SDK's potentially incorrect realizedPnL
- Calculate actual profit percentage and apply to closed position size for accurate dollar amounts
- Update database record for last trade from incorrect 6.58 to actual .66 P&L
- Update .github/copilot-instructions.md to reflect TP2-as-runner system changes
- Document 25% runner system (5x larger than old 5%) with ATR-based trailing
- Add critical P&L calculation pattern to common pitfalls section
- Mark Phase 5 complete in development roadmap
CHANGE: TP2 now activates trailing stop on full 25% remaining instead
of closing 80% and leaving 5% runner.
Benefits:
- 5x larger runner (25% vs 5%) = 25 vs 05 on 100 position
- Eliminates Drift minimum size issues completely
- Simplifies logic - no more canUseRunner() viability checks
- Better R:R on extended moves
New flow:
- TP1 (+0.4%): Close 75%, keep 25%
- TP2 (+0.7%): Skip close, activate trailing stop on full 25%
- Runner: 25% with ATR-based trailing (0.25-0.9%)
Config change: takeProfit2SizePercent: 80 → 0
Position Manager: Remove canUseRunner logic, activate trailing at TP2 hit
PROBLEM: Runner never activated because Drift force-closes positions below
minimum size. TP2 would close 80% leaving 5% runner (~$105), but Drift
automatically closed the entire position.
SOLUTION:
1. Created runner-calculator.ts with canUseRunner() to check if remaining
size would be above Drift minimums BEFORE executing TP2 close
2. If runner not viable: Skip TP2 close entirely, activate trailing stop
on full 25% remaining (from TP1)
3. If runner viable: Execute TP2 as normal, activate trailing on 5%
Benefits:
- Runner system will now actually work for viable position sizes
- Positions that are too small won't try to force-close below minimums
- Better logs showing why runner did/didn't activate
- Trailing stop works on larger % if runner not viable (better R:R)
Example: $2100 position → $525 after TP1 → $105 runner = VIABLE
$4 ETH position → $1 after TP1 → $0.20 runner = NOT VIABLE
Runner will trail with ATR-based dynamic % (0.25-0.9%) below peak price.
Corrects all historical SHORT trades affected by the P&L calculation bug.
Results:
- 44 SHORT trades corrected
- Total P&L improved from -$388.82 to -$72.98 (+$315.84 recovered)
- SHORT win rate revealed to be 60.5% (not 39% as incorrectly shown)
Script creates backup table before making changes and includes rollback instructions.
CRITICAL BUG FIX: SHORT positions were calculating P&L with inverted logic,
causing profits to be recorded as losses and vice versa.
Problem Example:
- SHORT at $156.58, exit at $154.66 (price dropped $1.92)
- Should be +~$25 profit
- Was recorded as -$499.23 LOSS
Root Cause:
Old formula: profitPercent = (exit - entry) / entry * (side === 'long' ? 1 : -1)
This multiplied the LONG formula by -1 for shorts, but then applied it to
full notional instead of properly accounting for direction.
Fix:
- LONG: priceDiff = (exit - entry) → profit when price rises
- SHORT: priceDiff = (entry - exit) → profit when price falls
- profitPercent = priceDiff / entry * 100
- Proper leverage calculation: realizedPnL = collateral * profitPercent * leverage
This fixes both dry-run and live close position calculations in lib/drift/orders.ts
Impact: All SHORT trades since bot launch have incorrect P&L in database.
Future trades will calculate correctly.
- Added signalQualityVersion field documentation (v1/v2/v3 tracking)
- Documented /api/analytics/version-comparison endpoint
- Added Prisma Decimal type handling pitfall (#18)
- Added signal quality version tracking section to Development Roadmap
- References SQL analysis file for version comparison queries
Enables AI agents to understand the version tracking system for
data-driven algorithm optimization.
- Changed numeric fields from typed as number to 'any' in raw query results
- Properly convert Prisma Decimal/BigInt types to JavaScript numbers
- Fixes TypeError: e.totalPnL.toFixed is not a function
- All numeric values (totalPnL, avgPnL, avgADX, etc.) now converted with Number()
Issue: Prisma returns Decimal objects from aggregation queries which don't have
toFixed() method. Frontend expects plain numbers for .toFixed(2) formatting.
- Added signalQualityVersion field to Trade model
- Tracks which scoring logic version was used for each trade
- v1: Original logic (price position < 5% threshold)
- v2: Added volume compensation for low ADX
- v3: CURRENT - Stricter logic requiring ADX > 18 for extreme positions (< 15%)
This enables future analysis to:
- Compare performance between logic versions
- Filter trades by scoring algorithm
- Data-driven improvements based on clean datasets
All new trades will be marked as v3. Old trades remain null/v1 for comparison.
- Shorts/longs at < 15% range require ADX > 18 AND volume > 1.2x
- OR RSI < 35 for shorts, RSI > 60 for longs
- Increased penalty from -10 to -15 when conditions not met
- Changed threshold from < 5% to < 15% to catch more edge cases
Test results:
- Big loser (01:35): ADX 16.1, price 9.3% → Score 60 (was 90) → BLOCKED
- Today's signal (10:05): ADX 17.3, price 0.9% → Score 55 (was 85) → BLOCKED
Rationale: False breakdowns in choppy ranges (ADX < 18) cause losses.
Tradeoff: May block some profitable breakdowns, but prevents chop losses.
- Allow shorts at range bottom (<5%) with volume >1.2x OR RSI <40
- Allow longs at range bottom with volume >1.2x OR RSI >60
- Reduce ADX penalty from -15 to -5 when strong volume (>1.2x) present
- Reduce price position penalties from -15 to -10 (less harsh)
- Volume compensation recognizes breakdowns start before ADX strengthens
Test case (blocked signal that would have profited):
- OLD: ATR 0.32, ADX 17.3, RSI 32.5, Vol 1.27x, Price 0.9% → Score 45 (blocked)
- NEW: Same metrics → Score 85 (executes)
Rationale: Breakdowns continue lower, volume confirms conviction, ADX lags price action
- Extended telegram_command_bot.py with MessageHandler for plain text messages
- Supports 'long sol/eth/btc' and 'short sol/eth/btc' syntax
- Calls /api/trading/execute directly with preset healthy metrics
- Increased timeout to 60s for on-chain transaction completion
- No changes to webhook flow or existing commands
PROBLEM:
- Trades with quality score 35 and 45 were executed (threshold: 60)
- Position opened without risk management after signal flips
- "Parse Signal" node didn't extract ATR/ADX/RSI/volumeRatio/pricePosition
- "Check Risk" node only sent symbol+direction, skipped quality validation
- "Execute Trade" node didn't forward metrics to backend
ROOT CAUSE:
n8n workflow had TWO paths:
1. NEW: Parse Signal Enhanced → Check Risk1 → Execute Trade1 (working)
2. OLD: Parse Signal → Check Risk → Execute Trade (broken)
Old path bypassed quality check because check-risk endpoint saw
hasContextMetrics=false and allowed trade without validation.
FIX:
1. Changed "Parse Signal" from 'set' to 'code' node with metric extraction
2. Updated "Check Risk" to send atr/adx/rsi/volumeRatio/pricePosition
3. Updated "Execute Trade" to forward all metrics to backend
IMPACT:
- All trades now validated against quality score threshold (60)
- Low-quality signals properly blocked before execution
- Prevents positions opening without proper risk management
Evidence from database showed 3 trades in 2 hours with scores <60:
- 10:00:31 SOL LONG - Score 35 (phantom detected)
- 09:55:30 SOL SHORT - Score 35 (executed)
- 09:35:14 SOL LONG - Score 45 (executed)
All three should have been blocked. Fix prevents future bypasses.
- Detect position size mismatches (>50% variance) after opening
- Save phantom trades to database with expectedSizeUSD, actualSizeUSD, phantomReason
- Return error from execute endpoint to prevent Position Manager tracking
- Add comprehensive documentation of phantom trade issue and solution
- Enable data collection for pattern analysis and future optimization
Fixes oracle price lag issue during volatile markets where transactions
confirm but positions don't actually open at expected size.
- Remove trade from Position Manager BEFORE closing Drift position (prevents race condition)
- Explicitly save closure to database with proper P&L calculation
- Mark flipped positions as 'manual' exit reason
- Increase delay from 1s to 2s for better on-chain confirmation
- Preserve MAE/MFE data in closure records
Fixes issue where SHORT signal would close LONG but not properly track the new SHORT position.
Database now correctly records both old position closure and new position opening.
- Position Manager was calculating P&L using tracked size instead of actual on-chain size
- Example: Tracked 100, actual 0.04 SOL () = -99.63% false loss instead of -0.32%
- Fixed external closure detection to use position.size * currentPrice as lastKnownSize
- Manually corrected phantom trade P&L from -092.25 to /bin/bash
- Total P&L corrected: -013.92 → +8.33 (accurate)
- Prevents all future phantom/mismatch trades from wildly incorrect P&L
Modified:
- lib/trading/position-manager.ts lines 421-445 (external closure P&L calculation)
