- Updated .github/copilot-instructions.md key constraints and signal quality system description
- Updated config/trading.ts minimum score from 60 to 81 with v8 performance rationale
- Updated SIGNAL_QUALITY_SETUP_GUIDE.md intro to reflect 81 threshold
- Updated SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md current system section
- Updated BLOCKED_SIGNALS_TRACKING.md quality score requirements
Context: After v8 Money Line indicator deployed with 0.6% flip threshold,
system achieving 66.7% win rate with average quality score 94.2. Raised
minimum threshold from 60 to 81 to maintain exceptional selectivity.
Current v8 stats: 6 trades, 4 wins, $649.32 profit, 94.2 avg quality
Account growth: $540 → $1,134.92 (110% gain in 2-3 days)
**ISSUE:** User operates at 100% capital allocation - no room for 1.2x sizing
- 1.2x would require 120% of capital (mathematically impossible)
- User: 'thats not gonna work. we are already using 100% of our portfolio'
**FIX:** Changed from 1.2x to 1.0x (same size as original trade)
- Focus on capturing reversal, not sizing bigger
- Maintains aggressive 15x leverage
- Example: Original ,350 → Revenge ,350 (not 0,020)
**FILES CHANGED:**
- lib/trading/stop-hunt-tracker.ts: sizingMultiplier 1.2 → 1.0
- Telegram notification: Updated to show 'same as original'
- Documentation: Updated all references to 1.0x strategy
**DEPLOYED:** Nov 20, 2025 ~20:30 CET
**BUILD TIME:** 71.8s, compiled successfully
**STATUS:** Container running stable, stop hunt tracker operational
- Updated section header: Nov 16, 2025 → Nov 16, 2025 - Enhanced Nov 20, 2025
- Added TP1 partial close notification to implemented features list
- Enhanced notification format example with runner split
- Added Key Features section explaining immediate TP1 feedback
- Updated commits list: b1ca454 (Nov 16) + 79e7ffe (Nov 20)
- Clarified separate notifications for TP1 and runner closures
**CRITICAL: Documentation is now a NON-NEGOTIABLE requirement**
- Added ironclad rule: Every git commit MUST update copilot-instructions.md
- NO EXCEPTIONS - this is a real money trading system
- Undocumented fixes = forgotten fixes = financial losses
- Future AI agents need complete context for data integrity
**What must be documented:**
- Bug fixes → Common Pitfalls (symptom, root cause, fix, lesson)
- New features → Architecture Overview, Critical Components
- Database changes → Important fields, filtering requirements
- Config changes → Configuration System section
- Breaking changes → When Making Changes section
**Examples of proper documentation:**
- Common Pitfall #56: Ghost orders (a3a6222)
- Common Pitfall #57: P&L inaccuracy (8e600c8)
- Common Pitfall #55: BlockedSignalTracker (6b00303)
**This is not optional - documentation is part of 'done'**
- Documented Nov 20, 2025 fix for 36% P&L calculation errors
- External closure handler was using monitoring loop's stale currentPrice
- Real incident: Database -$101.68 vs Drift -$138.35 actual
- Fix: Query Drift's userAccount.perpPositions[].settledPnl directly
- Fallback to calculation if Drift query fails
- Updated #56 commit reference from [pending] to a3a6222
- Both fixes deployed Nov 20, 2025 15:25 CET
- Added order cancellation to Position Manager's external closure handler
- When on-chain SL/TP orders close position, remaining orders now cancelled automatically
- Prevents ghost orders from triggering unintended positions
- Real incident: Nov 20 SHORT stop-out left 32 ghost orders on Drift
- Risk: Ghost TP1 at $140.66 could fill later, creating unwanted LONG position
- Fix: Import cancelAllOrders() and call after trade removed from monitoring
- Non-blocking: Logs errors but doesn't fail trade closure if cancellation fails
- Files: lib/trading/position-manager.ts (external closure handler ~line 920)
- Documented as Common Pitfall #56
- Documented Nov 20, 2025 fix for multi-timeframe data collection
- Tracker was using Pyth cache (empty) instead of Drift oracle prices
- Result: 30+ hours with no price tracking, all priceAfter* fields NULL
- Fix: Changed to initializeDriftService() + getOraclePrice()
- Now working: Price tracking every 5min, analysisComplete transitions, TP/SL detection
- Impact: Multi-timeframe data collection now operational for Phase 1 analysis
Documented critical bug where runner trailing stop never activated after TP1:
- Symptom: Runner exposed to full reversal with static SL
- Real incident: 24 profit at risk, user forced manual close
- Root cause: External closure detected before TP2 price check
- Fix: Three-part solution with TP2 pre-check, diagnostics, exit reason detection
- Impact: Runner now fully autonomous with trailing protection
Also renumbered subsequent pitfalls 43→45, 44→46, etc. for consistency.
Commit: 55582a4 "critical: Fix runner trailing stop protection after TP1"
Documented the stats API bug where recalculating P&L from entry/exit
prices doesn't work for TP1+runner partial closes.
Issue:
- Stats showed -$26.10 when actual was +$46.97
- Recalculation used average exit price × full size
- Doesn't account for different TP1 vs runner exit prices
Fix:
- Use database realizedPnL field directly
- Database values corrected to match Drift UI TP1+runner sums
- Each trade shows as 2 lines in Drift (TP1 + runner)
Commits referenced:
- cd6f590: Corrected database P&L values
- d8b0307: Fixed stats API calculation
Lesson: Partial closes require storing combined P&L, can't
recalculate from average exit price.
Corrected misleading statement that implied 5.1x trading gains.
Reality: 46 total invested (06 + 40 deposits), 40 current = - trading P&L
Changes:
- Removed '5.1x in 8 days!' claim (was deposit-driven, not trading)
- Added 'Total Invested' line showing 46 actual capital input
- Added 'Trading P&L: -' to show early testing results
- Clarified losses from v6/v7 indicator testing before v8 optimization
Honesty matters - deposits ≠ profits. System is in proof phase.
- Multi-Timeframe Price Tracking System section (Nov 19, 2025)
- Purpose and architecture overview
- BlockedSignalTracker background job details
- Database schema with price tracking fields
- API endpoints and integration points
- How it works (step-by-step flow)
- Analysis queries for cross-timeframe comparison
- Decision-making criteria for timeframe optimization
- Current status: 2 signals tracked, pending 4H/Daily alerts
- Docker Maintenance section (Item #13 in When Making Changes)
- Cleanup commands: image prune, builder prune, volume prune
- When to run: After builds, weekly, on disk warnings
- Space savings: 2-5GB images, 40-50GB cache, 0.5-1GB volumes
- Safety guidelines: What to delete vs keep
- Why critical: Prevent disk full from build cache accumulation
User requested: 'document in the copilot instructions with the latest updates'
Both sections provide complete reference for future AI agents
Two P&L corrections for v8 trades:
1. First losing trade: Updated from -7.88 to -9.93 (matches Drift UI)
2. Phantom trade bug: Updated from /bin/bash.00 to 4.19 (TP1 + runner combined)
Corrected v8 stats:
- 5 trades, 80% win rate
- Total P&L: 1.06 (was 6.87 before corrections)
- Average: 4.21 per trade
- System recovered all previous losses and turned profitable
Related: Common Pitfall #49 (P&L compounding) and #53 (phantom detection)
caused the /bin/bash.00 entry. Database now reflects actual Drift results.
Documents the TP1 detection bug where on-chain limit orders fill faster
than Position Manager monitoring loop can detect. When both TP1 (75%)
and runner (25%) close before next monitoring cycle, PM doesn't know
TP1 filled first, marks entire closure as 'SL' instead of 'TP1'.
Fix uses profit percentage thresholds instead of state flags:
- >1.2%: TP2 range
- 0.3-1.2%: TP1 range
- <0.3%: SL range
Simpler and more reliable than tracking order fill state.
Common Pitfall #50 updated with resolution details:
- Bug fixed: Removed previouslyRealized from external closure calculations
- Database corrected: 3 v8 trades updated with accurate P&L values
- System operational: New trade executed successfully at 08:40 CET
- Analytics baseline established: -$8.74 total P&L, 66.7% WR, 3 trades
Historical data gap (~3 trades) explains difference between
analytics (-$8.74) and Drift UI (~-$52). All future trades
will track accurately with fixed calculation.
Ready for Phase 1: Collect 50+ v8 trades for statistical validation.
- Database shows only 3 v8 trades when Drift UI shows 6 trades
- One trade has 10x inflated P&L (81 vs 9 expected)
- Bot receiving ZERO API requests after 06:51 restart despite n8n executions succeeding
- Real v8 performance: ~2 LOSS (from Drift), database shows 20 profit (WRONG)
- Two issues: P&L compounding bug still active + n8n→bot connection broken
- Need to verify n8n workflow endpoints and fix external closure P&L calculation
- Common Pitfall #50 added to copilot-instructions.md
- Updated copilot instructions with data collection system overview
- Documents 5min execute vs 15min/1H/4H/Daily collect pattern
- Explains zero-risk parallel data collection approach
- Notes SQL analysis capability for timeframe optimization
- References implementation location in execute endpoint
- Part of v8 indicator testing and optimization strategy
Documents the critical P&L compounding bug fixed in commit 6156c0f where
trade.realizedPnL mutation during external closure detection caused 15-20x
inflation of actual profit/loss values.
Includes:
- Root cause: Mutating trade.realizedPnL in monitoring loop
- Real incident: $6 actual profit → $92.46 in database
- Bug mechanism with code examples
- Why previous fixes (Common Pitfalls #27, #48) didn't prevent this
- Fix: Don't mutate shared state during calculations
- Related to other P&L compounding variants for cross-reference
- Document ATR × multiplier system (2.0/4.0/3.0 for TP1/TP2/SL)
- Add calculation formula and example with SOL at $140
- Document safety bounds (MIN/MAX percentages)
- Include data-driven ATR values (SOL median 0.43 from 162 trades)
- Document ENV configuration variables
- Add regime-agnostic benefits explanation
- Update Exit Strategy section with ATR-based details
- Update Telegram manual trade presets with accurate ATR
- Add TradingView integration requirements
- Update 'When Making Changes' with ATR modification guidance
- Explain performance analysis and expected improvements
Why: Major system upgrade (Nov 17, 2025) requires complete documentation
for future AI agents and developers. ATR-based targets solve bull/bear
optimization bias by adapting to actual market volatility.
Problem:
- Close transaction confirmed but Drift state takes 5-10s to propagate
- Position Manager returned needsVerification=true to keep monitoring
- BUT: Monitoring loop detected position as 'externally closed' EVERY 2 seconds
- Each detection called handleExternalClosure() and added P&L to database
- Result: .66 actual profit → 73.36 in database (20x compounding)
- Logs showed: $112.96 → $117.62 → $122.28 → ... → $173.36 (14+ updates)
Root Cause:
- Common Pitfall #47 fix introduced needsVerification flag to wait for propagation
- But NO flag to prevent external closure detection during wait period
- Monitoring loop thought position was 'closed externally' on every cycle
- Rate limiting (429 errors) made it worse by extending wait time
Fix (closingInProgress flag):
1. Added closingInProgress boolean to ActiveTrade interface
2. Set flag=true when needsVerification returned (close confirmed, waiting)
3. Skip external closure detection entirely while flag=true
4. Timeout after 60s if stuck (abnormal case - allows cleanup)
Impact:
- Every close with verification delay (most closes) had 10-20x P&L inflation
- This is variant of Common Pitfall #27 but during verification, not external closure
- Rate limited closes were hit hardest (longer wait = more compounding cycles)
Files:
- lib/trading/position-manager.ts: Added closingInProgress flag + skip logic
Incident: Nov 16, 11:50 CET - SHORT 41.64→40.08 showed 73.36 vs .66 real
Documented: Common Pitfall #48
- Extended verification checklist for sessions with multiple related fixes
- Added requirement to verify container newer than ALL commits
- Included example from Nov 16 session (3 fixes deployed together)
- Added bash commands for complete deployment verification
- Emphasized that ALL fixes must be deployed, not just latest commit
- Updated Common Pitfall #47 with deployment verification commands
- Prevents declaring fixes 'working' when only some are deployed
- Added comprehensive documentation for close verification gap bug
- Real incident: 6 hours unmonitored exposure after close confirmation
- Root cause: Transaction confirmed ≠ Drift state propagated (5-10s delay)
- Fix: 5s wait + verification + needsVerification flag for Position Manager
- Prevents premature database 'closed' marking while position still open
- TypeScript interface updated: ClosePositionResult.needsVerification
- Deployed: Nov 16, 2025 09:28:20 CET
- Commits: c607a66 (logic), b23dde0 (interface)
- CRITICAL BUG: Drift SDK's position.entryPrice RECALCULATES after partial closes
- After TP1, Drift returns COST BASIS of remaining position, NOT original entry
- Example: SHORT @ 38.52 → TP1 @ 70% → Drift shows entry 40.01 (runner's basis)
- Result: Breakeven SL set .50 ABOVE actual entry = guaranteed loss if triggered
Fix:
- Always use database trade.entryPrice for breakeven calculations
- Drift's position.entryPrice = current state (runner cost basis)
- Database entryPrice = original entry (authoritative for breakeven)
- Added logging to show both values for verification
Impact:
- Every TP1 → breakeven transition was using WRONG price
- Locking in losses instead of true breakeven protection
- Financial loss bug affecting every trade with TP1
Files:
- lib/trading/position-manager.ts: Line 513 - use trade.entryPrice not position.entryPrice
- .github/copilot-instructions.md: Added Common Pitfall #43, deprecated old #44
Incident: Nov 16, 02:47 CET - SHORT entry 38.52, breakeven SL set at 40.01
Position closed by ghost detection before SL could trigger (lucky)
Documents the fix for InsufficientCollateral errors when using 100% position
sizing despite having correct account leverage.
Issue: Drift's margin calculation includes fees, slippage buffers, and rounding.
Using exact 100% of collateral leaves no room, causing $0.03-0.10 shortages.
Example: $85.55 collateral
- Bot tries: 100% = $85.55
- Margin needed: $85.58 (includes fees)
- Result: Rejected
Solution: Automatically apply 99% safety buffer when configured at 100%.
Result: $85.55 × 99% = $84.69 (leaves $0.86 safety margin)
Includes real incident details, code implementation, and math proof.
User's Drift UI screenshot proved account leverage WAS set correctly to 15x.
Related commit: 7129cbf
Documents the fix for breakeven SL using database entry price instead of
Drift's actual on-chain entry price.
Issue: Database stored $139.18 but Drift actual was $139.07, causing
'breakeven' SL to lock in $0.11 loss per token for SHORT positions.
Solution: Query position.entryPrice from Drift SDK (calculated from
on-chain quoteAssetAmount / baseAssetAmount) when setting breakeven.
Includes real incident details, code comparison, and relationship to
Common Pitfall #33 (orphaned position restoration).
Related commit: 528a0f4
Added comprehensive documentation per user request: 'is everything documentet
and pushed to git? nothing we forget to mention which is crucial for other
developers/agents?'
Updates:
- Common Pitfall #40: Added refactoring note (removed time-based Layer 1)
- User feedback: Time-based cleanup too aggressive for long-running positions
- Now 100% Drift API-based ghost detection (commit 9db5f85)
- Common Pitfall #41: Telegram notifications for position closures (NEW)
- Implemented lib/notifications/telegram.ts with sendPositionClosedNotification()
- Direct Telegram API calls (no n8n dependency)
- Includes P&L, prices, hold time, MAE/MFE, exit reason with emojis
- Integrated into Position Manager (commit b1ca454)
- Common Pitfall #42: Telegram bot DNS retry logic (NEW)
- Python urllib3 transient DNS failures (same as Node.js)
- Added retry_request() wrapper with exponential backoff
- 3 attempts (2s → 4s → 8s), matches Node.js retryOperation pattern
- Applied to /status and manual trade execution (commit bdf1be1)
- Common Pitfall #43: Drift account leverage UI requirement (NEW)
- Account leverage is on-chain setting, CANNOT be changed via API
- Must use Drift UI settings page
- Confusion: Order leverage dropdown ≠ account leverage setting
- Temporary workaround: Reduced position size to 6% for 1x leverage
- User action needed: Set account leverage to 15x in Drift UI
- Telegram Notifications section: Added to main architecture documentation
- Purpose, format, configuration, integration points
- Reference implementation details
Session focus: Ghost detection refactoring, Telegram notifications, DNS retry,
Drift leverage diagnostics. User emphasized knowledge preservation for future
developers and AI agents working on this codebase.
CRITICAL FIX: Settings UI was completely broken with EACCES permission denied
Problem:
- .env file on host owned by root:root
- Docker mounts .env as volume, retains host ownership
- Container runs as nextjs user (UID 1001) for security
- Settings API attempts fs.writeFileSync() → permission denied
- Users could NOT adjust position size, leverage, TP/SL, or any config
User escalation: "thats a major flaw. THIS NEEDS TO WORK."
Solution:
- Changed .env ownership on HOST to UID 1001 (nextjs user)
- chown 1001:1001 /home/icke/traderv4/.env
- Restarted container to pick up new permissions
- .env now writable by nextjs user inside container
Verified: Settings UI now saves successfully
Documented as Common Pitfall #39 with:
- Symptom, root cause, and impact
- Why docker exec chown fails (mounted files)
- Correct fix with UID matching
- Alternative solutions and tradeoffs
- Lesson about Docker volume mount ownership
Files changed:
- .github/copilot-instructions.md (added Pitfall #39)
- .env (ownership changed from root:root to 1001:1001)
Pitfall #34 (Runner SL gap):
- Updated with live test results from Nov 15, 22:03 CET
- Runner SL triggered successfully with +.94 profit (validates fix works)
- Documented rate limit storm during close (20+ attempts, eventually succeeded)
- Proves software protection works even without on-chain orders
- This was the CRITICAL fix that prevented hours of unprotected exposure
Pitfall #38 (Analytics showing wrong size - NEW):
- Dashboard displayed original position size (2.54) instead of runner (2.59)
- Root cause: API returned positionSizeUSD, not currentSize from Position Manager state
- Fixed by checking configSnapshot.positionManagerState.currentSize for open positions
- API-only change, no container restart needed
- Provides accurate exposure visibility on dashboard
Both issues discovered and fixed during today's live testing session.
Added comprehensive documentation of the runner stop loss protection gap:
- Root cause analysis (SL check only before TP1)
- Bug sequence and impact details
- Code fix with examples
- Compounding factors (small runner + no on-chain orders)
- Lesson learned for future risk management code
CRITICAL BUG: Missing retry wrapper caused rate limit storm
Real Incident (Nov 15, 16:49 CET):
- Trade cmi0il8l30000r607l8aec701 triggered close attempt
- closePosition() had NO retryWithBackoff() wrapper
- Failed with 429 → Position Manager retried EVERY 2 SECONDS
- 100+ close attempts exhausted Helius rate limit
- On-chain TP2 filled during storm
- External closure detected 8 times: $0.14 → $0.51 (compounding bug)
Why This Was Missed:
- placeExitOrders() got retry wrapper on Nov 14
- openPosition() still has no wrapper (less critical - runs once)
- closePosition() overlooked - MOST CRITICAL because runs in monitoring loop
- Position Manager executeExit() catches 429 and returns early
- But monitoring continues, retries close every 2s = infinite loop
The Fix:
- Wrapped closePosition() placePerpOrder() with retryWithBackoff()
- 8s base delay, 3 max retries (same as placeExitOrders)
- Reduces RPC load by 30-50x during close operations
- Container deployed 18:05 CET Nov 15
Impact: Prevents rate limit exhaustion + duplicate external closure updates
Files: .github/copilot-instructions.md (added Common Pitfall #36)
- Documented bug where phantom auto-closure sets status='phantom' but left exitReason=NULL
- Startup validator only checks exitReason, not status field
- Ghost positions created false runner stop loss alerts (232% size mismatch)
- Fix: MUST set exitReason when closing phantom trades
- Manual cleanup: UPDATE Trade SET exitReason='manual' WHERE status='phantom' AND exitReason IS NULL
- Verified: System now shows 'Found 0 open trades' after cleanup
CRITICAL BUG DOCUMENTATION: Runner had ZERO stop loss protection between TP1-TP2
Context:
- User reported: 'runner close did not work. still open and the price is above 141,98'
- Investigation revealed Position Manager only checked SL before TP1 OR after TP2
- Runner between TP1-TP2 had NO stop loss checks = hours of unlimited loss exposure
Bug Impact:
- SHORT at $141.317, TP1 closed 70% at $140.942, runner had SL at $140.89
- Price rose to $141.98 (way above SL) → NO PROTECTION → Position stayed open
- Potential unlimited loss on 25-30% runner position
Fix Verification:
- After fix deployed: Runner closed at $141.133 with +$0.59 profit
- Database shows exitReason='SL', proving runner stop loss triggered correctly
- Log: '🔴 RUNNER STOP LOSS: SOL-PERP at 0.3% (profit lock triggered)'
Lesson: Every conditional branch in risk management MUST have explicit SL checks
Files: .github/copilot-instructions.md (added Common Pitfall #34)
Major Fix Summary:
- Position Manager was tracking wrong entry price after orphaned position restoration
- Used stale database value ($141.51) instead of Drift's actual entry ($141.31)
- 0.14% difference in stop loss placement - could mean profit vs loss difference
- Startup validation now queries Drift SDK for authoritative entry price
Impact: Critical for accurate P&L tracking and stop loss placement
Prevention: Always prefer on-chain data over cached DB values for trading params
Added to Common Pitfalls section with full bug sequence, fix code, and lessons learned.
- Added signalSource field documentation
- Emphasized CRITICAL exclusion from TradingView indicator analysis
- Reference to MANUAL_TRADE_FILTERING.md for SQL queries
- Manual Trading via Telegram section updated with contamination warning
Research findings:
- Alchemy Growth DOES support WebSocket subscriptions (up to 2,000 connections)
- All standard Solana RPC methods supported
- No documented Drift-Alchemy incompatibilities
- Rate limits enforced via CUPS (Compute Units Per Second)
Hypothesis for our failures:
- accountSubscribe 'errors' might be 429 rate limits, not 'method not found'
- Drift SDK may not handle Alchemy's rate limit pattern during init
- First trade works (subscriptions established) → subsequent fail (bad state)
Pragmatic decision:
- Helius works reliably NOW for production trading
- Theoretical investigation can wait until needed
- Future optimization possible with WebSocket-specific retry logic
This note preserves the research for future reference without changing
the current production recommendation (Helius only).
DEFINITIVE CONCLUSION:
- Alchemy 'breakthrough' at 14:25 was NOT sustainable
- First trade appeared perfect, subsequent trades consistently fail
- Multiple attempts with pure Alchemy config = same failures
- Helius is the ONLY reliable RPC provider for Drift SDK
Timeline documented:
- 14:01: Switched to Alchemy
- 14:25: First trade perfect (false breakthrough)
- 15:00-20:00: Hybrid/fallback attempts (all failed)
- 20:00: Pure Alchemy retry (still broke)
- 20:05: Helius final revert (works reliably)
User confirmations:
- 'SO IT WAS THE FUCKING RPC...' (initial discovery)
- 'after changing back the settings it started to act up again' (Alchemy breaks)
- 'telegram works again' (Helius works)
This is the complete story for future reference.
- Documented both Helius rate limit issue AND Alchemy WebSocket incompatibility
- Added user confirmation quote
- Explained why Helius is required (WebSocket subscriptions)
- Explained why Alchemy fails (no accountSubscribe support)
- This is the definitive RPC provider guidance for Drift Protocol
CATASTROPHIC BUG DISCOVERY (Nov 14, 2025):
- Helius free tier (10 req/sec) was the ROOT CAUSE of all Position Manager failures
- Switched to Alchemy (300M compute units/month) = INSTANT FIX
- System went from completely broken to perfectly functional in one change
Evidence:
BEFORE (Helius):
- 239 rate limit errors in 10 minutes
- Trades hit SL immediately after opening
- Duplicate close attempts
- Position Manager lost tracking
- Database save failures
- TP1/TP2 never triggered correctly
AFTER (Alchemy) - FIRST TRADE:
- ZERO rate limit errors
- Clean execution with 2s delays
- TP1 hit correctly at +0.4%
- 70% closed automatically
- Runner activated with trailing stop
- Position Manager tracking perfectly
- Currently up +0.77% on runner
Changes:
- Added CRITICAL RPC section to Architecture Overview
- Made RPC provider Common Pitfall #1 (most important)
- Documented symptoms, root cause, fix, and evidence
- Marked Nov 14, 2025 as the day EVERYTHING started working
This was the missing piece that caused weeks of debugging.
User quote: 'SO IT WAS THE FUCKING RPC THAT WAS CAUSING ALL THE ISSUES!!!!!!!!!!!!'
