Commit Graph

71 Commits

Author SHA1 Message Date
mindesbunister
d28da02089 docs: Add withdrawal system fixes to copilot instructions
Added recent commit examples to git workflow documentation:
- ATA implementation for USDC withdrawals (c37a9a3)
- MIN_SIGNAL_QUALITY_SCORE variable name fix
- Withdrawal statistics accuracy fix (8d53c4b)

Shows proper commit message format with types (feat, fix, critical)
and detailed descriptions of changes.
2025-11-19 20:38:11 +01:00
mindesbunister
c42bf94c1f fix: Clarify 40 is from deposits, not trading profits
Corrected misleading statement that implied 5.1x trading gains.
Reality: 46 total invested (06 + 40 deposits), 40 current = - trading P&L

Changes:
- Removed '5.1x in 8 days!' claim (was deposit-driven, not trading)
- Added 'Total Invested' line showing 46 actual capital input
- Added 'Trading P&L: -' to show early testing results
- Clarified losses from v6/v7 indicator testing before v8 optimization

Honesty matters - deposits ≠ profits. System is in proof phase.
2025-11-19 17:42:38 +01:00
mindesbunister
a6794d00dc docs: Update roadmap with 40 current capital and latest system improvements
Financial Updates:
- Current capital: 7.55 → 40 USDC (with deposits)
- Starting capital: 06 (Nov 11) + 40 deposits
- Phase 1 target: ,500 (4.6x growth from current)
- Monthly return target: 20-30% → 15-20% (more achievable with larger base)
- Risk tolerance: EXTREME → HIGH (increased capital cushion)
- Notional position size: ~,463 → ~,100 (at 15x leverage)

Recent Improvements (Nov 19, 2025):
 v8 Money Line Sticky Trend (0.6% flip threshold, anti-whipsaw)
 ATR-based TP/SL system (adaptive to volatility)
 ADX-based runner positioning (trend-strength adaptive SL)
 Multi-timeframe price tracking (15min/1H/4H/Daily data collection)
 Quality score increased to 81 (filters small chop trades)
 60/40 TP1/Runner split (captures runners on strong trends)
 Automated MFE/MAE analysis (30-minute tracking per signal)

System Status:
- Readiness: READY → PRODUCTION
- Confidence: HIGH → VERY HIGH
- Multi-timeframe: Fully operational, 2 signals tracked
- Background tracker: Runs every 5 minutes autonomously
- Next milestone: ,000 account (1.85x from current)
- Days to first withdrawal: ~60 days (Early January 2026)

Updated both TRADING_GOALS.md and copilot-instructions.md
2025-11-19 17:40:54 +01:00
mindesbunister
c450a78456 docs: Add multi-timeframe tracking and Docker maintenance to copilot instructions
- Multi-Timeframe Price Tracking System section (Nov 19, 2025)
  - Purpose and architecture overview
  - BlockedSignalTracker background job details
  - Database schema with price tracking fields
  - API endpoints and integration points
  - How it works (step-by-step flow)
  - Analysis queries for cross-timeframe comparison
  - Decision-making criteria for timeframe optimization
  - Current status: 2 signals tracked, pending 4H/Daily alerts

- Docker Maintenance section (Item #13 in When Making Changes)
  - Cleanup commands: image prune, builder prune, volume prune
  - When to run: After builds, weekly, on disk warnings
  - Space savings: 2-5GB images, 40-50GB cache, 0.5-1GB volumes
  - Safety guidelines: What to delete vs keep
  - Why critical: Prevent disk full from build cache accumulation

User requested: 'document in the copilot instructions with the latest updates'
Both sections provide complete reference for future AI agents
2025-11-19 17:27:12 +01:00
mindesbunister
e1bce56065 fix: Correct v8 P&L values in database
Two P&L corrections for v8 trades:
1. First losing trade: Updated from -7.88 to -9.93 (matches Drift UI)
2. Phantom trade bug: Updated from /bin/bash.00 to 4.19 (TP1 + runner combined)

Corrected v8 stats:
- 5 trades, 80% win rate
- Total P&L: 1.06 (was 6.87 before corrections)
- Average: 4.21 per trade
- System recovered all previous losses and turned profitable

Related: Common Pitfall #49 (P&L compounding) and #53 (phantom detection)
caused the /bin/bash.00 entry. Database now reflects actual Drift results.
2025-11-19 15:52:10 +01:00
mindesbunister
57790d04d6 docs: Document ADX-based adaptive runner SL system
Updated copilot-instructions.md with Nov 19, 2025 enhancement:

Architecture Overview - Exit Strategy:
- Added runner SL section with ADX-based positioning
- Documented three-tier system: <20 (0%), 20-25 (-0.3%), >25 (-0.55%)
- Explained rationale: Entry at candle close = natural pullbacks
- Risk management: Only accept drawdown on strong trends
- Example outcomes: ADX 18 → +$38.70, ADX 29 → +$22.20 worst case

Position Manager Section:
- Replaced "Dynamic SL adjustments: breakeven" with ADX-based logic
- Implementation details: Checks trade.adxAtEntry in TP1 handler
- Logging format: "🔒 ADX-based runner SL: 29.3 → -0.55%"
- Data collection phase: 50-100 trades for threshold optimization
- Noted TP1 default is 60% (not 75% - old value)

Key points documented:
- Adaptive vs fixed approach (capital preservation on weak trends)
- Integration with ADX trailing stop multiplier (both trend-adaptive)
- Natural retracement tolerance (entry at top requires -1% room)
- Future optimization plan (adjust 20/25 thresholds based on data)

Related commits:
- 66b2922: Implementation
- 7cf00cf: Fixed runner SL to -0.55% (replaced by ADX-based)
- d09838d: ADX trailing stop multiplier
2025-11-19 13:12:16 +01:00
mindesbunister
6515e1054b docs: Document ADX-based trailing stop multiplier system
Added comprehensive documentation for Nov 19, 2025 enhancement:

Architecture Overview:
- Updated Exit Strategy section with ADX multiplier details
- Documented graduated system (>30: 1.5, 25-30: 1.25×, <25: 1.0×)
- Explained profit acceleration (>2%: 1.3× additional)
- Noted backward compatibility (trades without ADX use base)

Position Manager Section:
- Added detailed ADX multiplier calculation logic
- Documented combined effect examples (ADX 29.3 + 2% profit = 1.95×)
- Explained purpose (capture more of 38% MFE trades)
- Noted trail distance clamping for safety

When Making Changes Section:
- Added item #11 for trailing stop modifications
- Documented verification logs to watch for
- Listed ActiveTrade interface requirement (adxAtEntry field)
- Included example log output for validation

Related commit: d09838d (implementation)
2025-11-19 12:03:06 +01:00
mindesbunister
f65af9530f feat: Enhanced timeframe extraction for multi-timeframe data collection
Updated n8n Parse Signal Enhanced to support multiple timeframe formats:
- 5m, 15m timeframes (buy 5, buy 15)
- Hourly: buy 60, buy 1h → 60
- 4-hour: buy 240, buy 4h → 240
- Daily: buy D, buy 1d → D
- Weekly/Monthly: buy W, buy M

Fixes:
- Default timeframe changed from '15' to '5' (5min is production)
- Added indicator version extraction (IND:v8)
- Proper conversion of hour/day notation to minutes
- Case-insensitive matching for D/W/M

Related: Multi-timeframe data collection system (execute endpoint saves
non-5min signals to BlockedSignal for cross-timeframe analysis). Now
15min signals from TradingView will be properly parsed and saved.

Files:
- workflows/trading/parse_signal_enhanced.json (updated regex + conversion)
- .github/copilot-instructions.md (documented supported formats)
2025-11-19 10:33:12 +01:00
mindesbunister
96f5cfae77 docs: Add Common Pitfall #51 - TP1 detection for fast on-chain fills
Documents the TP1 detection bug where on-chain limit orders fill faster
than Position Manager monitoring loop can detect. When both TP1 (75%)
and runner (25%) close before next monitoring cycle, PM doesn't know
TP1 filled first, marks entire closure as 'SL' instead of 'TP1'.

Fix uses profit percentage thresholds instead of state flags:
- >1.2%: TP2 range
- 0.3-1.2%: TP1 range
- <0.3%: SL range

Simpler and more reliable than tracking order fill state.
2025-11-19 09:09:53 +01:00
mindesbunister
120a4b499e docs: Mark P&L compounding bug as RESOLVED
Common Pitfall #50 updated with resolution details:
- Bug fixed: Removed previouslyRealized from external closure calculations
- Database corrected: 3 v8 trades updated with accurate P&L values
- System operational: New trade executed successfully at 08:40 CET
- Analytics baseline established: -$8.74 total P&L, 66.7% WR, 3 trades

Historical data gap (~3 trades) explains difference between
analytics (-$8.74) and Drift UI (~-$52). All future trades
will track accurately with fixed calculation.

Ready for Phase 1: Collect 50+ v8 trades for statistical validation.
2025-11-19 08:50:22 +01:00
mindesbunister
a69d2b51a5 docs: Document Nov 19 critical findings - database/Drift mismatch and P&L inflation
- Database shows only 3 v8 trades when Drift UI shows 6 trades
- One trade has 10x inflated P&L (81 vs 9 expected)
- Bot receiving ZERO API requests after 06:51 restart despite n8n executions succeeding
- Real v8 performance: ~2 LOSS (from Drift), database shows 20 profit (WRONG)
- Two issues: P&L compounding bug still active + n8n→bot connection broken
- Need to verify n8n workflow endpoints and fix external closure P&L calculation
- Common Pitfall #50 added to copilot-instructions.md
2025-11-19 08:33:03 +01:00
mindesbunister
5146f37acc docs: Add multi-timeframe data collection documentation
- Updated copilot instructions with data collection system overview
- Documents 5min execute vs 15min/1H/4H/Daily collect pattern
- Explains zero-risk parallel data collection approach
- Notes SQL analysis capability for timeframe optimization
- References implementation location in execute endpoint
- Part of v8 indicator testing and optimization strategy
2025-11-18 20:39:08 +01:00
mindesbunister
c330c60d88 docs: Add v8 Money Line indicator documentation to copilot instructions
- Added v8 to indicator version tracking section (Common Pitfall #26)
- Created comprehensive indicator comparison section (v6/v7/v8)
- Documented v8 improvements: flip threshold, momentum confirmation, stickier multipliers
- Added A/B testing expectations and visual backtest results
- Referenced README.md lines 48-120 for user-facing documentation
- Noted v7 deprecation (no fundamental improvements over v6)
2025-11-18 15:57:14 +01:00
mindesbunister
becd5631e7 docs: Add Common Pitfall #49 - P&L exponential compounding bug
Documents the critical P&L compounding bug fixed in commit 6156c0f where
trade.realizedPnL mutation during external closure detection caused 15-20x
inflation of actual profit/loss values.

Includes:
- Root cause: Mutating trade.realizedPnL in monitoring loop
- Real incident: $6 actual profit → $92.46 in database
- Bug mechanism with code examples
- Why previous fixes (Common Pitfalls #27, #48) didn't prevent this
- Fix: Don't mutate shared state during calculations
- Related to other P&L compounding variants for cross-reference
2025-11-17 15:34:22 +01:00
mindesbunister
84bd2e27f0 docs: Add comprehensive ATR-based risk management documentation
- Document ATR × multiplier system (2.0/4.0/3.0 for TP1/TP2/SL)
- Add calculation formula and example with SOL at $140
- Document safety bounds (MIN/MAX percentages)
- Include data-driven ATR values (SOL median 0.43 from 162 trades)
- Document ENV configuration variables
- Add regime-agnostic benefits explanation
- Update Exit Strategy section with ATR-based details
- Update Telegram manual trade presets with accurate ATR
- Add TradingView integration requirements
- Update 'When Making Changes' with ATR modification guidance
- Explain performance analysis and expected improvements

Why: Major system upgrade (Nov 17, 2025) requires complete documentation
for future AI agents and developers. ATR-based targets solve bull/bear
optimization bias by adapting to actual market volatility.
2025-11-17 12:36:44 +01:00
mindesbunister
018f973609 critical: Fix P&L compounding during close verification (20x inflation bug)
Problem:
- Close transaction confirmed but Drift state takes 5-10s to propagate
- Position Manager returned needsVerification=true to keep monitoring
- BUT: Monitoring loop detected position as 'externally closed' EVERY 2 seconds
- Each detection called handleExternalClosure() and added P&L to database
- Result: .66 actual profit → 73.36 in database (20x compounding)
- Logs showed: $112.96 → $117.62 → $122.28 → ... → $173.36 (14+ updates)

Root Cause:
- Common Pitfall #47 fix introduced needsVerification flag to wait for propagation
- But NO flag to prevent external closure detection during wait period
- Monitoring loop thought position was 'closed externally' on every cycle
- Rate limiting (429 errors) made it worse by extending wait time

Fix (closingInProgress flag):
1. Added closingInProgress boolean to ActiveTrade interface
2. Set flag=true when needsVerification returned (close confirmed, waiting)
3. Skip external closure detection entirely while flag=true
4. Timeout after 60s if stuck (abnormal case - allows cleanup)

Impact:
- Every close with verification delay (most closes) had 10-20x P&L inflation
- This is variant of Common Pitfall #27 but during verification, not external closure
- Rate limited closes were hit hardest (longer wait = more compounding cycles)

Files:
- lib/trading/position-manager.ts: Added closingInProgress flag + skip logic

Incident: Nov 16, 11:50 CET - SHORT 41.64→40.08 showed 73.36 vs .66 real
Documented: Common Pitfall #48
2025-11-16 15:07:27 +01:00
mindesbunister
54815a0daa docs: Add multi-fix deployment verification to VERIFICATION MANDATE
- Extended verification checklist for sessions with multiple related fixes
- Added requirement to verify container newer than ALL commits
- Included example from Nov 16 session (3 fixes deployed together)
- Added bash commands for complete deployment verification
- Emphasized that ALL fixes must be deployed, not just latest commit
- Updated Common Pitfall #47 with deployment verification commands
- Prevents declaring fixes 'working' when only some are deployed
2025-11-16 10:33:58 +01:00
mindesbunister
84f40f3e15 docs: Document position close verification fix (Common Pitfall #47)
- Added comprehensive documentation for close verification gap bug
- Real incident: 6 hours unmonitored exposure after close confirmation
- Root cause: Transaction confirmed ≠ Drift state propagated (5-10s delay)
- Fix: 5s wait + verification + needsVerification flag for Position Manager
- Prevents premature database 'closed' marking while position still open
- TypeScript interface updated: ClosePositionResult.needsVerification
- Deployed: Nov 16, 2025 09:28:20 CET
- Commits: c607a66 (logic), b23dde0 (interface)
2025-11-16 10:31:23 +01:00
mindesbunister
9905ab4f5a docs: Add Common Pitfall #47 - position close verification gap 2025-11-16 10:05:12 +01:00
mindesbunister
673a49302a critical: Fix breakeven SL using wrong entry price after TP1
- CRITICAL BUG: Drift SDK's position.entryPrice RECALCULATES after partial closes
- After TP1, Drift returns COST BASIS of remaining position, NOT original entry
- Example: SHORT @ 38.52 → TP1 @ 70% → Drift shows entry 40.01 (runner's basis)
- Result: Breakeven SL set .50 ABOVE actual entry = guaranteed loss if triggered

Fix:
- Always use database trade.entryPrice for breakeven calculations
- Drift's position.entryPrice = current state (runner cost basis)
- Database entryPrice = original entry (authoritative for breakeven)
- Added logging to show both values for verification

Impact:
- Every TP1 → breakeven transition was using WRONG price
- Locking in losses instead of true breakeven protection
- Financial loss bug affecting every trade with TP1

Files:
- lib/trading/position-manager.ts: Line 513 - use trade.entryPrice not position.entryPrice
- .github/copilot-instructions.md: Added Common Pitfall #43, deprecated old #44

Incident: Nov 16, 02:47 CET - SHORT entry 38.52, breakeven SL set at 40.01
Position closed by ghost detection before SL could trigger (lucky)
2025-11-16 03:00:22 +01:00
mindesbunister
737e0c295f docs: Add Common Pitfall #45 - 100% position sizing InsufficientCollateral
Documents the fix for InsufficientCollateral errors when using 100% position
sizing despite having correct account leverage.

Issue: Drift's margin calculation includes fees, slippage buffers, and rounding.
Using exact 100% of collateral leaves no room, causing $0.03-0.10 shortages.

Example: $85.55 collateral
- Bot tries: 100% = $85.55
- Margin needed: $85.58 (includes fees)
- Result: Rejected

Solution: Automatically apply 99% safety buffer when configured at 100%.
Result: $85.55 × 99% = $84.69 (leaves $0.86 safety margin)

Includes real incident details, code implementation, and math proof.
User's Drift UI screenshot proved account leverage WAS set correctly to 15x.

Related commit: 7129cbf
2025-11-16 01:58:18 +01:00
mindesbunister
56b2195b88 docs: Add Common Pitfall #44 - Breakeven SL price discrepancy
Documents the fix for breakeven SL using database entry price instead of
Drift's actual on-chain entry price.

Issue: Database stored $139.18 but Drift actual was $139.07, causing
'breakeven' SL to lock in $0.11 loss per token for SHORT positions.

Solution: Query position.entryPrice from Drift SDK (calculated from
on-chain quoteAssetAmount / baseAssetAmount) when setting breakeven.

Includes real incident details, code comparison, and relationship to
Common Pitfall #33 (orphaned position restoration).

Related commit: 528a0f4
2025-11-16 01:45:48 +01:00
mindesbunister
dc6625404a docs: Document session improvements for developer continuity
Added comprehensive documentation per user request: 'is everything documentet
and pushed to git? nothing we forget to mention which is crucial for other
developers/agents?'

Updates:
- Common Pitfall #40: Added refactoring note (removed time-based Layer 1)
  - User feedback: Time-based cleanup too aggressive for long-running positions
  - Now 100% Drift API-based ghost detection (commit 9db5f85)

- Common Pitfall #41: Telegram notifications for position closures (NEW)
  - Implemented lib/notifications/telegram.ts with sendPositionClosedNotification()
  - Direct Telegram API calls (no n8n dependency)
  - Includes P&L, prices, hold time, MAE/MFE, exit reason with emojis
  - Integrated into Position Manager (commit b1ca454)

- Common Pitfall #42: Telegram bot DNS retry logic (NEW)
  - Python urllib3 transient DNS failures (same as Node.js)
  - Added retry_request() wrapper with exponential backoff
  - 3 attempts (2s → 4s → 8s), matches Node.js retryOperation pattern
  - Applied to /status and manual trade execution (commit bdf1be1)

- Common Pitfall #43: Drift account leverage UI requirement (NEW)
  - Account leverage is on-chain setting, CANNOT be changed via API
  - Must use Drift UI settings page
  - Confusion: Order leverage dropdown ≠ account leverage setting
  - Temporary workaround: Reduced position size to 6% for 1x leverage
  - User action needed: Set account leverage to 15x in Drift UI

- Telegram Notifications section: Added to main architecture documentation
  - Purpose, format, configuration, integration points
  - Reference implementation details

Session focus: Ghost detection refactoring, Telegram notifications, DNS retry,
Drift leverage diagnostics. User emphasized knowledge preservation for future
developers and AI agents working on this codebase.
2025-11-16 01:28:03 +01:00
mindesbunister
bbab693cc1 docs: Document ghost position death spiral fix as Common Pitfall #40
Added comprehensive documentation for 3-layer ghost prevention system:
- Root cause analysis (validation skipped during rate limiting)
- Death spiral explanation (ghosts → rate limits → skipped validation)
- User requirement context (must be fully autonomous)
- Real incident details (Nov 15, 2025)
- Complete solution with code examples for all 3 layers
- Impact and verification notes
- Key lesson: validation logic must never skip during errors

Files changed:
- .github/copilot-instructions.md (added Pitfall #40)
2025-11-15 23:52:39 +01:00
mindesbunister
e057cda990 fix: Settings UI .env permission error - container user writability
CRITICAL FIX: Settings UI was completely broken with EACCES permission denied

Problem:
- .env file on host owned by root:root
- Docker mounts .env as volume, retains host ownership
- Container runs as nextjs user (UID 1001) for security
- Settings API attempts fs.writeFileSync() → permission denied
- Users could NOT adjust position size, leverage, TP/SL, or any config

User escalation: "thats a major flaw. THIS NEEDS TO WORK."

Solution:
- Changed .env ownership on HOST to UID 1001 (nextjs user)
- chown 1001:1001 /home/icke/traderv4/.env
- Restarted container to pick up new permissions
- .env now writable by nextjs user inside container

Verified: Settings UI now saves successfully

Documented as Common Pitfall #39 with:
- Symptom, root cause, and impact
- Why docker exec chown fails (mounted files)
- Correct fix with UID matching
- Alternative solutions and tradeoffs
- Lesson about Docker volume mount ownership

Files changed:
- .github/copilot-instructions.md (added Pitfall #39)
- .env (ownership changed from root:root to 1001:1001)
2025-11-15 23:33:41 +01:00
mindesbunister
c8535bc5b6 docs: Document runner SL live test results and analytics fix as Common Pitfalls
Pitfall #34 (Runner SL gap):
- Updated with live test results from Nov 15, 22:03 CET
- Runner SL triggered successfully with +.94 profit (validates fix works)
- Documented rate limit storm during close (20+ attempts, eventually succeeded)
- Proves software protection works even without on-chain orders
- This was the CRITICAL fix that prevented hours of unprotected exposure

Pitfall #38 (Analytics showing wrong size - NEW):
- Dashboard displayed original position size (2.54) instead of runner (2.59)
- Root cause: API returned positionSizeUSD, not currentSize from Position Manager state
- Fixed by checking configSnapshot.positionManagerState.currentSize for open positions
- API-only change, no container restart needed
- Provides accurate exposure visibility on dashboard

Both issues discovered and fixed during today's live testing session.
2025-11-15 23:15:18 +01:00
mindesbunister
9cd3a015f5 docs: Document runner SL gap as Common Pitfall #27
Added comprehensive documentation of the runner stop loss protection gap:
- Root cause analysis (SL check only before TP1)
- Bug sequence and impact details
- Code fix with examples
- Compounding factors (small runner + no on-chain orders)
- Lesson learned for future risk management code
2025-11-15 22:12:04 +01:00
mindesbunister
ffccf84676 docs: Add Common Pitfall #37 - Ghost position accumulation
Documented:
- Root cause: Failed DB updates leaving exitReason NULL
- Impact: Rate limit storms from managing non-existent positions
- Real incidents: Nov 14-15, 4+ ghost positions tracked
- Solution: Periodic validation every 5 minutes with auto-cleanup
- Implementation details with code examples
- Benefits: Self-healing, minimal overhead, prevents recurrence
- Why paid RPC doesn't fix (state management vs capacity)
2025-11-15 19:22:06 +01:00
mindesbunister
c6b34c45c4 docs: Document closePosition retry logic bug (Common Pitfall #36)
CRITICAL BUG: Missing retry wrapper caused rate limit storm

Real Incident (Nov 15, 16:49 CET):
- Trade cmi0il8l30000r607l8aec701 triggered close attempt
- closePosition() had NO retryWithBackoff() wrapper
- Failed with 429 → Position Manager retried EVERY 2 SECONDS
- 100+ close attempts exhausted Helius rate limit
- On-chain TP2 filled during storm
- External closure detected 8 times: $0.14 → $0.51 (compounding bug)

Why This Was Missed:
- placeExitOrders() got retry wrapper on Nov 14
- openPosition() still has no wrapper (less critical - runs once)
- closePosition() overlooked - MOST CRITICAL because runs in monitoring loop
- Position Manager executeExit() catches 429 and returns early
- But monitoring continues, retries close every 2s = infinite loop

The Fix:
- Wrapped closePosition() placePerpOrder() with retryWithBackoff()
- 8s base delay, 3 max retries (same as placeExitOrders)
- Reduces RPC load by 30-50x during close operations
- Container deployed 18:05 CET Nov 15

Impact: Prevents rate limit exhaustion + duplicate external closure updates

Files: .github/copilot-instructions.md (added Common Pitfall #36)
2025-11-15 18:07:26 +01:00
mindesbunister
1a990054ab docs: Add Common Pitfall #35 - phantom trades need exitReason
- Documented bug where phantom auto-closure sets status='phantom' but left exitReason=NULL
- Startup validator only checks exitReason, not status field
- Ghost positions created false runner stop loss alerts (232% size mismatch)
- Fix: MUST set exitReason when closing phantom trades
- Manual cleanup: UPDATE Trade SET exitReason='manual' WHERE status='phantom' AND exitReason IS NULL
- Verified: System now shows 'Found 0 open trades' after cleanup
2025-11-15 12:24:00 +01:00
mindesbunister
f8141009a8 docs: Document runner stop loss gap bug (Common Pitfall #34)
CRITICAL BUG DOCUMENTATION: Runner had ZERO stop loss protection between TP1-TP2

Context:
- User reported: 'runner close did not work. still open and the price is above 141,98'
- Investigation revealed Position Manager only checked SL before TP1 OR after TP2
- Runner between TP1-TP2 had NO stop loss checks = hours of unlimited loss exposure

Bug Impact:
- SHORT at $141.317, TP1 closed 70% at $140.942, runner had SL at $140.89
- Price rose to $141.98 (way above SL) → NO PROTECTION → Position stayed open
- Potential unlimited loss on 25-30% runner position

Fix Verification:
- After fix deployed: Runner closed at $141.133 with +$0.59 profit
- Database shows exitReason='SL', proving runner stop loss triggered correctly
- Log: '🔴 RUNNER STOP LOSS: SOL-PERP at 0.3% (profit lock triggered)'

Lesson: Every conditional branch in risk management MUST have explicit SL checks

Files: .github/copilot-instructions.md (added Common Pitfall #34)
2025-11-15 11:36:16 +01:00
mindesbunister
5fa946acbd docs: Document entry price correction fix as Common Pitfall #33
Major Fix Summary:
- Position Manager was tracking wrong entry price after orphaned position restoration
- Used stale database value ($141.51) instead of Drift's actual entry ($141.31)
- 0.14% difference in stop loss placement - could mean profit vs loss difference
- Startup validation now queries Drift SDK for authoritative entry price

Impact: Critical for accurate P&L tracking and stop loss placement
Prevention: Always prefer on-chain data over cached DB values for trading params

Added to Common Pitfalls section with full bug sequence, fix code, and lessons learned.
2025-11-15 11:17:46 +01:00
mindesbunister
d654ad3e5e docs: Add Drift SDK memory leak to Common Pitfalls #1
- Documented memory leak fix from Nov 15, 2025
- Symptoms: Heap grows to 4GB+, Telegram timeouts, OOM crash after 10+ hours
- Root cause: WebSocket subscription accumulation in Drift SDK
- Solution: Automatic reconnection every 4 hours
- Renumbered all subsequent pitfalls (2-33)
- Added monitoring guidance and manual control endpoint info
2025-11-15 09:37:13 +01:00
mindesbunister
8862c300e6 docs: Add mandatory instruction update step to When Making Changes
- Added step 14: UPDATE COPILOT-INSTRUCTIONS.MD (MANDATORY)
- Ensures future agents have complete context for data integrity
- Examples: database fields, filtering requirements, analysis exclusions
- Prevents breaking changes to analytics and indicator optimization
- Meta-documentation: instructions about updating instructions
2025-11-14 23:00:22 +01:00
mindesbunister
a9ed814960 docs: Update copilot-instructions with manual trade filtering
- Added signalSource field documentation
- Emphasized CRITICAL exclusion from TradingView indicator analysis
- Reference to MANUAL_TRADE_FILTERING.md for SQL queries
- Manual Trading via Telegram section updated with contamination warning
2025-11-14 22:58:01 +01:00
mindesbunister
3f6fee7e1a docs: Update Common Pitfall #1 with definitive Alchemy investigation results
- Replaced speculation with hard data from diagnostic tests
- Alchemy: 17-71 subscription errors per init (PROVEN)
- Helius: 0 subscription errors per init (PROVEN)
- Root cause: Rate limit enforcement breaks burst subscription pattern
- Investigation CLOSED - Helius is the only solution
- Reference: docs/ALCHEMY_RPC_INVESTIGATION_RESULTS.md
2025-11-14 22:22:04 +01:00
mindesbunister
c1464834d2 docs: Add technical note about Alchemy RPC for future investigation
Research findings:
- Alchemy Growth DOES support WebSocket subscriptions (up to 2,000 connections)
- All standard Solana RPC methods supported
- No documented Drift-Alchemy incompatibilities
- Rate limits enforced via CUPS (Compute Units Per Second)

Hypothesis for our failures:
- accountSubscribe 'errors' might be 429 rate limits, not 'method not found'
- Drift SDK may not handle Alchemy's rate limit pattern during init
- First trade works (subscriptions established) → subsequent fail (bad state)

Pragmatic decision:
- Helius works reliably NOW for production trading
- Theoretical investigation can wait until needed
- Future optimization possible with WebSocket-specific retry logic

This note preserves the research for future reference without changing
the current production recommendation (Helius only).
2025-11-14 21:11:28 +01:00
mindesbunister
47d0969e51 docs: Complete Common Pitfall #1 with full Alchemy testing timeline
DEFINITIVE CONCLUSION:
- Alchemy 'breakthrough' at 14:25 was NOT sustainable
- First trade appeared perfect, subsequent trades consistently fail
- Multiple attempts with pure Alchemy config = same failures
- Helius is the ONLY reliable RPC provider for Drift SDK

Timeline documented:
- 14:01: Switched to Alchemy
- 14:25: First trade perfect (false breakthrough)
- 15:00-20:00: Hybrid/fallback attempts (all failed)
- 20:00: Pure Alchemy retry (still broke)
- 20:05: Helius final revert (works reliably)

User confirmations:
- 'SO IT WAS THE FUCKING RPC...' (initial discovery)
- 'after changing back the settings it started to act up again' (Alchemy breaks)
- 'telegram works again' (Helius works)

This is the complete story for future reference.
2025-11-14 21:08:47 +01:00
mindesbunister
832c9c329e docs: Update Common Pitfall #1 with complete Alchemy incompatibility details
- Documented both Helius rate limit issue AND Alchemy WebSocket incompatibility
- Added user confirmation quote
- Explained why Helius is required (WebSocket subscriptions)
- Explained why Alchemy fails (no accountSubscribe support)
- This is the definitive RPC provider guidance for Drift Protocol
2025-11-14 20:54:17 +01:00
mindesbunister
d5183514bc docs: CRITICAL - document RPC provider as root cause of ALL system failures
CATASTROPHIC BUG DISCOVERY (Nov 14, 2025):
- Helius free tier (10 req/sec) was the ROOT CAUSE of all Position Manager failures
- Switched to Alchemy (300M compute units/month) = INSTANT FIX
- System went from completely broken to perfectly functional in one change

Evidence:
BEFORE (Helius):
- 239 rate limit errors in 10 minutes
- Trades hit SL immediately after opening
- Duplicate close attempts
- Position Manager lost tracking
- Database save failures
- TP1/TP2 never triggered correctly

AFTER (Alchemy) - FIRST TRADE:
- ZERO rate limit errors
- Clean execution with 2s delays
- TP1 hit correctly at +0.4%
- 70% closed automatically
- Runner activated with trailing stop
- Position Manager tracking perfectly
- Currently up +0.77% on runner

Changes:
- Added CRITICAL RPC section to Architecture Overview
- Made RPC provider Common Pitfall #1 (most important)
- Documented symptoms, root cause, fix, and evidence
- Marked Nov 14, 2025 as the day EVERYTHING started working

This was the missing piece that caused weeks of debugging.
User quote: 'SO IT WAS THE FUCKING RPC THAT WAS CAUSING ALL THE ISSUES!!!!!!!!!!!!'
2025-11-14 14:25:29 +01:00
mindesbunister
5a1d51a429 docs: add Nextcloud Deck sync instructions to copilot-instructions
- Added item #13 to 'When Making Changes' section for mandatory Deck updates
- Added new section 'Project-Specific Patterns #5: Nextcloud Deck Roadmap Sync'
- Documents when to sync, how to use scripts, stack mapping, card structure
- Includes best practices: dry-run first, manual deletion required, no duplicates

Integration complete:
- 21 cards: 3 initiatives + 18 phases
- Proper distribution: Backlog (6), Planning (1), In Progress (10), Complete (4)
- No duplicates verified
2025-11-14 11:43:50 +01:00
mindesbunister
a0dc80e96b docs: add Docker cleanup instructions to prevent disk full issues
- Document build cache accumulation problem (40-50 GB typical)
- Add cleanup commands: image prune, builder prune, volume prune
- Recommend running after each deployment or weekly
- Typical space freed: 40-55 GB per cleanup
- Clarify what's safe vs not safe to delete
- Part of maintaining healthy development environment
2025-11-14 10:46:15 +01:00
mindesbunister
6c5a235ea5 docs: update copilot instructions with rate limit fixes and startup validation
- Added /api/trading/sync-positions endpoint to key endpoints list
- Updated retryWithBackoff baseDelay from 2s to 5s with rationale
- Added DNS retry vs rate limit retry distinction (2s vs 5s)
- Updated Position Manager section with startup validation and rate limit-aware exit
- Referenced docs/HELIUS_RATE_LIMITS.md for detailed analysis
- All documentation now reflects Nov 14, 2025 fixes for orphaned positions
2025-11-14 10:22:00 +01:00
mindesbunister
8335699f27 docs: document flip-flop price data bug and fix
Updated documentation to reflect critical bug found and fixed:

SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md:
- Added bug fix commit (795026a) to Phase 1.5
- Documented price source (Pyth price monitor)
- Added validation and logging details
- Included Known Issues section with real incident details
- Updated monitoring examples with detailed price logging

.github/copilot-instructions.md:
- Added Common Pitfall #31: Flip-flop price context bug
- Documented root cause: currentPrice undefined in check-risk
- Real incident: Nov 14 06:05, -$1.56 loss from false positive
- Two-part fix with code examples (price fetch + validation)
- Lesson: Always validate financial calculation inputs
- Monitoring guidance: Watch for flip-flop price check logs

This ensures future AI agents and developers understand:
1. Why Pyth price fetch is needed in check-risk
2. Why validation before calculation is critical
3. The real financial impact of missing validation
2025-11-14 08:27:51 +01:00
mindesbunister
5c0412bcf2 docs: add mandatory git workflow to instructions
- Added 'When Making Changes' item #12: Git commit and push
- Make git workflow mandatory after ANY feature/fix/change
- User should not have to ask - it's part of completion
- Include commit message format and types (feat/fix/docs/refactor)
- Emphasize: code only exists when committed and pushed
- Update trade count: 161 -> 168 (as of Nov 14, 2025)
2025-11-14 05:39:01 +01:00
mindesbunister
6590f4fb1e feat: phantom trade auto-closure system
- Auto-close phantom positions immediately via market order
- Return HTTP 200 (not 500) to allow n8n workflow continuation
- Save phantom trades to database with full P&L tracking
- Exit reason: 'manual' category for phantom auto-closes
- Protects user during unavailable hours (sleeping, no phone)
- Add Docker build best practices to instructions (background + tail)
- Document phantom system as Critical Component #1
- Add Common Pitfall #30: Phantom notification workflow

Why auto-close:
- User can't always respond to phantom alerts
- Unmonitored position = unlimited risk exposure
- Better to exit with small loss/gain than leave exposed
- Re-entry possible if setup actually good

Files changed:
- app/api/trading/execute/route.ts: Auto-close logic
- .github/copilot-instructions.md: Documentation + build pattern
2025-11-14 05:37:51 +01:00
mindesbunister
4ad509928f Update copilot-instructions with Nov 13 critical fixes
Added documentation for two critical fixes:

1. Database-First Pattern (Pitfall #27):
   - Documents the unprotected position bug from today
   - Explains why database save MUST happen before Position Manager add
   - Includes fix code example and impact analysis
   - References CRITICAL_INCIDENT_UNPROTECTED_POSITION.md

2. DNS Retry Logic (Pitfall #28):
   - Documents automatic retry for transient DNS failures
   - Explains EAI_AGAIN, ENOTFOUND, ETIMEDOUT handling
   - Includes retry code example and success logs
   - 99% of DNS failures now auto-recover

Also updated Execute Trade workflow to highlight critical execution order
with explanation of why it's a safety requirement, not just a convention.
2025-11-13 16:10:56 +01:00
mindesbunister
a21ae6d622 Add v7-momentum indicator (experimental, disabled)
- Created momentum scalper indicator for catching rapid price acceleration
- ROC-based detection: 2.0% threshold over 5 bars
- Volume confirmation: 2.0x spike (checks last 3 bars)
- ADX filter: Requires 12+ minimum directional movement
- Anti-chop filter: Blocks signals in dead markets
- Debug table: Real-time metric display for troubleshooting

Status: Functional but signal quality inferior to v6 HalfTrend
Decision: Shelved for now, continue with proven v6 strategy
File: docs/guides/MOMENTUM_INDICATOR_V1.pine (239 lines)

Lessons learned:
- Momentum indicators inherently noisy (40-50% WR expected)
- Signals either too early (false breakouts) or too late (miss move)
- Volume spike timing issue: Often lags price move by 1-2 bars
- Better to optimize proven strategy than add complexity

Related: Position Manager duplicate update bug fixed (awaiting verification)
2025-11-12 19:55:19 +01:00
mindesbunister
04d686a71d docs: update quality score threshold 65→60 based on data
CHANGE: MIN_QUALITY_SCORE lowered from 65 to 60

REASON: Data analysis of 161 trades showed score 60-64 tier
significantly outperformed higher quality scores:
- 60-64: 2 trades, +$45.78 total, 100% WR, +$22.89 avg/trade
- 65-69: 13 trades, +$28.28 total, 53.8% WR, +$2.18 avg/trade

PARADOX DISCOVERED: Higher quality scores don't correlate with
better trading results in current data. Stricter 65 threshold
was blocking profitable 60-64 range setups.

EXPECTED IMPACT:
- 2-3 additional trades per week in 60-64 quality range
- Estimated +$46-69 weekly profit potential based on avg
- Enables blocked signal collection at 55-59 range for Phase 2
- Win rate should remain 50%+ (60-64 tier is 100%, 65-69 is 53.8%)

RISK MANAGEMENT:
- Small sample size (2 trades at 60-64) could be outliers
- Downside limited - can raise back to 65 if performance degrades
- Will monitor first 10 trades at new threshold closely

Added as Common Pitfall #25 with full SQL analysis details.

Updated references in Mission section and Signal Quality System
description to reflect new 60+ threshold.
2025-11-12 13:35:10 +01:00
mindesbunister
381aa168b5 docs: add VERIFICATION MANDATE section to prevent future bugs
Added comprehensive "VERIFICATION MANDATE" section requiring proof
before declaring features working. This addresses pattern of bugs
slipping through despite documentation.

NEW SECTION INCLUDES:
- Core principle: "working" = verified with real data, not code review
- Critical path verification checklists for:
  * Position Manager changes (test trade + logs + SQL verification)
  * Exit logic changes (expected vs actual behavior)
  * API endpoint changes (curl + database + notifications)
  * Calculation changes (verbose logging + SQL validation)
  * SDK integration (never trust docs, verify with console.log)

- Red flags requiring extra verification (unit conversions, state
  transitions, config precedence, display values, timing logic)

- SQL verification queries for Position Manager and P&L calculations

- Real example: How position.size bug should have been caught with
  one console.log statement showing tokens vs USD mismatch

- Deployment checklist: code review → tests → logs → database →
  edge cases → documentation → user notification

- When to escalate: Don't say "it's working" without proof

UPDATED "When Making Changes" section:
#9: Position Manager changes require test trade + log monitoring + SQL
#10: Calculation changes require verbose logging + SQL verification

This creates "prove it works" culture vs "looks like it works".

Root cause of recent bugs: confirmation bias without verification.
- position.size tokens vs USD: looked right, wasn't tested
- leverage display: looked right, notification showed wrong value
- Both would've been caught with one test trade + log observation

Impact: At $97.55 capital with 15x leverage, each bug costs 5-20%
of account. Verification mandate makes this unacceptable going forward.
2025-11-12 13:02:58 +01:00