Commit Graph

61 Commits

Author SHA1 Message Date
mindesbunister
08482b43fe critical: Raise quality threshold from 81 to 91 after trade #7 analysis
Trade #7 post-mortem (Nov 21, 2025):
- SHORT SOL-PERP: Entry $123.77, Exit $126.33
- Loss: -$386.62 (2.06%, emergency stop)
- Quality: 90, ADX: 19.0 (weak trend, below 20 threshold)
- Duration: 18 minutes
- Result: Price never moved favorably, immediate reversal

Impact on v8 performance:
- 7 trades: 4 wins, 3 losses (57.1% WR, below 60% target)
- Total P&L: $262.70 (down from $649.32)
- Avg quality: 93.6 (still excellent)

Decision rationale:
- ADX 19.0 is borderline weak/chop territory
- Quality 90 insufficient to filter weak-trend entries
- Raised threshold to 91+ to restore 60%+ win rate
- v8 avg 93.6 quality supports higher threshold
- Next blocked signal after loss: quality 80 (correctly blocked)

This implements 3-loss circuit breaker analysis - system paused
for threshold adjustment before resuming.
2025-11-21 18:55:11 +01:00
mindesbunister
0d6e95bbab fix: Read TOTAL_WITHDRAWN from file instead of cached process.env
Bug: Second withdrawal showed $5.92 cumulative instead of $12.50
Root cause: process.env caches values at container startup, doesn't
update when .env file changes.

Fix: Parse TOTAL_WITHDRAWN directly from .env file content instead
of using cached process.env value.

Added: Debug logging showing calculation
Manual fix: Updated .env to $12.50 (sum of $6.58 + $5.92)

Result: Withdrawal stats now correctly show cumulative total.
2025-11-19 21:13:26 +01:00
mindesbunister
c37a9a37d3 fix: Implement Associated Token Account for USDC withdrawals
- Fixed PublicKey undefined error (derive from DRIFT_WALLET_PRIVATE_KEY)
- Implemented ATA resolution using @solana/spl-token
- Added comprehensive debug logging for withdrawal flow
- Fixed AccountOwnedByWrongProgram error (need ATA not wallet address)
- Successfully tested .58 withdrawal with on-chain confirmation
- Updated .env with TOTAL_WITHDRAWN and LAST_WITHDRAWAL_TIME tracking

Key changes:
- lib/drift/withdraw.ts: Added getAssociatedTokenAddress() for USDC ATA
- tsconfig.json: Excluded archive folders from compilation
- package.json: Added bn.js as direct dependency

Transaction: 4drNfMR1xBosGCQtfJ2a4r6oEawUByrT6L7Thyqu6QQWz555hX3QshFuJqiLZreL7KrheSgTdCEqMcXP26fi54JF
Wallet: 3dG7wayp7b9NBMo92D2qL2sy1curSC4TTmskFpaGDrtA
USDC ATA: 8ZEMwErnwxPNNNHJigUcMfrkBG14LCREDdKbqKm49YY7
2025-11-19 20:35:32 +01:00
mindesbunister
9cd317887a fix: Correct BN import for withdrawal system
Changed from '@project-serum/anchor' to 'bn.js' to match
other Drift SDK integrations. Fixes 'Cannot read properties
of undefined (reading '_bn')' error.

User can now test withdrawal with $5 minimum.
2025-11-19 18:39:45 +01:00
mindesbunister
7cf00cf7cb feat: Set runner SL to -0.55% after TP1 for retracement room
Changed PROFIT_LOCK_AFTER_TP1_PERCENT from 0.3% to -0.55%

Reasoning based on user's chart analysis:
- Entry signals trigger on candle close = always entering at top
- Price naturally retraces below entry (screenshots show -1% to -1.5%)
- Old 0.3% profit lock would stop runner out on normal retracements
- New -0.55% allows breathing room while TP1 profit already banked

Risk/Reward:
- 60% already closed at TP1 profit (guaranteed)
- 40% runner can handle -0.55% pullback without stopping out
- Worst case: -0.55% on 40% = -0.22% total position loss
- Best case: Runner catches 38% MFE moves with ADX trailing stop

Example (entry at $140):
- TP1: $140.86 → 60% closed 
- Runner SL: $139.23 (-0.55%)
- Allows pullback to $139.30-139.50 (typical retracement)
- TP2 trigger: $141.72 → ADX trail activates
- Captures big trend moves instead of premature runner stops
2025-11-19 12:30:26 +01:00
mindesbunister
0d546dc267 fix: Correct MIN_QUALITY_SCORE to MIN_SIGNAL_QUALITY_SCORE
Settings UI was using wrong variable name (MIN_QUALITY_SCORE) while
code reads MIN_SIGNAL_QUALITY_SCORE. This caused quality score changes
in settings UI to have no effect.

Fixed:
- Settings API now reads/writes MIN_SIGNAL_QUALITY_SCORE
- Updated .env file to use correct variable name
- User's quality score increase to 81 will now work

Related: User increased min quality from 60 to 81 to filter out
small chop trades (avoiding -$99 trade with quality score 80).
2025-11-19 09:53:49 +01:00
mindesbunister
01427ce3eb fix: Display MAE/MFE as dollars not percentages in analytics UI
- Changed 'Avg MFE: +X.XX%' to 'Avg MFE: +$X.XX'
- Changed 'Avg MAE: -X.XX%' to 'Avg MAE: -$X.XX'
- Database already stores dollar values (fixed Nov 19)
- UI was incorrectly displaying dollars with % suffix
- V8 trades: Avg MFE $34.23, Avg MAE -$11.06 (correct)
- Part of MAE/MFE data corruption fix series

Also corrected existing v8 trade in database:
- Before: MFE 19.73% (account %), MAE -1.53% (account %)
- After: MFE $95.11, MAE -$7.39 (actual dollars)
- SQL: UPDATE based on maxFavorablePrice/maxAdversePrice
2025-11-19 07:57:17 +01:00
mindesbunister
267456f699 critical: Fix MAE/MFE storing percentages instead of dollars + duplicate Telegram notifications
CRITICAL BUGS FIXED (Nov 19, 2025):

1. MAE/MFE Bug:
   - Was storing: account percentage (profit % × leverage)
   - Example: 1.31% move × 15x = 19.73% stored as MFE
   - Should store: actual dollar P&L (81 not 19.73%)
   - Impact: Telegram shows 'Max Gain: +19.73%' instead of '+.XX'
   - Fix: Changed from accountPnL (leverage-adjusted %) to currentPnLDollars
   - Lines 964-987: Removed accountPnL calculation, use currentPnLDollars

2. Duplicate Notification Bug:
   - handleExternalClosure() was checking if trade removed AFTER removal
   - Result: 16 duplicate Telegram notifications with compounding P&L
   - Example: 6 → 2 → 11 → ... → 81 (16 notifications for 1 close)
   - Fix: Check if trade already removed BEFORE processing
   - Lines 382-391: Move duplicate check to START of function
   - Early return prevents notification send if already processed

3. Database Compounding (NOT A BUG):
   - Nov 17 fix (Common Pitfall #49) still working correctly
   - Only 1 database record with 81 P&L
   - Issue was notification duplication, not DB duplication

IMPACT:
- MAE/MFE data now usable for TP/SL optimization
- Telegram notifications accurate (1 per close, correct P&L)
- Database analytics will show real dollar movements
- Next trade will have correct Max Gain/Drawdown display

FILES:
- lib/trading/position-manager.ts: MAE/MFE calculation + duplicate check
2025-11-19 07:45:58 +01:00
mindesbunister
d309eb783c docs: Update roadmaps with v8 indicator progress
- Added v8 Money Line indicator status to master roadmap
- Updated signal quality roadmap with v8 deployment (Nov 18)
- Ready for live testing and v6 vs v8 comparison
- Awaiting first signals with indicatorVersion='v8' tracking
2025-11-18 14:10:06 +01:00
mindesbunister
3aeb00f998 critical: Fix P&L calculation and TP1 false detection bugs
- Add originalPositionSize tracking to prevent stale size usage
- Add price validation to TP1 detection (prevents manual closes misidentified as TP1)
- Fix external closure P&L to use originalPositionSize not currentSize
- Add handleManualClosure method for proper exit reason detection
- Add isPriceAtTarget helper for TP/SL price validation (0.2% tolerance)
- Update all ActiveTrade creation points (execute, test, sync-positions, test-db)

Bug fixes:
- Manual close at 42.34 was detected as TP1 (target 40.71) - FIXED
- P&L showed -$1.71 instead of actual -$2.92 - FIXED
- Exit reason showed SL instead of manual - FIXED

Root cause: Position Manager detected size reduction without validating
price was actually at TP1 level. Used stale currentSize for P&L calculation.

Files modified:
- lib/trading/position-manager.ts (core fixes)
- app/api/trading/execute/route.ts
- app/api/trading/test/route.ts
- app/api/trading/sync-positions/route.ts
- app/api/trading/test-db/route.ts
2025-11-17 15:10:15 +01:00
mindesbunister
b6437295b6 feat: Update Telegram bot ATR to match actual SOL data
Data Analysis (162 SOL trades):
- Average ATR: 0.407 absolute
- Median ATR: 0.43 absolute (~0.32% of price)
- Range: 0.24-0.66 absolute

Updated MANUAL_METRICS:
- Old: atr = 0.45 (estimated)
- New: atr = 0.43 (data-driven median)

Impact on manual Telegram trades:
- TP1: 0.43 × 2.0 = 0.86% (vs 0.90%)
- TP2: 0.43 × 4.0 = 1.72% (vs 1.80%)
- SL: 0.43 × 3.0 = 1.29% (vs 1.35%)

Slightly tighter but more accurate to actual SOL volatility.
2025-11-17 12:28:42 +01:00
mindesbunister
141022243a feat: Implement ATR-based TP/SL system for regime-agnostic trading
CRITICAL UPGRADE - Nov 17, 2025

Problem Solved:
- v6 shorts averaging +20.74% MFE but TP exits at +0.7% (leaving 95% on table)
- Fixed % targets don't adapt to bull/bear regime changes
- User must manually adjust settings when sentiment flips
- Market-regime bias in optimization (bearish now ≠ bullish later)

Solution - ATR-Based Dynamic TP/SL:
- TP1 = ATR × 2.0 (adaptive to volatility)
- TP2 = ATR × 4.0 (captures extended moves)
- SL = ATR × 3.0 (proportional risk)
- Safety bounds prevent extremes (min/max caps)

Example with SOL ATR = 0.45%:
- TP1: 0.45% × 2.0 = 0.90% (vs old fixed 0.4%)
- TP2: 0.45% × 4.0 = 1.80% (vs old fixed 0.7%)
- SL: 0.45% × 3.0 = 1.35% (vs old fixed 1.5%)

Benefits:
 Adapts automatically to bull/bear regime changes
 Asset-agnostic (SOL vs BTC have different ATR)
 Captures more profit in volatile conditions
 Tighter risk in calm conditions
 No manual intervention when sentiment shifts
 Consistent with existing ATR-based trailing stop

Implementation:
- Added TradingConfig fields: atrMultiplierTp1/Tp2/Sl with min/max bounds
- New calculatePercentFromAtr() helper function
- Execute endpoint calculates dynamic % from ATR, falls back to fixed % if unavailable
- ENV variables: ATR_MULTIPLIER_TP1/TP2/SL, MIN_TP1/TP2/SL_PERCENT, MAX_TP1/TP2/SL_PERCENT
- Updated .env with new defaults based on v6 MAE/MFE analysis

Configuration:
- USE_ATR_BASED_TARGETS=true (enabled by default)
- Runner: 40% (TAKE_PROFIT_1_SIZE_PERCENT=60)
- Trailing: 1.3x ATR (existing system, unchanged)
- Legacy fixed % used as fallback when ATR unavailable

Files Modified:
- config/trading.ts (interface + defaults + ENV reading)
- app/api/trading/execute/route.ts (ATR calculation logic)
- .env (new ATR multiplier variables)

Expected Impact:
- Capture 2-3x more profit per winning trade
- Maintain same risk management rigor
- Perform well in BOTH bull and bear markets
- Fix v6 underperformance (-$47.70 → positive)

Testing Required:
- Monitor first 10 trades with ATR-based targets
- Verify TP/SL prices match ATR calculations in logs
- Compare P&L to historical fixed-% performance
2025-11-17 11:41:13 +01:00
mindesbunister
9dfc6da449 feat: Optimize v6 indicator settings for better signal quality
- Moved confirmBars to separate 'Signal Timing' section (no longer under optional filters)
- Made timing setting always-active status clear in UI with improved tooltip
- Updated default settings based on backtesting analysis:
  * confirmBars: 0 → 1 (wait one bar for confirmation, reduces false signals)
  * ADX filter: Enabled by default (was disabled)
  * ADX Length: 14 → 16 (better trend detection)
  * ADX minimum: 20 → 14 (catches trends earlier)
  * Volume filter: Enabled by default (was disabled)
  * Volume max: 3.0x → 3.5x (allows bigger breakout moves)
  * RSI filter: Enabled by default (was disabled)
  * RSI long minimum: 45 → 35 (catches momentum earlier)
  * RSI short maximum: 55 → 70 (CRITICAL: allows shorts during breakdowns)

Why these changes matter:
- Old RSI short max of 55-61 blocked profitable breakdown entries (RSI 62-70 range)
- ADX 21 minimum missed early trend starts, lowering to 14 catches them
- Volume max 3.5x allows high-volume breakouts (was blocking with 3.0x)
- confirmBars=1 reduces wick flips while still catching good moves

Expected impact: Should fix v6 underperformance (-$47.70 over 25 trades)
Target: Positive P&L and 50%+ win rate over next 20-30 trades

Files modified:
- workflows/trading/moneyline_v6_improved.pinescript
2025-11-17 09:57:44 +01:00
mindesbunister
40d69b13ef wip: Emergency order restoration endpoint (has singleton issues)
- Created /api/trading/place-exit-orders endpoint
- Created restore-orders.mjs script
- Issue: Next.js creates separate Drift instances per route
- Workaround: Use /api/trading/cancel-orders to remove orphaned orders

Current situation:
- 32 orphaned orders existed and were cancelled
- Position Manager should auto-place new orders
- Manual order placement endpoint needs refactoring
2025-11-16 22:10:15 +01:00
mindesbunister
673a49302a critical: Fix breakeven SL using wrong entry price after TP1
- CRITICAL BUG: Drift SDK's position.entryPrice RECALCULATES after partial closes
- After TP1, Drift returns COST BASIS of remaining position, NOT original entry
- Example: SHORT @ 38.52 → TP1 @ 70% → Drift shows entry 40.01 (runner's basis)
- Result: Breakeven SL set .50 ABOVE actual entry = guaranteed loss if triggered

Fix:
- Always use database trade.entryPrice for breakeven calculations
- Drift's position.entryPrice = current state (runner cost basis)
- Database entryPrice = original entry (authoritative for breakeven)
- Added logging to show both values for verification

Impact:
- Every TP1 → breakeven transition was using WRONG price
- Locking in losses instead of true breakeven protection
- Financial loss bug affecting every trade with TP1

Files:
- lib/trading/position-manager.ts: Line 513 - use trade.entryPrice not position.entryPrice
- .github/copilot-instructions.md: Added Common Pitfall #43, deprecated old #44

Incident: Nov 16, 02:47 CET - SHORT entry 38.52, breakeven SL set at 40.01
Position closed by ghost detection before SL could trigger (lucky)
2025-11-16 03:00:22 +01:00
mindesbunister
4779a9f732 fix: 3-layer ghost position prevention system (CRITICAL autonomous reliability fix)
PROBLEM: Ghost positions caused death spirals
- Position Manager tracked 2 positions that were actually closed
- Caused massive rate limit storms (100+ RPC calls)
- Telegram /status showed wrong data
- Periodic validation SKIPPED during rate limiting (fatal flaw)
- Created death spiral: ghosts → rate limits → validation skipped → more rate limits

USER REQUIREMENT: "bot has to work all the time especially when i am not on my laptop"
- System MUST be fully autonomous
- Must self-heal from ghost accumulation
- Cannot rely on manual container restarts

SOLUTION: 3-layer protection system (Nov 15, 2025)

**LAYER 1: Database-based age check**
- Runs every 5 minutes during validation
- Removes positions >6 hours old (likely ghosts)
- Doesn't require RPC calls - ALWAYS works even during rate limiting
- Prevents long-term ghost accumulation

**LAYER 2: Death spiral detector**
- Monitors close attempt failures during rate limiting
- After 20+ failed close attempts (40+ seconds), forces removal
- Breaks rate limit death spirals immediately
- Prevents infinite retry loops

**LAYER 3: Monitoring loop integration**
- Every 20 price checks (~40 seconds), verifies position exists on Drift
- Catches ghosts quickly during normal monitoring
- No 5-minute wait - immediate detection
- Silently skips check during RPC errors (no log spam)

**Key fixes:**
- validatePositions(): Now runs database cleanup FIRST before Drift checks
- Changed 'skipping validation' to 'using database-only validation'
- Added cleanupStalePositions() function (>6h age threshold)
- Added death spiral detection in executeExit() rate limit handler
- Added ghost check in checkTradeConditions() every 20 price updates
- All layers work together - if one fails, others protect

**Impact:**
- System now self-healing - no manual intervention needed
- Ghost positions cleaned within 40-360 seconds (depending on layer)
- Works even during severe rate limiting (Layer 1 always runs)
- Telegram /status always shows correct data
- User can be away from laptop - bot handles itself

**Testing:**
- Container restart cleared ghosts (as expected - DB shows all closed)
- New fixes will prevent future accumulation autonomously

Files changed:
- lib/trading/position-manager.ts (3 layers added)
2025-11-15 23:51:19 +01:00
mindesbunister
be36d6aa86 feat: Add live position monitor to analytics dashboard
FEATURE: Real-time position monitoring with auto-refresh every 3 seconds

Implementation:
- New LivePosition interface for real-time trade data
- Auto-refresh hook fetches from /api/trading/positions every 3s
- Displays when Position Manager has active trades
- Shows: P&L (realized + unrealized), current price, TP/SL status, position age

Live Display Includes:
- Header: Symbol, direction (LONG/SHORT), leverage, age, price checks
- Real-time P&L: Profit %, account P&L %, color-coded green/red
- Price Info: Entry, current, position size (with % after TP1), total P&L
- Exit Targets: TP1 (✓ when hit), TP2/Runner, SL (@ B/E when moved)
- P&L Breakdown: Realized, unrealized, peak P&L

Technical:
- Added NEXT_PUBLIC_API_SECRET_KEY to .env for frontend auth
- Positions endpoint requires Bearer token authorization
- Updates every 3s via useEffect interval
- Only shows when monitoring.isActive && positions.length > 0

User Experience:
- Live pulsing green dot indicator
- Auto-updates without page refresh
- Position size shows % remaining after TP1 hit
- SL shows '@ B/E' badge when moved to breakeven
- Color-coded P&L (green profit, red loss)

Files:
- app/analytics/page.tsx: Live position monitor section + auto-refresh
- .env: Added NEXT_PUBLIC_API_SECRET_KEY

User Request: 'i would like to see a live status on the analytics page about an open position'
2025-11-15 18:29:33 +01:00
mindesbunister
8717f72a54 fix: Add retry logic to exit order placement (TP/SL)
CRITICAL FIX: Exit orders failed without retry on 429 rate limits

Root Cause:
- placeExitOrders() placed TP1/TP2/SL orders directly without retry wrapper
- cancelAllOrders() HAD retry logic (8s → 16s → 32s progression)
- Rate limit errors during exit order placement = unprotected positions
- If container crashes after opening, no TP/SL orders on-chain

Fix Applied:
- Wrapped ALL order placements in retryWithBackoff():
  * TP1 limit order (line ~310)
  * TP2 limit order (line ~334)
  * Soft stop trigger-limit (dual stop system)
  * Hard stop trigger-market (dual stop system)
  * Single stop trigger-limit
  * Single stop trigger-market (default)

Retry Behavior:
- Base delay: 8 seconds (was 5s, increased Nov 14)
- Progression: 8s → 16s → 32s (max 3 retries)
- Logs rate_limit_recovered to database on success
- Logs rate_limit_exhausted on max retries exceeded

Impact:
- Exit orders now retry up to 3x on 429 errors (56 seconds total wait)
- Positions protected even during RPC rate limit spikes
- Reduces need for immediate Helius upgrade
- Database analytics track retry success/failure

Files: lib/drift/orders.ts (6 placePerpOrder calls wrapped)

Note: cancelAllOrders() already had retry logic - this completes coverage
2025-11-15 17:34:01 +01:00
mindesbunister
1a990054ab docs: Add Common Pitfall #35 - phantom trades need exitReason
- Documented bug where phantom auto-closure sets status='phantom' but left exitReason=NULL
- Startup validator only checks exitReason, not status field
- Ghost positions created false runner stop loss alerts (232% size mismatch)
- Fix: MUST set exitReason when closing phantom trades
- Manual cleanup: UPDATE Trade SET exitReason='manual' WHERE status='phantom' AND exitReason IS NULL
- Verified: System now shows 'Found 0 open trades' after cleanup
2025-11-15 12:24:00 +01:00
mindesbunister
0ef6b82106 feat: Hybrid RPC strategy (Helius init + Alchemy trades)
CRITICAL: Fix rate limiting by using dual RPC approach

Problem:
- Helius RPC gets overwhelmed during trade execution (429 errors)
- Exit orders fail to place, leaving positions UNPROTECTED
- No on-chain TP/SL orders = unlimited risk if container crashes

Solution: Hybrid RPC Strategy
- Helius for Drift SDK initialization (handles burst subscriptions well)
- Alchemy for trade operations (better sustained rate limits)
- Falls back to Helius if Alchemy not configured

Implementation:
- DriftService now has two connections: connection (Helius) + tradeConnection (Alchemy)
- Added getTradeConnection() method for trade operations
- Updated openPosition() and closePosition() to use trade connection
- Added ALCHEMY_RPC_URL to .env (optional, falls back to Helius)

Benefits:
- Helius: 0 subscription errors during init (proven reliable for SDK setup)
- Alchemy: 300M compute units/month for sustained trade operations
- Best of both worlds: reliable init + reliable trades

Files:
- lib/drift/client.ts: Dual connection support
- lib/drift/orders.ts: Use getTradeConnection() for confirmations
- .env: Added ALCHEMY_RPC_URL

Testing: Deploy and execute test trade to verify orders place successfully
2025-11-15 12:00:57 +01:00
mindesbunister
324e5ba002 refactor: Rename breakEvenTriggerPercent to profitLockAfterTP1Percent for clarity
- Renamed config variable to accurately reflect behavior (locks profit, not breakeven)
- Updated log messages to say 'lock +X% profit' instead of misleading 'breakeven'
- Maintains backwards compatibility (accepts old BREAKEVEN_TRIGGER_PERCENT env var)
- Updated .env with new variable name and explanatory comment

Why: Config was named 'breakeven' but actually locks profit at entry ± X%
For SHORT at $141.51 with 0.3% lock: SL moves to $141.08 (not breakeven $141.51)
This protects remaining runner position after TP1 by allowing small profit giveback

Files changed:
- config/trading.ts: Interface + default + env parsing
- lib/trading/position-manager.ts: Usage + log message
- .env: Variable rename with migration comment
2025-11-15 11:06:44 +01:00
mindesbunister
19beaf9c02 fix: Revert to Helius - Alchemy 'breakthrough' was not sustainable
FINAL CONCLUSION after extensive testing:
- Alchemy appeared to work perfectly at 14:25 CET (first trade)
- User quote: 'SO IT WAS THE FUCKING RPC THAT WAS CAUSING ALL THE ISSUES!!!!!!!!!!!!'
- BUT: Alchemy consistently fails after that initial success
- Multiple attempts to use Alchemy (pure config, no fallback) = same result
- Symptoms: timeouts, positions open WITHOUT TP/SL orders, no Position Manager tracking

HELIUS = ONLY RELIABLE OPTION:
- User confirmed: 'telegram works again' after reverting to Helius
- Works consistently across multiple tests
- Supports WebSocket subscriptions (accountSubscribe) that Drift SDK requires
- Rate limits manageable with 5s exponential backoff

ALCHEMY INCOMPATIBILITY CONFIRMED:
- Does NOT support WebSocket subscriptions (accountSubscribe method)
- SDK appears to initialize but is fundamentally broken
- First trade might work, then SDK gets into bad state
- Cannot be used reliably for Drift Protocol trading

Files restored from working Helius state.
This is the definitive answer: Helius only, no alternatives work.
2025-11-14 21:07:58 +01:00
mindesbunister
f30a2c4ed4 fix: CRITICAL - Revert to Helius RPC (Alchemy breaks Drift SDK)
ISSUE CONFIRMED:
- Alchemy RPC does NOT support WebSocket subscriptions (accountSubscribe method)
- Drift SDK REQUIRES WebSocket support to function properly
- When using Alchemy:
  * SDK initializes with 100+ accountSubscribe errors
  * Claims 'initialized successfully' but is actually broken
  * First API call (openPosition) sometimes works
  * Subsequent calls hang indefinitely OR
  * Positions open without TP/SL orders (NO RISK MANAGEMENT)
  * Position Manager doesn't track positions

SOLUTION:
- Use Helius as primary RPC (supports all Solana methods + WebSocket)
- Helius free tier: 10 req/sec sustained, 100 burst
- Rate limits manageable with retry logic (5s exponential backoff)
- System fully operational with Helius

ALCHEMY INCOMPATIBILITY:
- Alchemy Growth (10,000 CU/s) excellent for raw transaction throughput
- But completely incompatible with Drift SDK architecture
- Cannot be used as primary RPC for Drift Protocol trading

User confirmed: 'after changing back the settings it started to act up again'
This is Common Pitfall #1 - NEVER use RPC without WebSocket support
2025-11-14 20:53:16 +01:00
mindesbunister
78ab9e1a94 fix: Increase transaction confirmation timeout to 60s for Alchemy Growth
- Alchemy Growth (10,000 CU/s) can handle longer confirmation waits
- Increased timeout from 30s to 60s in both openPosition() and closePosition()
- Added debug logging to execute endpoint to trace hang points
- Configured dual RPC: Alchemy primary (transactions), Helius fallback (subscriptions)
- Previous 30s timeout was causing premature failures during Solana congestion
- This should resolve 'Transaction was not confirmed in 30.00 seconds' errors

Related: User reported n8n webhook returning 500 with timeout error
2025-11-14 20:42:59 +01:00
mindesbunister
6dccea5d91 revert: Back to last known working state (27eb5d4)
- Restored Drift client, orders, and .env from commit 27eb5d4
- Updated to current Helius API key
- ISSUE: Execute/check-risk endpoints still hang
- Root cause appears to be Drift SDK initialization hanging at runtime
- Bot initializes successfully at startup but hangs on subsequent Drift calls
- Non-Drift endpoints work fine (settings, positions query)
- Needs investigation: Drift SDK behavior or RPC interaction issue
2025-11-14 20:17:50 +01:00
mindesbunister
6445a135a8 feat: Helius primary + Alchemy fallback for trade execution
- Helius HTTPS: Primary RPC for Drift SDK initialization and subscriptions
- Alchemy HTTPS (10K CU/s): Fallback RPC for transaction confirmations
- Added getFallbackConnection() method to DriftService
- openPosition() and closePosition() now use Alchemy for tx confirmations
- accountSubscribe errors are non-fatal warnings (SDK falls back gracefully)
- System fully operational: Drift initialized, Position Manager ready
- Trade execution will use high-throughput Alchemy for confirmations
2025-11-14 16:51:14 +01:00
mindesbunister
1cf5c9aba1 feat: Smart startup RPC strategy (Helius → Alchemy)
Strategy:
1. Start with Helius (handles startup burst better - 10 req/sec sustained)
2. After successful init, switch to Alchemy (more stable for trading)
3. On 429 errors during operations, fall back to Helius, then return to Alchemy

Implementation:
- lib/drift/client.ts: Smart constructor checks for fallback, uses it for startup
- After initialize() completes, automatically switches to primary RPC
- Swaps connections and reinitializes Drift SDK with Alchemy
- Falls back to Helius on rate limits, switches back after recovery

Benefits:
- Helius absorbs SDK subscribe() burst (many concurrent calls)
- Alchemy provides stability for normal trading operations
- Best of both worlds: burst tolerance + operational stability

Status:
- Code complete and tested
- Helius API key needs updating (current key returns 401)
- Fallback temporarily disabled in .env until key fixed
- Position Manager working perfectly (trade monitored via Alchemy)

To enable:
1. Get fresh Helius API key from helius.dev
2. Set SOLANA_FALLBACK_RPC_URL in .env
3. Restart bot - will use Helius for startup automatically
2025-11-14 15:41:52 +01:00
mindesbunister
7ff78ee0bd feat: Hybrid RPC fallback system (Alchemy → Helius)
- Automatic fallback after 2 consecutive rate limits
- Primary: Alchemy (300M CU/month, stable for normal ops)
- Fallback: Helius (10 req/sec, backup for startup bursts)
- Reduced startup validation: 6h window, 5 trades (was 24h, 20 trades)
- Multi-position safety check (prevents order cancellation conflicts)
- Rate limit-aware retry logic with exponential backoff

Implementation:
- lib/drift/client.ts: Added fallbackConnection, switchToFallbackRpc()
- .env: SOLANA_FALLBACK_RPC_URL configuration
- lib/startup/init-position-manager.ts: Reduced validation scope
- lib/trading/position-manager.ts: Multi-position order protection

Tested: System switched to fallback on startup, Position Manager active
Result: 1 active trade being monitored after automatic RPC switch
2025-11-14 15:28:07 +01:00
mindesbunister
7afd7d5aa1 feat: switch from Helius to Alchemy RPC provider
Changes:
- Updated SOLANA_RPC_URL to use Alchemy (https://solana-mainnet.g.alchemy.com/v2/...)
- Migrated from Helius free tier to Alchemy free tier
- Includes previous rate limit fixes (8s backoff, 2s operation delays)

Context:
- Helius free tier: 10 req/sec sustained, 100 req/sec burst
- Alchemy free tier: 300M compute units/month (more generous)
- User hit 239 rate limit errors in 10 minutes on Helius
- User registered Alchemy account and provided API key

Impact:
- Should significantly reduce 429 rate limit errors
- Better free tier limits for trading bot operations
- Combined with delay fixes for optimal RPC usage
2025-11-14 14:01:52 +01:00
mindesbunister
28c1110a85 feat: Integrate dynamic ATR analysis into TP/SL optimization endpoint
- Added dynamicATRAnalysis section to /api/analytics/tp-sl-optimization
- Analyzes v6 trades with ATR data to compare fixed vs dynamic targets
- Dynamic targets: TP2=2x ATR, SL=1.5x ATR (from config)
- Shows +39.8% advantage with 14 trades (.72 improvement)
- Includes data sufficiency check (need 30+ trades)
- Recommendation logic: WAIT/IMPLEMENT/CONSIDER/NEUTRAL based on sample size and advantage
- Returns detailed metrics: sample size, avg ATR, hit rates, P&L comparison
- Integrates seamlessly with existing MAE/MFE analysis

Current status: 14/30 trades collected, insufficient for implementation
Expected: Frontend will display this data to track progress toward 30-trade threshold
2025-11-14 09:03:15 +01:00
mindesbunister
6590f4fb1e feat: phantom trade auto-closure system
- Auto-close phantom positions immediately via market order
- Return HTTP 200 (not 500) to allow n8n workflow continuation
- Save phantom trades to database with full P&L tracking
- Exit reason: 'manual' category for phantom auto-closes
- Protects user during unavailable hours (sleeping, no phone)
- Add Docker build best practices to instructions (background + tail)
- Document phantom system as Critical Component #1
- Add Common Pitfall #30: Phantom notification workflow

Why auto-close:
- User can't always respond to phantom alerts
- Unmonitored position = unlimited risk exposure
- Better to exit with small loss/gain than leave exposed
- Re-entry possible if setup actually good

Files changed:
- app/api/trading/execute/route.ts: Auto-close logic
- .github/copilot-instructions.md: Documentation + build pattern
2025-11-14 05:37:51 +01:00
mindesbunister
a21ae6d622 Add v7-momentum indicator (experimental, disabled)
- Created momentum scalper indicator for catching rapid price acceleration
- ROC-based detection: 2.0% threshold over 5 bars
- Volume confirmation: 2.0x spike (checks last 3 bars)
- ADX filter: Requires 12+ minimum directional movement
- Anti-chop filter: Blocks signals in dead markets
- Debug table: Real-time metric display for troubleshooting

Status: Functional but signal quality inferior to v6 HalfTrend
Decision: Shelved for now, continue with proven v6 strategy
File: docs/guides/MOMENTUM_INDICATOR_V1.pine (239 lines)

Lessons learned:
- Momentum indicators inherently noisy (40-50% WR expected)
- Signals either too early (false breakouts) or too late (miss move)
- Volume spike timing issue: Often lags price move by 1-2 bars
- Better to optimize proven strategy than add complexity

Related: Position Manager duplicate update bug fixed (awaiting verification)
2025-11-12 19:55:19 +01:00
mindesbunister
7f355f38f5 docs: add ATR-based take profit roadmap
Added comprehensive roadmap for implementing ATR-based TP/SL targets
as alternative to fixed percentage targets.

Key points:
- Phase 1: Data collection (1/50 trades with ATR tracking)
- Phase 2: Backtest analysis with SQL queries ready
- Phase 3: Implementation with config toggles
- Phase 4: A/B testing (50% fixed vs 50% ATR-based)
- Phase 5: Full deployment if results show improvement

Benefits:
- Adapts to market volatility automatically
- Tight targets in calm markets, wider in volatile markets
- Already collecting ATR data with every trade
- Aligns with existing ATR-based trailing stop

Timeline: 6-8 weeks (need 50+ trades for meaningful backtest)
Target: 10%+ P&L improvement, maintain 60%+ win rate

See ATR_BASED_TP_ROADMAP.md for complete implementation plan.
2025-11-12 12:28:06 +01:00
mindesbunister
6a192bfb76 docs: update copilot-instructions for ATR trailing + dynamic runner% + rate limits
Updated .github/copilot-instructions.md to reflect recent system improvements:

**ATR-Based Trailing Stop:**
- Dynamic trailing calculation formula documented
- Configurable runner % (default 25%, adjustable via TAKE_PROFIT_1_SIZE_PERCENT)
- All UI displays now dynamically calculate runner% as 100 - TP1_SIZE
- Removed hardcoded '25%' references, replaced with dynamic language

**Rate Limit Monitoring:**
- NEW Section #4: Rate Limit Monitoring
- Exponential backoff mechanism (2s→4s→8s)
- Database logging (3 event types: hit/recovered/exhausted)
- Analytics endpoint for monitoring
- Links to RATE_LIMIT_MONITORING.md for SQL queries

**Section Renumbering:**
- Old Section #4 (Order Placement) → Section #5
- Old Section #5 (Database) → Section #6
- Maintains logical flow and consistency

**Updated References:**
- Exit Strategy: Dynamic runner% description
- Position Manager: ATR trailing formula + on-chain sync notes
- Common Pitfalls: Dynamic runner % configuration notes
- Roadmap: Phase 5 shows configurable runner with formula

All documentation now accurately reflects user's 70/30 TP1/Runner split
and recent infrastructure improvements (ATR trailing, rate limits).

Related: settings UI updated in previous commit (app/settings/page.tsx)
2025-11-11 20:40:05 +01:00
mindesbunister
03e91fc18d feat: ATR-based trailing stop + rate limit monitoring
MAJOR FIXES:
- ATR-based trailing stop for runners (was fixed 0.3%, now adapts to volatility)
- Fixes runners with +7-9% MFE exiting for losses
- Typical improvement: 2.24x more room (0.3% → 0.67% at 0.45% ATR)
- Enhanced rate limit logging with database tracking
- New /api/analytics/rate-limits endpoint for monitoring

DETAILS:
- Position Manager: Calculate trailing as (atrAtEntry / price × 100) × multiplier
- Config: TRAILING_STOP_ATR_MULTIPLIER=1.5, MIN=0.25%, MAX=0.9%
- Settings UI: Added ATR multiplier controls
- Rate limits: Log hits/recoveries/exhaustions to SystemEvent table
- Documentation: ATR_TRAILING_STOP_FIX.md + RATE_LIMIT_MONITORING.md

IMPACT:
- Runners can now capture big moves (like morning's $172→$162 SOL drop)
- Rate limit visibility prevents silent failures
- Data-driven optimization for RPC endpoint health
2025-11-11 14:51:41 +01:00
mindesbunister
6f0a1bb49b feat: Implement percentage-based position sizing
- Add usePercentageSize flag to SymbolSettings and TradingConfig
- Add calculateActualPositionSize() and getActualPositionSizeForSymbol() helpers
- Update execute and test endpoints to calculate position size from free collateral
- Add SOLANA_USE_PERCENTAGE_SIZE, ETHEREUM_USE_PERCENTAGE_SIZE, USE_PERCENTAGE_SIZE env vars
- Configure SOL to use 100% of portfolio (auto-adjusts to available balance)
- Fix TypeScript errors: replace fillNotionalUSD with actualSizeUSD
- Remove signalQualityVersion and fullyClosed references (not in interfaces)
- Add comprehensive documentation in PERCENTAGE_SIZING_FEATURE.md

Benefits:
- Prevents insufficient collateral errors by using available balance
- Auto-scales positions as account grows/shrinks
- Maintains risk proportional to capital
- Flexible per-symbol configuration (SOL percentage, ETH fixed)
2025-11-10 13:35:10 +01:00
mindesbunister
d2fbd125a0 fix: Make minSignalQualityScore configurable via settings + anti-chop improvements
CRITICAL BUG FIX:
- Settings page saved MIN_SIGNAL_QUALITY_SCORE to .env but check-risk had hardcoded value
- Now reads from config.minSignalQualityScore (defaults to 65, editable via /settings)
- Prevents settings changes from being ignored after restart

ANTI-CHOP FILTER FIXES:
- Fixed volume breakout bonus conflicting with anti-chop filter
- Volume breakout now requires ADX > 18 (trending market)
- Prevents high volume + low ADX from getting rewarded instead of penalized
- Anti-chop filter now properly blocks whipsaw traps at score 60

TESTING INFRASTRUCTURE:
- Added backtest script showing +17.1% P&L improvement (saved $242 in losses)
- Added test-signals.sh for comprehensive signal quality validation
- Added test-recent-signals.sh for analyzing actual trading session signals
- All tests passing: timeframe awareness, anti-chop, score thresholds

CHANGES:
- config/trading.ts: Added minSignalQualityScore to interface and defaults
- app/api/trading/check-risk/route.ts: Use config value instead of hardcoded 65
- lib/trading/signal-quality.ts: Fixed volume breakout bonus logic
- .env: Added MIN_SIGNAL_QUALITY_SCORE=65
- scripts/: Added comprehensive testing tools

BACKTEST RESULTS (Last 30 trades):
- Old system (score ≥60): $1,412.79 P&L
- New system (score ≥65 + anti-chop): $1,654.79 P&L
- Improvement: +$242.00 (+17.1%)
- Blocked 5 losing trades, missed 0 winners
2025-11-10 11:22:52 +01:00
mindesbunister
6d5991172a feat: Implement ATR-based dynamic TP2 system and fix P&L calculation
- Add ATR-based dynamic TP2 scaling from 0.7% to 3.0% based on volatility
- New config options: useAtrBasedTargets, atrMultiplierForTp2, minTp2Percent, maxTp2Percent
- Enhanced settings UI with ATR controls and updated risk calculator
- Fix external closure P&L calculation using unrealized P&L instead of volatile current price
- Update execute and test endpoints to use calculateDynamicTp2() function
- Maintain 25% runner system for capturing extended moves (4-5% targets)
- Add environment variables for ATR-based configuration
- Better P&L accuracy for manual position closures
2025-11-07 17:01:22 +01:00
mindesbunister
5acc61cf66 Fix P&L calculation and update Copilot instructions
- Fix P&L calculation in Position Manager to use actual entry vs exit price instead of SDK's potentially incorrect realizedPnL
- Calculate actual profit percentage and apply to closed position size for accurate dollar amounts
- Update database record for last trade from incorrect 6.58 to actual .66 P&L
- Update .github/copilot-instructions.md to reflect TP2-as-runner system changes
- Document 25% runner system (5x larger than old 5%) with ATR-based trailing
- Add critical P&L calculation pattern to common pitfalls section
- Mark Phase 5 complete in development roadmap
2025-11-07 16:24:43 +01:00
mindesbunister
625dc44c59 Add signal quality version tracking to database
- Added signalQualityVersion field to Trade model
- Tracks which scoring logic version was used for each trade
- v1: Original logic (price position < 5% threshold)
- v2: Added volume compensation for low ADX
- v3: CURRENT - Stricter logic requiring ADX > 18 for extreme positions (< 15%)

This enables future analysis to:
- Compare performance between logic versions
- Filter trades by scoring algorithm
- Data-driven improvements based on clean datasets

All new trades will be marked as v3. Old trades remain null/v1 for comparison.
2025-11-07 12:56:35 +01:00
mindesbunister
6c7eaf5f04 Add TP1/SL consistency check on trade restore 2025-11-06 12:18:31 +01:00
mindesbunister
6b1d32a72d fix: Add phantom trade detection and prevention safeguards
**Root Causes:**
1. Auto-flip logic could create phantom trades if close failed
2. Position size mismatches (0.01 SOL vs 11.92 SOL expected) not caught
3. Multiple trades for same symbol+direction in database

**Preventive Measures:**

1. **Startup Validation (lib/startup/init-position-manager.ts)**
   - Validates all open trades against Drift positions on startup
   - Auto-closes phantom trades with <50% expected size
   - Logs size mismatches for manual review
   - Prevents Position Manager from tracking ghost positions

2. **Duplicate Position Prevention (app/api/trading/execute/route.ts)**
   - Blocks opening same-direction position on same symbol
   - Returns 400 error if duplicate detected
   - Only allows auto-flip (opposite direction close + open)

3. **Runtime Phantom Detection (lib/trading/position-manager.ts)**
   - Checks position size every 2s monitoring cycle
   - Auto-closes if size ratio <50% (extreme mismatch)
   - Logs as 'manual' exit with AUTO_CLEANUP tx
   - Removes from monitoring immediately

4. **Quality Score Fix (app/api/trading/check-risk/route.ts)**
   - Hardcoded minScore=60 (removed non-existent config reference)

**Prevention Summary:**
-  Startup validation catches historical phantoms
-  Duplicate check prevents new phantoms
-  Runtime detection catches size mismatches <30s after they occur
-  All three layers work together for defense-in-depth

Issue: User had LONG (phantom) + SHORT (undersized 0.01 SOL vs 11.92 expected)
Fix: Both detected and closed, bot now clean with 0 active trades
2025-11-03 13:53:12 +01:00
mindesbunister
881a99242d feat: Add per-symbol trading controls for SOL and ETH
- Add SymbolSettings interface with enabled/positionSize/leverage fields
- Implement per-symbol ENV variables (SOLANA_*, ETHEREUM_*)
- Add SOL and ETH sections to settings UI with enable/disable toggles
- Add symbol-specific test buttons (SOL LONG/SHORT, ETH LONG/SHORT)
- Update execute and test endpoints to check symbol enabled status
- Add real-time risk/reward calculator per symbol
- Rename 'Position Sizing' to 'Global Fallback' for clarity
- Fix position manager P&L calculation for externally closed positions
- Fix zero P&L bug affecting 12 historical trades
- Add SQL scripts for recalculating historical P&L data
- Move archive TypeScript files to .archive to fix build

Defaults:
- SOL: 10 base × 10x leverage = 100 notional (profit trading)
- ETH:  base × 1x leverage =  notional (data collection)
- Global: 10 × 10x for BTC and other symbols

Configuration priority: Per-symbol ENV > Market config > Global ENV > Defaults
2025-11-03 10:28:48 +01:00
mindesbunister
49a09ef04e Add 'Clear Manual Closes' button to analytics
- New button in analytics page to clear orphaned trades
- API endpoint /api/trading/clear-manual-closes
- Intelligently checks Drift positions before deleting
- Only removes trades with no matching position or mismatched entry price
- Safe operation: keeps trades on error (false positives better than deletions)
- User-friendly confirmation dialog
2025-11-01 02:41:26 +01:00
mindesbunister
c82da51bdc CRITICAL FIX: Add transaction confirmation to detect failed orders
- Added getConnection() method to DriftService
- Added proper transaction confirmation in openPosition()
- Check confirmation.value.err to detect on-chain failures
- Return error if transaction fails instead of assuming success
- Prevents phantom trades that never actually execute

This fixes the issue where bot was recording trades with transaction
signatures that don't exist on-chain (like 2gqrPxnvGzdRp56...).
2025-11-01 02:26:47 +01:00
mindesbunister
a6005b6a5b Add configurable minimum quality score setting
- Added minQualityScore to TradingConfig (default: 60)
- Updated settings UI with slider control (0-100, step 5)
- Updated check-risk endpoint to use config value
- Made scoreSignalQuality function accept minScore parameter
- Updated API to read/write MIN_QUALITY_SCORE env variable
- Allows users to adjust quality threshold from settings page
2025-11-01 01:59:08 +01:00
mindesbunister
781b88f803 Enhance TradingView indicator with context metrics for signal quality
Added 5 context metrics to alert messages:
- ATR% (volatility as % of price)
- ADX (trend strength)
- RSI (momentum)
- VOL (volume ratio vs 20-bar MA)
- POS (price position in 20-bar range 0-100%)

Changes to Pine Script:
- Always calculate ADX (needed for context even if filter disabled)
- Extract ta.rma() calls outside ternary operators (Pine Script requirement)
- Use alert() instead of alertcondition() for dynamic message support
- Changed to single-line string concatenation for compatibility

Alert message format:
OLD: 'Buy SOL 15 | Profile=Hours ATR=10 Mult=3.0'
NEW: 'SOL buy .P 15 | ATR:1.85 | ADX:28.3 | RSI:62.5 | VOL:1.45 | POS:75.3'

Next: Update n8n to parse these metrics, implement signal quality scoring in bot
2025-10-30 15:53:48 +01:00
mindesbunister
b7b0fb9bb2 Change cooldown unit from seconds to minutes
- Updated minTimeBetweenTrades config to use minutes instead of seconds
- Changed default from 600 seconds to 10 minutes
- Updated Settings UI label from 'seconds' to 'minutes' and adjusted range (0-60 min)
- Updated .env comments to reflect new unit
- No functional change since cooldown enforcement not yet implemented (TODO in check-risk route)
2025-10-30 10:35:47 +01:00
mindesbunister
797e80b56a CRITICAL FIX: TP/SL orders using wrong size calculation
**ROOT CAUSE:** placeExitOrders() calculated position size using TP/SL prices instead of entry price

**Problem:**
- TP1 order size: 85 / TP1_price (00.746) = 2.914 SOL
- Actual position: 80 / entry_price (99.946) = 3.901 SOL
- TP1 should close: 3.901 * 75% = 2.926 SOL
- But it only closed: 2.914 SOL = 74.7%  WRONG!

**Result:** TP1 closed ~25% instead of 75%, no runner left

**Fix:**
- Changed usdToBase() to use entryPrice for ALL size calculations
- Added entryPrice param to PlaceExitOrdersOptions interface
- Updated all API routes to pass entryPrice

**Testing:** Next trade will have correctly sized TP/SL orders
2025-10-29 17:34:10 +01:00
mindesbunister
19f5b7ab14 Fix: Critical Position Manager monitoring issues
**Root Cause:** Position Manager didn't detect when on-chain TP/SL orders closed positions externally, causing endless error loops and stale position data.

**Issues Fixed:**
1. Position Manager now checks if on-chain position still exists before attempting to close
2. Detects external closures (by on-chain orders) and updates database accordingly
3. Determines likely exit reason based on price vs TP/SL levels
4. Automatically cancels leftover orders when position detected as closed
5. Analytics now properly shows stopped-out trades

**Technical Changes:**
- Added position existence check at start of checkTradeConditions()
- Calls DriftService.getPosition() to verify on-chain state
- Updates database with exitPrice, exitReason, realizedPnL when external closure detected
- Removes trade from monitoring after external closure
- Handles size mismatches for partial closes (TP1 hit externally)

**Database Fix:**
- Manually closed orphaned trade (stopped out 9 hours ago but still marked 'open')
- Calculated and set realizedPnL = -$12.00 for stopped-out SHORT position
- Analytics now shows 3 total trades instead of missing the SL exit

**Testing:**
- Bot starts cleanly with no error loops
- Position monitoring active with 0 trades (as expected)
- Analytics correctly shows stopped-out trade in statistics
2025-10-28 07:51:40 +01:00