CRITICAL FIX: Rate limit storm causing infinite close attempts
Root Cause Analysis (Trade cmi0il8l30000r607l8aec701):
- Position Manager tried to close position (SL or TP trigger)
- closePosition() in orders.ts had NO retry wrapper
- Failed with 429 error, returned to Position Manager
- Position Manager caught 429, kept monitoring
- EVERY 2 SECONDS: Attempted close again → 429 → retry
- Result: 100+ close attempts in logs, exhausted Helius rate limit
- Meanwhile: On-chain TP2 limit order filled (not affected by SDK limits)
- External closure detected, updated DB 8 TIMES ($0.14 → $0.51 compounding bug)
Why This Happened:
- placeExitOrders() has retryWithBackoff() wrapper (Nov 14 fix)
- openPosition() has NO retry wrapper (but less critical - only runs once)
- closePosition() had NO retry wrapper (CRITICAL - runs in monitoring loop)
- When closePosition() failed, Position Manager retried EVERY monitoring cycle
The Fix:
- Wrapped closePosition() placePerpOrder() call with retryWithBackoff()
- 8s base delay, 3 max retries (8s → 16s → 32s progression)
- Same pattern as placeExitOrders() for consistency
- Position Manager executeExit() already handles 429 by returning early
- Now: 3 SDK retries (24s) + Position Manager monitoring retry = robust
Impact:
- Prevents rate limit exhaustion from infinite close attempts
- Reduces RPC load by 30-50x during close operations
- Protects against external closure duplicate update bug
- User saw: $0.51 profit (8 DB updates) vs actual $0.14 (1 fill)
Files: lib/drift/orders.ts (line ~567: wrapped placePerpOrder in retryWithBackoff)
Verification: Container restarted 18:05 CET, code deployed
54c68b45d2