Add Position Sync feature for recovering tracking after partial fills

- New /api/trading/sync-positions endpoint (no auth) - Fetches actual Drift positions and compares with Position Manager - Removes stale tracking, adds missing positions with calculated TP/SL - Settings UI: Orange 'Sync Positions' button added - CLI script: scripts/sync-positions.sh for terminal access - Full documentation in docs/guides/POSITION_SYNC_GUIDE.md - Quick reference: POSITION_SYNC_QUICK_REF.md - Updated AI instructions with pitfall #23 Problem solved: Manual Telegram trades with partial fills can cause Position Manager to lose tracking, leaving positions without software- based stop loss protection. This feature restores dual-layer protection. Note: Docker build not picking up route yet (cache issue), needs investigation
2025-11-10 17:05:32 +01:00
parent 2e47731e8e
commit 089308a07e
6 changed files with 440 additions and 0 deletions
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -251,6 +251,7 @@ const driftSymbol = normalizeTradingViewSymbol(body.symbol)
 - `/api/trading/test` - Test trades from settings UI (no auth required, **respects symbol enable/disable**)
 - `/api/trading/close` - Manual position closing
 - `/api/trading/positions` - Query open positions from Drift
+- `/api/trading/sync-positions` - **Re-sync Position Manager with actual Drift positions** (no auth, for recovery from partial fills/restarts)
 - `/api/settings` - Get/update config (writes to .env file, **includes per-symbol settings**)
 - `/api/analytics/last-trade` - Fetch most recent trade details for dashboard (includes quality score)
 - `/api/analytics/version-comparison` - Compare performance across signal quality logic versions (v1/v2/v3)
@@ -486,6 +487,16 @@ trade.realizedPnL += actualRealizedPnL  // NOT: result.realizedPnL from SDK
    - Result: Win rate improved from 43.8% to 55.6%, profit per trade +86%
    - Implementation: `lib/trading/signal-quality.ts` checks both conditions before price position scoring

+23. **Position Manager sync issues:** Partial fills from on-chain orders can cause Position Manager to lose tracking:
+    - Symptom: Database shows position "closed", but Drift shows position still open without stop loss
+    - Cause: On-chain orders partially fill (0.29 SOL × 3 times), Position Manager closes database record, but remainder stays open
+    - Impact: Remaining position has NO software-based stop loss protection (only relies on on-chain orders)
+    - Solution: Use `/api/trading/sync-positions` endpoint to re-sync Position Manager with actual Drift positions
+    - Access: Settings UI "Sync Positions" button (orange), or CLI `scripts/sync-positions.sh`
+    - When: After manual Telegram trades, bot restarts, rate limiting issues, or suspected tracking loss
+    - Recovery: Endpoint fetches actual Drift positions, re-adds missing ones to Position Manager with calculated TP/SL
+    - Documentation: See `docs/guides/POSITION_SYNC_GUIDE.md` for details
+
 ## File Conventions

 - **API routes:** `app/api/[feature]/[action]/route.ts` (Next.js 15 App Router)
--- a/POSITION_SYNC_QUICK_REF.md
+++ b/POSITION_SYNC_QUICK_REF.md
@@ -0,0 +1,61 @@
+# Position Sync - Quick Reference
+
+## 🚨 When to Use
+- Position open on Drift but Position Manager shows 0 trades
+- Database says "closed" but Drift shows position still open  
+- After manual Telegram trades with partial fills
+- Bot restart lost in-memory tracking
+- Rate limiting (429 errors) disrupted monitoring
+
+## ✅ Three Ways to Sync
+
+### 1. Settings UI (Easiest)
+1. Go to http://localhost:3001/settings
+2. Click the orange **"🔄 Sync Positions"** button (next to Restart Bot)
+3. View results in green success message
+
+### 2. Terminal Script
+```bash
+cd /home/icke/traderv4
+bash scripts/sync-positions.sh
+```
+
+### 3. Direct API Call
+```bash
+source /home/icke/traderv4/.env
+curl -X POST http://localhost:3001/api/trading/sync-positions \
+  -H "Authorization: Bearer ${API_SECRET_KEY}"
+```
+
+## 📊 What It Does
+
+**Fetches** all open positions from Drift (SOL-PERP, BTC-PERP, ETH-PERP)
+
+**Compares** against Position Manager's tracked trades
+
+**Removes** tracking for positions closed externally
+
+**Adds** tracking for unmonitored positions with:
+- Stop loss at configured %
+- TP1/TP2 at configured %
+- Emergency stop protection
+- Trailing stop (if TP2 hit)
+- MAE/MFE tracking
+
+**Result**: Dual-layer protection restored ✅
+
+## 🎯 Your Current Situation
+
+- **Before Sync:** 4.93 SOL SHORT open, NO software protection
+- **After Sync:** Position Manager monitors it every 2s with full TP/SL system
+
+## ⚠️ Limitations
+
+- Entry time unknown (assumes 1 hour ago - doesn't affect TP/SL)
+- Signal quality metrics missing (only matters for scaling feature)
+- Uses current config (not original config from when trade opened)
+- Synthetic position ID (manual-{timestamp} instead of real TX)
+
+## 📖 Full Documentation
+
+See: `docs/guides/POSITION_SYNC_GUIDE.md`
--- a/app/api/trading/sync-positions/route.ts
+++ b/app/api/trading/sync-positions/route.ts
@@ -0,0 +1,193 @@
+/**
+ * Sync Positions API Endpoint
+ * 
+ * Re-synchronizes Position Manager with actual Drift positions
+ * Useful when:
+ * - Partial fills cause tracking issues
+ * - Bot restarts and loses in-memory state
+ * - Manual interventions on Drift
+ * - Database gets out of sync
+ * 
+ * POST /api/trading/sync-positions
+ */
+
+import { NextRequest, NextResponse } from 'next/server'
+import { initializeDriftService, getDriftService } from '@/lib/drift/client'
+import { getInitializedPositionManager } from '@/lib/trading/position-manager'
+import { getPrismaClient } from '@/lib/database/trades'
+import { getMergedConfig } from '@/config/trading'
+
+export async function POST(request: NextRequest): Promise<NextResponse> {
+  try {
+    console.log('🔄 Position sync requested...')
+
+    const config = getMergedConfig()
+    const driftService = await getDriftService()
+    const positionManager = await getInitializedPositionManager()
+    const prisma = getPrismaClient()
+
+    // Get all current Drift positions
+    const driftPositions = await driftService.getAllPositions()
+    console.log(`📊 Found ${driftPositions.length} positions on Drift`)
+
+    // Get all currently tracked positions
+    const trackedTrades = Array.from(positionManager.getActiveTrades().values())
+    console.log(`📋 Position Manager tracking ${trackedTrades.length} trades`)
+
+    const syncResults = {
+      drift_positions: driftPositions.length,
+      tracked_positions: trackedTrades.length,
+      added: [] as string[],
+      removed: [] as string[],
+      unchanged: [] as string[],
+      errors: [] as string[],
+    }
+
+    // Step 1: Remove tracked positions that don't exist on Drift
+    for (const trade of trackedTrades) {
+      const existsOnDrift = driftPositions.some(p => p.symbol === trade.symbol)
+      
+      if (!existsOnDrift) {
+        console.log(`🗑️ Removing ${trade.symbol} (not on Drift)`)
+        await positionManager.removeTrade(trade.id)
+        syncResults.removed.push(trade.symbol)
+        
+        // Mark as closed in database
+        try {
+          await prisma.trade.update({
+            where: { positionId: trade.positionId },
+            data: {
+              status: 'closed',
+              exitReason: 'sync_cleanup',
+              exitTime: new Date(),
+            },
+          })
+        } catch (dbError) {
+          console.error(`❌ Failed to update database for ${trade.symbol}:`, dbError)
+        }
+      } else {
+        syncResults.unchanged.push(trade.symbol)
+      }
+    }
+
+    // Step 2: Add Drift positions that aren't being tracked
+    for (const driftPos of driftPositions) {
+      const isTracked = trackedTrades.some(t => t.symbol === driftPos.symbol)
+      
+      if (!isTracked) {
+        console.log(`➕ Adding ${driftPos.symbol} to Position Manager`)
+        
+        try {
+          // Get current oracle price for this market
+          const currentPrice = await driftService.getOraclePrice(driftPos.marketIndex)
+          
+          // Calculate targets based on current config
+          const direction = driftPos.side
+          const entryPrice = driftPos.entryPrice
+          
+          // Calculate TP/SL prices
+          const calculatePrice = (entry: number, percent: number, dir: 'long' | 'short') => {
+            if (dir === 'long') {
+              return entry * (1 + percent / 100)
+            } else {
+              return entry * (1 - percent / 100)
+            }
+          }
+          
+          const stopLossPrice = calculatePrice(entryPrice, config.stopLossPercent, direction)
+          const tp1Price = calculatePrice(entryPrice, config.takeProfit1Percent, direction)
+          const tp2Price = calculatePrice(entryPrice, config.takeProfit2Percent, direction)
+          const emergencyStopPrice = calculatePrice(entryPrice, config.emergencyStopPercent, direction)
+
+          // Calculate position size in USD
+          const positionSizeUSD = driftPos.size * currentPrice
+
+          // Create ActiveTrade object
+          const activeTrade = {
+            id: `sync-${Date.now()}-${driftPos.symbol}`,
+            positionId: `manual-${Date.now()}`, // Synthetic ID since we don't have the original
+            symbol: driftPos.symbol,
+            direction: direction,
+            entryPrice: entryPrice,
+            entryTime: Date.now() - (60 * 60 * 1000), // Assume 1 hour ago (we don't know actual time)
+            positionSize: positionSizeUSD,
+            leverage: config.leverage,
+            stopLossPrice: stopLossPrice,
+            tp1Price: tp1Price,
+            tp2Price: tp2Price,
+            emergencyStopPrice: emergencyStopPrice,
+            currentSize: positionSizeUSD,
+            tp1Hit: false,
+            tp2Hit: false,
+            slMovedToBreakeven: false,
+            slMovedToProfit: false,
+            trailingStopActive: false,
+            realizedPnL: 0,
+            unrealizedPnL: driftPos.unrealizedPnL,
+            peakPnL: driftPos.unrealizedPnL,
+            peakPrice: currentPrice,
+            maxFavorableExcursion: 0,
+            maxAdverseExcursion: 0,
+            maxFavorablePrice: currentPrice,
+            maxAdversePrice: currentPrice,
+            originalAdx: undefined,
+            timesScaled: 0,
+            totalScaleAdded: 0,
+            atrAtEntry: undefined,
+            runnerTrailingPercent: undefined,
+            priceCheckCount: 0,
+            lastPrice: currentPrice,
+            lastUpdateTime: Date.now(),
+          }
+
+          await positionManager.addTrade(activeTrade)
+          syncResults.added.push(driftPos.symbol)
+          
+          console.log(`✅ Added ${driftPos.symbol} to monitoring`)
+        } catch (error) {
+          console.error(`❌ Failed to add ${driftPos.symbol}:`, error)
+          syncResults.errors.push(`${driftPos.symbol}: ${error instanceof Error ? error.message : 'Unknown error'}`)
+        }
+      }
+    }
+
+    const summary = {
+      success: true,
+      message: 'Position sync complete',
+      results: syncResults,
+      details: {
+        drift_positions: driftPositions.map(p => ({
+          symbol: p.symbol,
+          direction: p.side,
+          size: p.size,
+          entry: p.entryPrice,
+          pnl: p.unrealizedPnL,
+        })),
+        now_tracking: Array.from(positionManager.getActiveTrades().values()).map(t => ({
+          symbol: t.symbol,
+          direction: t.direction,
+          entry: t.entryPrice,
+        })),
+      },
+    }
+
+    console.log('✅ Position sync complete')
+    console.log(`   Added: ${syncResults.added.length}`)
+    console.log(`   Removed: ${syncResults.removed.length}`)
+    console.log(`   Unchanged: ${syncResults.unchanged.length}`)
+    console.log(`   Errors: ${syncResults.errors.length}`)
+
+    return NextResponse.json(summary)
+
+  } catch (error) {
+    console.error('❌ Position sync error:', error)
+    return NextResponse.json(
+      {
+        success: false,
+        error: 'Internal server error',
+        message: error instanceof Error ? error.message : 'Unknown error',
+      },
+      { status: 500 }
+    )
+  }
+}
--- a/app/settings/page.tsx
+++ b/app/settings/page.tsx
@@ -125,6 +125,33 @@ export default function SettingsPage() {
    setRestarting(false)
  }

+  const syncPositions = async () => {
+    setLoading(true)
+    setMessage(null)
+    try {
+      const response = await fetch('/api/trading/sync-positions', {
+        method: 'POST',
+      })
+      
+      const data = await response.json()
+      
+      if (data.success) {
+        const { results } = data
+        let msg = '✅ Position sync complete! '
+        if (results.added.length > 0) msg += `Added: ${results.added.join(', ')}. `
+        if (results.removed.length > 0) msg += `Removed: ${results.removed.join(', ')}. `
+        if (results.unchanged.length > 0) msg += `Already tracking: ${results.unchanged.join(', ')}. `
+        if (results.errors.length > 0) msg += `⚠️ Errors: ${results.errors.length}`
+        setMessage({ type: 'success', text: msg })
+      } else {
+        setMessage({ type: 'error', text: `Sync failed: ${data.error || data.message}` })
+      }
+    } catch (error) {
+      setMessage({ type: 'error', text: `Sync failed: ${error instanceof Error ? error.message : 'Unknown error'}` })
+    }
+    setLoading(false)
+  }
+
  const testTrade = async (direction: 'long' | 'short', symbol: string = 'SOLUSDT') => {
    if (!confirm(`⚠️ This will execute a REAL ${direction.toUpperCase()} trade on ${symbol} with current settings. Continue?`)) {
      return
@@ -793,6 +820,14 @@ export default function SettingsPage() {
            >
              {restarting ? '🔄 Restarting...' : '🔄 Restart Bot'}
            </button>
+            <button
+              onClick={syncPositions}
+              disabled={loading}
+              className="flex-1 bg-gradient-to-r from-orange-500 to-red-500 text-white font-bold py-4 px-6 rounded-lg hover:from-orange-600 hover:to-red-600 transition-all disabled:opacity-50 disabled:cursor-not-allowed"
+              title="Re-sync Position Manager with actual Drift positions"
+            >
+              {loading ? '🔄 Syncing...' : '🔄 Sync Positions'}
+            </button>
            <button
              onClick={loadSettings}
              className="bg-slate-700 text-white font-bold py-4 px-6 rounded-lg hover:bg-slate-600 transition-all"
--- a/docs/guides/POSITION_SYNC_GUIDE.md
+++ b/docs/guides/POSITION_SYNC_GUIDE.md
@@ -0,0 +1,128 @@
+# Position Re-Sync Feature
+
+## Problem Solved
+
+When manual Telegram trades are partially closed by on-chain orders, the Position Manager can lose tracking of the remaining position. This leaves the position without software-based stop loss protection, creating risk.
+
+## Solution
+
+Created `/api/trading/sync-positions` endpoint that:
+1. Fetches all actual open positions from Drift
+2. Compares against Position Manager's tracked trades
+3. Removes tracking for positions that don't exist on Drift (cleanup)
+4. Adds tracking for positions that exist on Drift but aren't being monitored
+
+## Usage
+
+### Via UI (Settings Page)
+1. Go to http://localhost:3001/settings
+2. Click "🔄 Sync Positions" button (orange button next to Restart Bot)
+3. View sync results in success message
+
+### Via Terminal
+```bash
+cd /home/icke/traderv4
+bash scripts/sync-positions.sh
+```
+
+### Via API
+```bash
+curl -X POST http://localhost:3001/api/trading/sync-positions \
+  -H "Authorization: Bearer $API_SECRET_KEY" \
+  | jq '.'
+```
+
+## Response Format
+
+```json
+{
+  "success": true,
+  "message": "Position sync complete",
+  "results": {
+    "drift_positions": 1,
+    "tracked_positions": 0,
+    "added": ["SOL-PERP"],
+    "removed": [],
+    "unchanged": [],
+    "errors": []
+  },
+  "details": {
+    "drift_positions": [
+      {
+        "symbol": "SOL-PERP",
+        "direction": "short",
+        "size": 4.93,
+        "entry": 167.38,
+        "pnl": 8.15
+      }
+    ],
+    "now_tracking": [
+      {
+        "symbol": "SOL-PERP",
+        "direction": "short",
+        "entry": 167.38
+      }
+    ]
+  }
+}
+```
+
+## When to Use
+
+- **After Telegram manual trades** - If position remains open but isn't being tracked
+- **After bot restarts** - If Position Manager lost in-memory state
+- **After partial fills** - When on-chain orders close position in chunks
+- **Rate limiting issues** - If 429 errors prevented proper monitoring
+- **Manual interventions** - If you modified position directly on Drift
+
+## How It Works
+
+1. Queries Drift for all open positions (SOL-PERP, BTC-PERP, ETH-PERP)
+2. Gets current oracle price for each position
+3. Calculates TP/SL targets based on current config
+4. Creates ActiveTrade objects with synthetic IDs (since we don't know original TX)
+5. Adds to Position Manager for monitoring
+6. Position Manager then protects position with emergency stop, trailing stop, etc.
+
+## Limitations
+
+- **Entry time unknown** - Assumes position opened 1 hour ago (doesn't affect TP/SL logic)
+- **Signal quality metrics missing** - No ATR/ADX data (only matters for scaling)
+- **Original config unknown** - Uses current config, not config when trade opened
+- **Synthetic position ID** - Uses `manual-{timestamp}` instead of actual TX signature
+
+## Safety Notes
+
+- No auth required (same as test endpoint) - internal use only
+- Won't open new positions - only adds tracking for existing ones
+- Cleans up tracking for positions that were closed externally
+- Marks cleaned positions as "sync_cleanup" in database
+
+## Files Changed
+
+1. `app/api/trading/sync-positions/route.ts` - Main endpoint
+2. `app/settings/page.tsx` - Added UI button and sync function
+3. `scripts/sync-positions.sh` - CLI helper script
+
+## Example Scenario (Today's Issue)
+
+**Before Sync:**
+- Drift: 4.93 SOL SHORT position open at $167.38
+- Position Manager: 0 active trades
+- Database: Position marked as "closed"
+- Result: NO STOP LOSS PROTECTION ⚠️
+
+**After Sync:**
+- Position Manager detects 4.93 SOL SHORT
+- Calculates SL at $168.89 (-0.9%)
+- Calculates TP1 at $166.71 (+0.4%)
+- Starts monitoring every 2 seconds
+- Result: DUAL-LAYER PROTECTION RESTORED ✅
+
+## Future Improvements
+
+- Auto-sync on startup (currently manual only)
+- Periodic auto-sync every N minutes
+- Alert if positions drift out of sync
+- Restore original signal quality metrics from database
+- Better handling of partial fill history
--- a/scripts/sync-positions.sh
+++ b/scripts/sync-positions.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+# 
+# Sync Position Manager with actual Drift positions
+# Useful when things get out of sync (partial fills, restarts, manual trades)
+#
+
+source /home/icke/traderv4/.env
+
+curl -X POST http://localhost:3001/api/trading/sync-positions \
+  -H "Authorization: Bearer ${API_SECRET_KEY}" \
+  -H "Content-Type: application/json" \
+  | jq '.'