feat: add quality score display and timezone fixes

- Add qualityScore to ExecuteTradeResponse interface and response object - Update analytics page to always show Signal Quality card (N/A if unavailable) - Fix n8n workflow to pass context metrics and qualityScore to execute endpoint - Fix timezone in Telegram notifications (Europe/Berlin) - Fix symbol normalization in /api/trading/close endpoint - Update Drift ETH-PERP minimum order size (0.002 ETH not 0.01) - Add transaction confirmation to closePosition() to prevent phantom closes - Add 30-second grace period for new trades in Position Manager - Fix execution order: database save before Position Manager.addTrade() - Update copilot instructions with transaction confirmation pattern
2025-11-01 17:00:37 +01:00
parent 7788327a4e
commit 056440bf8f
9 changed files with 110 additions and 49 deletions
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -34,6 +34,7 @@ await positionManager.addTrade(activeTrade)
 - Closes positions via `closePosition()` market orders when targets hit
 - Acts as backup if on-chain orders don't fill
 - State persistence: Saves to database, restores on restart via `configSnapshot.positionManagerState`
+- **Grace period for new trades:** Skips "external closure" detection for positions <30 seconds old (Drift positions take 5-10s to propagate)

 ### 2. Drift Client (`lib/drift/client.ts`)
 **Purpose:** Solana/Drift Protocol SDK wrapper for order execution
@@ -47,7 +48,25 @@ const health = await driftService.getAccountHealth()
 **Wallet handling:** Supports both JSON array `[91,24,...]` and base58 string formats from Phantom wallet

 ### 3. Order Placement (`lib/drift/orders.ts`)
-**Critical function:** `placeExitOrders()` - places TP/SL orders on-chain
+**Critical functions:** 
+- `openPosition()` - Opens market position with transaction confirmation
+- `closePosition()` - Closes position with transaction confirmation
+- `placeExitOrders()` - Places TP/SL orders on-chain
+
+**CRITICAL: Transaction Confirmation Pattern**
+Both `openPosition()` and `closePosition()` MUST confirm transactions on-chain:
+```typescript
+const txSig = await driftClient.placePerpOrder(orderParams)
+console.log('⏳ Confirming transaction on-chain...')
+const connection = driftService.getConnection()
+const confirmation = await connection.confirmTransaction(txSig, 'confirmed')
+
+if (confirmation.value.err) {
+  throw new Error(`Transaction failed: ${JSON.stringify(confirmation.value.err)}`)
+}
+console.log('✅ Transaction confirmed on-chain')
+```
+Without this, the SDK returns signatures for transactions that never execute, causing phantom trades/closes.

 **Dual Stop System** (USE_DUAL_STOPS=true):
 ```typescript
@@ -253,7 +272,7 @@ docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\dt"

 2. **Wrong DATABASE_URL:** Container runtime needs `trading-bot-postgres`, Prisma CLI from host needs `localhost:5432`

-3. **Symbol format mismatch:** Always normalize with `normalizeTradingViewSymbol()` before calling Drift
+3. **Symbol format mismatch:** Always normalize with `normalizeTradingViewSymbol()` before calling Drift (applies to ALL endpoints including `/api/trading/close`)

 4. **Missing reduce-only flag:** Exit orders without `reduceOnly: true` can accidentally open new positions

@@ -268,6 +287,24 @@ docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\dt"
   - `TAKE_PROFIT_2_SIZE_PERCENT=80` means "close 80% of REMAINING" (not of original)
   - Actual runner size = (100 - TP1%) × (100 - TP2%) / 100 = 5% with defaults

+9. **Transaction confirmation CRITICAL:** Both `openPosition()` AND `closePosition()` MUST call `connection.confirmTransaction()` after `placePerpOrder()`. Without this, the SDK returns transaction signatures that aren't confirmed on-chain, causing "phantom trades" or "phantom closes". Always check `confirmation.value.err` before proceeding.
+
+10. **Execution order matters:** When creating trades via API endpoints, the order MUST be:
+    1. Open position + place exit orders
+    2. Save to database (`createTrade()`)
+    3. Add to Position Manager (`positionManager.addTrade()`)
+    
+    If Position Manager is added before database save, race conditions occur where monitoring checks before the trade exists in DB.
+
+11. **New trade grace period:** Position Manager skips "external closure" detection for trades <30 seconds old because Drift positions take 5-10 seconds to propagate after opening. Without this grace period, new positions are immediately detected as "closed externally" and cancelled.
+
+12. **Drift minimum position sizes:** Actual minimums differ from documentation:
+    - SOL-PERP: 0.1 SOL (~$5-15 depending on price)
+    - ETH-PERP: 0.002 ETH (~$7-8 at $4000/ETH) - NOT 0.01 ETH
+    - BTC-PERP: 0.0001 BTC (~$10-12 at $100k/BTC)
+    
+    Always calculate: `minOrderSize × currentPrice` must exceed Drift's $4 minimum. Add buffer for price movement.
+
 ## File Conventions

 - **API routes:** `app/api/[feature]/[action]/route.ts` (Next.js 15 App Router)
--- a/app/analytics/page.tsx
+++ b/app/analytics/page.tsx
@@ -313,28 +313,27 @@ export default function AnalyticsPage() {
                  </div>
                </div>
                
-                {lastTrade.signalQualityScore !== undefined ? (
-                  <div className="bg-gray-700/30 rounded-lg p-4">
-                    <div className="text-sm text-gray-400 mb-1">Signal Quality</div>
-                    <div className={`text-xl font-bold ${lastTrade.signalQualityScore >= 80 ? 'text-green-400' : lastTrade.signalQualityScore >= 70 ? 'text-yellow-400' : 'text-orange-400'}`}>
-                      {lastTrade.signalQualityScore}/100
-                    </div>
-                    <div className="text-xs text-gray-500">
-                      {lastTrade.signalQualityScore >= 80 ? 'Excellent' : lastTrade.signalQualityScore >= 70 ? 'Good' : 'Marginal'}
-                    </div>
-                  </div>
-                ) : lastTrade.exitTime && lastTrade.exitPrice ? (
-                  <div className="bg-gray-700/30 rounded-lg p-4">
-                    <div className="text-sm text-gray-400 mb-1">Exit</div>
-                    <div className="text-xl font-bold text-white">${lastTrade.exitPrice.toFixed(4)}</div>
-                    <div className="text-xs text-gray-500">
-                      {new Date(lastTrade.exitTime).toLocaleString()}
-                    </div>
-                  </div>
-                ) : null}
+                <div className="bg-gray-700/30 rounded-lg p-4">
+                  <div className="text-sm text-gray-400 mb-1">Signal Quality</div>
+                  {lastTrade.signalQualityScore !== undefined ? (
+                    <>
+                      <div className={`text-xl font-bold ${lastTrade.signalQualityScore >= 80 ? 'text-green-400' : lastTrade.signalQualityScore >= 70 ? 'text-yellow-400' : 'text-orange-400'}`}>
+                        {lastTrade.signalQualityScore}/100
+                      </div>
+                      <div className="text-xs text-gray-500">
+                        {lastTrade.signalQualityScore >= 80 ? 'Excellent' : lastTrade.signalQualityScore >= 70 ? 'Good' : 'Marginal'}
+                      </div>
+                    </>
+                  ) : (
+                    <>
+                      <div className="text-xl font-bold text-gray-500">N/A</div>
+                      <div className="text-xs text-gray-500">No score available</div>
+                    </>
+                  )}
+                </div>
              </div>
              
-              {lastTrade.exitTime && lastTrade.exitPrice && lastTrade.signalQualityScore !== undefined && (
+              {lastTrade.exitTime && lastTrade.exitPrice && (
                <div className="grid md:grid-cols-1 gap-4 mb-4">
                  <div className="bg-gray-700/30 rounded-lg p-4">
                    <div className="text-sm text-gray-400 mb-1">Exit</div>
--- a/app/api/trading/close/route.ts
+++ b/app/api/trading/close/route.ts
@@ -7,12 +7,13 @@
 import { NextRequest, NextResponse } from 'next/server'
 import { closePosition } from '@/lib/drift/orders'
 import { initializeDriftService } from '@/lib/drift/client'
+import { normalizeTradingViewSymbol } from '@/config/trading'

 export const dynamic = 'force-dynamic'
 export const runtime = 'nodejs'

 interface CloseRequest {
-  symbol: string // e.g., 'SOL-PERP'
+  symbol: string // e.g., 'SOL-PERP' or 'SOLUSDT'
  percentToClose?: number // 0-100, default 100 (close entire position)
 }

@@ -46,14 +47,16 @@ export async function POST(request: NextRequest) {
      )
    }

-    console.log(`📊 Closing position: ${symbol} (${percentToClose}%)`)
+    // Normalize symbol (SOLUSDT -> SOL-PERP)
+    const driftSymbol = normalizeTradingViewSymbol(symbol)
+    console.log(`📊 Closing position: ${driftSymbol} (${percentToClose}%)`)

    // Initialize Drift service if not already initialized
    await initializeDriftService()

    // Close position
    const result = await closePosition({
-      symbol,
+      symbol: driftSymbol,
      percentToClose,
      slippageTolerance: 1.0,
    })
@@ -72,7 +75,7 @@ export async function POST(request: NextRequest) {
    return NextResponse.json({
      success: true,
      transactionSignature: result.transactionSignature,
-      symbol,
+      symbol: driftSymbol,
      closePrice: result.closePrice,
      closedSize: result.closedSize,
      realizedPnL: result.realizedPnL,
--- a/app/api/trading/execute/route.ts
+++ b/app/api/trading/execute/route.ts
@@ -25,6 +25,7 @@ export interface ExecuteTradeRequest {
  rsi?: number
  volumeRatio?: number
  pricePosition?: number
+  qualityScore?: number // Calculated by check-risk endpoint
 }

 export interface ExecuteTradeResponse {
@@ -43,6 +44,7 @@ export interface ExecuteTradeResponse {
  tp2Percent?: number
  entrySlippage?: number
  timestamp?: string
+  qualityScore?: number // Signal quality score (0-100)
  error?: string
  message?: string
 }
@@ -281,11 +283,6 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
      console.error('❌ Unexpected error placing exit orders:', err)
    }

-    // Add to position manager for monitoring AFTER orders are placed
-    await positionManager.addTrade(activeTrade)
-    
-    console.log('✅ Trade added to position manager for monitoring')
-
    // Create response object
    const response: ExecuteTradeResponse = {
      success: true,
@@ -303,6 +300,7 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
      tp2Percent: config.takeProfit2Percent,
      entrySlippage: openResult.slippage,
      timestamp: new Date().toISOString(),
+      qualityScore: body.qualityScore, // Include quality score in response
    }

    // Attach exit order signatures to response
@@ -341,6 +339,7 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
        rsiAtEntry: body.rsi,
        volumeAtEntry: body.volumeRatio,
        pricePositionAtEntry: body.pricePosition,
+        signalQualityScore: body.qualityScore,
      })
      
      console.log('💾 Trade saved to database')
@@ -349,6 +348,11 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
      // Don't fail the trade if database save fails
    }

+    // NOW add to position manager for monitoring (after database save)
+    await positionManager.addTrade(activeTrade)
+    
+    console.log('✅ Trade added to position manager for monitoring')
+
    console.log('✅ Trade executed successfully!')
    
    return NextResponse.json(response)
--- a/app/api/trading/test/route.ts
+++ b/app/api/trading/test/route.ts
@@ -190,12 +190,6 @@ export async function POST(request: NextRequest): Promise<NextResponse<TestTrade
      lastUpdateTime: Date.now(),
    }

-    // Add to position manager for monitoring
-    const positionManager = await getInitializedPositionManager()
-    await positionManager.addTrade(activeTrade)
-    
-    console.log('✅ Trade added to position manager for monitoring')
-
    // Create response object
    const response: TestTradeResponse = {
      success: true,
@@ -248,7 +242,7 @@ export async function POST(request: NextRequest): Promise<NextResponse<TestTrade
      console.error('❌ Unexpected error placing exit orders:', err)
    }

-    // Save trade to database
+    // Save trade to database FIRST (before Position Manager)
    try {
      await createTrade({
        positionId: openResult.transactionSignature!,
@@ -281,6 +275,12 @@ export async function POST(request: NextRequest): Promise<NextResponse<TestTrade
      // Don't fail the trade if database save fails
    }

+    // NOW add to position manager for monitoring (after database save)
+    const positionManager = await getInitializedPositionManager()
+    await positionManager.addTrade(activeTrade)
+    
+    console.log('✅ Trade added to position manager for monitoring')
+
    console.log('✅ Test trade executed successfully!')
    
    return NextResponse.json(response)
--- a/config/trading.ts
+++ b/config/trading.ts
@@ -127,11 +127,11 @@ export const SUPPORTED_MARKETS: Record<string, MarketConfig> = {
    symbol: 'ETH-PERP',
    driftMarketIndex: 2,
    pythPriceFeedId: '0xff61491a931112ddf1bd8147cd1b641375f79f5825126d665480874634fd0ace',
-    minOrderSize: 0.01, // 0.01 ETH minimum
+    minOrderSize: 0.002, // 0.002 ETH minimum (~$7-8 for safety above $4 Drift minimum)
    tickSize: 0.01,
-    // DATA COLLECTION MODE: Minimal risk (Drift minimum 0.01 ETH = ~$38)
-    positionSize: 40,  // $40 base capital (meets exchange minimum)
-    leverage: 1,       // 1x leverage = $40 total exposure
+    // DATA COLLECTION MODE: Minimal risk
+    positionSize: 8,   // $8 base capital (ensures 0.002 ETH at ~$4000/ETH)
+    leverage: 1,       // 1x leverage = $8 total exposure
  },
 }

--- a/lib/drift/orders.ts
+++ b/lib/drift/orders.ts
@@ -522,9 +522,17 @@ export async function closePosition(
    
    console.log(`✅ Close order placed! Transaction: ${txSig}`)
    
-    // Wait for confirmation (transaction is likely already confirmed by placeAndTakePerpOrder)
-    console.log('⏳ Waiting for transaction confirmation...')
-    console.log('✅ Transaction confirmed')
+    // CRITICAL: Confirm transaction on-chain to prevent phantom closes
+    console.log('⏳ Confirming transaction on-chain...')
+    const connection = driftService.getConnection()
+    const confirmation = await connection.confirmTransaction(txSig, 'confirmed')
+    
+    if (confirmation.value.err) {
+      console.error('❌ Transaction failed on-chain:', confirmation.value.err)
+      throw new Error(`Transaction failed: ${JSON.stringify(confirmation.value.err)}`)
+    }
+    
+    console.log('✅ Transaction confirmed on-chain')
    
    // Calculate realized P&L
    const pnlPerUnit = oraclePrice - position.entryPrice
--- a/lib/trading/position-manager.ts
+++ b/lib/trading/position-manager.ts
@@ -286,7 +286,17 @@ export class PositionManager {
      const marketConfig = getMarketConfig(trade.symbol)
      const position = await driftService.getPosition(marketConfig.driftMarketIndex)
      
+      // Calculate trade age in seconds
+      const tradeAgeSeconds = (Date.now() - trade.entryTime) / 1000
+      
      if (position === null || position.size === 0) {
+        // IMPORTANT: Skip "external closure" detection for NEW trades (<30 seconds old)
+        // Drift positions may not be immediately visible after opening due to blockchain delays
+        if (tradeAgeSeconds < 30) {
+          console.log(`⏳ Trade ${trade.symbol} is new (${tradeAgeSeconds.toFixed(1)}s old) - skipping external closure check`)
+          return // Skip this check cycle, position might still be propagating
+        }
+        
        // Position closed externally (by on-chain TP/SL order)
        console.log(`⚠️ Position ${trade.symbol} was closed externally (by on-chain order)`)
      } else {
--- a/workflows/trading/Money_Machine.json
+++ b/workflows/trading/Money_Machine.json
@@ -151,7 +151,7 @@
        },
        "sendBody": true,
        "specifyBody": "json",
-        "jsonBody": "={\n  \"symbol\": \"{{ $('Parse Signal').item.json.symbol }}\",\n  \"direction\": \"{{ $('Parse Signal').item.json.direction }}\",\n  \"timeframe\": \"{{ $('Parse Signal').item.json.timeframe }}\",\n  \"signalStrength\": \"strong\"\n}",
+        "jsonBody": "={\n  \"symbol\": \"{{ $('Parse Signal Enhanced').item.json.symbol }}\",\n  \"direction\": \"{{ $('Parse Signal Enhanced').item.json.direction }}\",\n  \"timeframe\": \"{{ $('Parse Signal Enhanced').item.json.timeframe }}\",\n  \"signalStrength\": \"strong\",\n  \"atr\": {{ $('Parse Signal Enhanced').item.json.atr }},\n  \"adx\": {{ $('Parse Signal Enhanced').item.json.adx }},\n  \"rsi\": {{ $('Parse Signal Enhanced').item.json.rsi }},\n  \"volumeRatio\": {{ $('Parse Signal Enhanced').item.json.volumeRatio }},\n  \"pricePosition\": {{ $('Parse Signal Enhanced').item.json.pricePosition }},\n  \"qualityScore\": {{ $('Check Risk').item.json.qualityScore }}\n}",
        "options": {
          "timeout": 120000
        }
@@ -197,7 +197,7 @@
          "values": [
            {
              "name": "message",
-              "stringValue": "={{ `🟢 TRADE OPENED\n\n📊 Symbol: ${$('Parse Signal').item.json.symbol}\n${$('Parse Signal').item.json.direction === 'long' ? '📈' : '📉'} Direction: ${$('Parse Signal').item.json.direction.toUpperCase()}\n\n💵 Position: $${$('Execute Trade').item.json.positionSize}\n⚡ Leverage: ${$('Execute Trade').item.json.leverage}x\n\n💰 Entry: $${$('Execute Trade').item.json.entryPrice.toFixed(4)}\n🎯 TP1: $${$('Execute Trade').item.json.takeProfit1.toFixed(4)} (${$('Execute Trade').item.json.tp1Percent}%)\n🎯 TP2: $${$('Execute Trade').item.json.takeProfit2.toFixed(4)} (${$('Execute Trade').item.json.tp2Percent}%)\n🛑 SL: $${$('Execute Trade').item.json.stopLoss.toFixed(4)} (${$('Execute Trade').item.json.stopLossPercent}%)\n\n⏰ ${$now.toFormat('HH:mm:ss')}\n✅ Position monitored` }}"
+              "stringValue": "={{ `🟢 TRADE OPENED\n\n📊 Symbol: ${$('Parse Signal Enhanced').item.json.symbol}\n${$('Parse Signal Enhanced').item.json.direction === 'long' ? '📈' : '📉'} Direction: ${$('Parse Signal Enhanced').item.json.direction.toUpperCase()}\n\n💵 Position: $${$('Execute Trade').item.json.positionSize}\n⚡ Leverage: ${$('Execute Trade').item.json.leverage}x${$('Execute Trade').item.json.qualityScore ? `\n⭐ Quality: ${$('Execute Trade').item.json.qualityScore}/100` : ''}\n\n💰 Entry: $${$('Execute Trade').item.json.entryPrice.toFixed(4)}\n🎯 TP1: $${$('Execute Trade').item.json.takeProfit1.toFixed(4)} (${$('Execute Trade').item.json.tp1Percent}%)\n🎯 TP2: $${$('Execute Trade').item.json.takeProfit2.toFixed(4)} (${$('Execute Trade').item.json.tp2Percent}%)\n🛑 SL: $${$('Execute Trade').item.json.stopLoss.toFixed(4)} (${$('Execute Trade').item.json.stopLossPercent}%)\n\n⏰ ${$now.setZone('Europe/Berlin').toFormat('HH:mm:ss')}\n✅ Position monitored` }}"
            }
          ]
        },
@@ -218,7 +218,7 @@
          "values": [
            {
              "name": "message",
-              "stringValue": "🔴 TRADE FAILED\\n\\n{{ $('Parse Signal').item.json.rawMessage }}\\n\\n❌ Error: {{ $json.error || $json.message }}\\n⏰ {{ $now.toFormat('HH:mm') }}"
+              "stringValue": "🔴 TRADE FAILED\\n\\n{{ $('Parse Signal').item.json.rawMessage }}\\n\\n❌ Error: {{ $json.error || $json.message }}\\n⏰ {{ $now.setZone('Europe/Berlin').toFormat('HH:mm') }}"
            }
          ]
        },
@@ -242,7 +242,7 @@
            {
              "id": "risk_message",
              "name": "message",
-              "value": "={{ '⚠️ TRADE BLOCKED\\n\\n' + $('Parse Signal Enhanced').item.json.rawMessage + '\\n\\n🛑 Reason: ' + $json.reason + '\\n📋 Details: ' + ($json.details || 'N/A') + '\\n\\n📊 Quality Score: ' + ($json.qualityScore || 'N/A') + '/100' + ($json.qualityReasons && $json.qualityReasons.length > 0 ? '\\n⚠️ Issues:\\n   • ' + $json.qualityReasons.join('\\n   • ') : '') + '\\n\\n⏰ ' + $now.format('HH:mm:ss') }}",
+              "value": "={{ '⚠️ TRADE BLOCKED\\n\\n' + $('Parse Signal Enhanced').item.json.rawMessage + '\\n\\n🛑 Reason: ' + $json.reason + '\\n📋 Details: ' + ($json.details || 'N/A') + '\\n\\n📊 Quality Score: ' + ($json.qualityScore || 'N/A') + '/100' + ($json.qualityReasons && $json.qualityReasons.length > 0 ? '\\n⚠️ Issues:\\n   • ' + $json.qualityReasons.join('\\n   • ') : '') + '\\n\\n⏰ ' + $now.setZone('Europe/Berlin').toFormat('HH:mm:ss') }}",
              "type": "string"
            }
          ]