CRITICAL FIX: Prevent unprotected positions via database-first pattern
Root Cause: - Execute endpoint saved to database AFTER adding to Position Manager - Database save failures were silently caught and ignored - API returned success even when DB save failed - Container restarts lost in-memory Position Manager state - Result: Unprotected positions with no TP/SL monitoring Fixes Applied: 1. Database-First Pattern (app/api/trading/execute/route.ts): - MOVED createTrade() BEFORE positionManager.addTrade() - If database save fails, return HTTP 500 with critical error - Error message: 'CLOSE POSITION MANUALLY IMMEDIATELY' - Position Manager only tracks database-persisted trades - Ensures container restarts can restore all positions 2. Transaction Timeout (lib/drift/orders.ts): - Added 30s timeout to confirmTransaction() in closePosition() - Prevents API from hanging during network congestion - Uses Promise.race() pattern for timeout enforcement 3. Telegram Error Messages (telegram_command_bot.py): - Parse JSON for ALL responses (not just 200 OK) - Extract detailed error messages from 'message' field - Shows critical warnings to user immediately - Fail-open: proceeds if analytics check fails 4. Position Manager (lib/trading/position-manager.ts): - Move lastPrice update to TOP of monitoring loop - Ensures /status endpoint always shows current price Verification: - Test trade cmhxj8qxl0000od076m21l58z executed successfully - Database save completed BEFORE Position Manager tracking - SL triggered correctly at -$4.21 after 15 minutes - All protection systems working as expected Impact: - Eliminates risk of unprotected positions - Provides immediate critical warnings if DB fails - Enables safe container restarts with full position recovery - Verified with live test trade on production See: CRITICAL_INCIDENT_UNPROTECTED_POSITION.md for full incident report
This commit is contained in:
@@ -506,39 +506,11 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
|
||||
console.error('❌ Unexpected error placing exit orders:', err)
|
||||
}
|
||||
|
||||
// Add to position manager for monitoring AFTER orders are placed
|
||||
await positionManager.addTrade(activeTrade)
|
||||
|
||||
console.log('✅ Trade added to position manager for monitoring')
|
||||
|
||||
// Create response object
|
||||
const response: ExecuteTradeResponse = {
|
||||
success: true,
|
||||
positionId: openResult.transactionSignature,
|
||||
symbol: driftSymbol,
|
||||
direction: body.direction,
|
||||
entryPrice: entryPrice,
|
||||
positionSize: positionSizeUSD,
|
||||
leverage: leverage, // Use actual symbol-specific leverage, not global config
|
||||
stopLoss: stopLossPrice,
|
||||
takeProfit1: tp1Price,
|
||||
takeProfit2: tp2Price,
|
||||
stopLossPercent: config.stopLossPercent,
|
||||
tp1Percent: config.takeProfit1Percent,
|
||||
tp2Percent: config.takeProfit2Percent,
|
||||
entrySlippage: openResult.slippage,
|
||||
timestamp: new Date().toISOString(),
|
||||
}
|
||||
|
||||
// Attach exit order signatures to response
|
||||
if (exitOrderSignatures.length > 0) {
|
||||
(response as any).exitOrderSignatures = exitOrderSignatures
|
||||
}
|
||||
|
||||
// Save trade to database
|
||||
// Save trade to database FIRST (CRITICAL: Must succeed before Position Manager)
|
||||
let qualityResult
|
||||
try {
|
||||
// Calculate quality score if metrics available
|
||||
const qualityResult = scoreSignalQuality({
|
||||
qualityResult = scoreSignalQuality({
|
||||
atr: body.atr || 0,
|
||||
adx: body.adx || 0,
|
||||
rsi: body.rsi || 0,
|
||||
@@ -584,8 +556,49 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
|
||||
console.log(`💾 Trade saved with quality score: ${qualityResult.score}/100`)
|
||||
console.log(`📊 Quality reasons: ${qualityResult.reasons.join(', ')}`)
|
||||
} catch (dbError) {
|
||||
console.error('❌ Failed to save trade to database:', dbError)
|
||||
// Don't fail the trade if database save fails
|
||||
console.error('❌ CRITICAL: Failed to save trade to database:', dbError)
|
||||
console.error(' Position is OPEN on Drift but NOT tracked!')
|
||||
console.error(' Manual intervention required - close position immediately')
|
||||
|
||||
// CRITICAL: If database save fails, we MUST NOT add to Position Manager
|
||||
// Return error to user so they know to close manually
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: 'Database save failed - position unprotected',
|
||||
message: `Position opened on Drift but database save failed. CLOSE POSITION MANUALLY IMMEDIATELY. Transaction: ${openResult.transactionSignature}`,
|
||||
},
|
||||
{ status: 500 }
|
||||
)
|
||||
}
|
||||
|
||||
// Add to position manager for monitoring ONLY AFTER database save succeeds
|
||||
await positionManager.addTrade(activeTrade)
|
||||
|
||||
console.log('✅ Trade added to position manager for monitoring')
|
||||
|
||||
// Create response object
|
||||
const response: ExecuteTradeResponse = {
|
||||
success: true,
|
||||
positionId: openResult.transactionSignature,
|
||||
symbol: driftSymbol,
|
||||
direction: body.direction,
|
||||
entryPrice: entryPrice,
|
||||
positionSize: positionSizeUSD,
|
||||
leverage: leverage, // Use actual symbol-specific leverage, not global config
|
||||
stopLoss: stopLossPrice,
|
||||
takeProfit1: tp1Price,
|
||||
takeProfit2: tp2Price,
|
||||
stopLossPercent: config.stopLossPercent,
|
||||
tp1Percent: config.takeProfit1Percent,
|
||||
tp2Percent: config.takeProfit2Percent,
|
||||
entrySlippage: openResult.slippage,
|
||||
timestamp: new Date().toISOString(),
|
||||
}
|
||||
|
||||
// Attach exit order signatures to response
|
||||
if (exitOrderSignatures.length > 0) {
|
||||
(response as any).exitOrderSignatures = exitOrderSignatures
|
||||
}
|
||||
|
||||
console.log('✅ Trade executed successfully!')
|
||||
|
||||
Reference in New Issue
Block a user