trading_bot_v4/docs/history/PHANTOM_TRADE_DETECTION.md

Phantom Trade Detection & Prevention

Date: November 4, 2025
Issue: SOL-PERP SHORT position showed as opened in Telegram but no actual position existed on Drift

Problem Description

When a SHORT signal arrived after a LONG position:

1. Bot closed LONG successfully (manual exit)
2. Bot attempted to open SHORT for $2,100
3. Oracle price was $166.79 but actual market price was $158.51 (-5% discrepancy!)
4. Drift rejected or partially filled the order (only 0.05 SOL = $8 instead of 12.59 SOL = $2,100)
5. Position Manager detected size mismatch and marked as "phantom trade" with $0 P&L
6. No actual SHORT position existed on Drift

Root Cause

Oracle price lag during volatile market movement:

• During signal flip, the market moved significantly
• Oracle price hadn't updated to reflect actual market price
• Drift rejected/partially filled order due to excessive price discrepancy
• Transaction was confirmed on-chain but position was tiny/nonexistent

Solution Implemented

1. Enhanced Post-Entry Position Validation ✅

Modified openPosition() in /lib/drift/orders.ts:

• After position opens, verify actual size vs expected size
• Flag as "phantom" if actual size < 50% of expected
• Return isPhantom flag and actualSizeUSD in result

export interface OpenPositionResult {
  success: boolean
  transactionSignature?: string
  fillPrice?: number
  fillSize?: number
  slippage?: number
  error?: string
  isPhantom?: boolean         // NEW: Position opened but size mismatch
  actualSizeUSD?: number      // NEW: Actual position size from Drift
}

2. Phantom Trade Database Tracking 📊

Added new fields to Trade model in Prisma schema:

status           String   @default("open") // "open", "closed", "failed", "phantom"
isPhantom        Boolean  @default(false)
expectedSizeUSD  Float?
actualSizeUSD    Float?
phantomReason    String?  // "ORACLE_PRICE_MISMATCH", "PARTIAL_FILL", "ORDER_REJECTED"

Why track phantom trades:

• Measure how often this happens
• Analyze conditions that cause phantoms (volatility, time of day, etc.)
• Optimize entry logic based on data
• Provide transparency in trade history

3. Immediate Phantom Detection in Execute Endpoint 🚨

Modified /app/api/trading/execute/route.ts:

• After openPosition() returns, check isPhantom flag
• If phantom detected:
  • Save to database with status: 'phantom' and all metrics
  • Log detailed error with expected vs actual size
  • Return 500 error (prevents adding to Position Manager)
  • NO cleanup needed (tiny position ignored, will auto-close eventually)

if (openResult.isPhantom) {
  console.error(`🚨 PHANTOM TRADE DETECTED - Not adding to Position Manager`)
  
  // Save for analysis
  await createTrade({
    ...params,
    status: 'phantom',
    isPhantom: true,
    expectedSizeUSD: positionSizeUSD,
    actualSizeUSD: openResult.actualSizeUSD,
    phantomReason: 'ORACLE_PRICE_MISMATCH',
  })
  
  return NextResponse.json({
    success: false,
    error: 'Phantom trade detected',
    message: 'Oracle price mismatch - position not opened correctly'
  }, { status: 500 })
}

4. What We Did NOT Implement ❌

Based on user preferences:

• ❌ 20-minute cooldown: Too long, defeats purpose of flips
• ✅ Keep 1-minute cooldown: Already configured
• ✅ Use quality score: Already implemented in check-risk endpoint
• ❌ Pre-entry oracle validation: Not needed - post-entry detection is sufficient and catches the actual problem
• ❌ Auto-close phantom positions: Not needed - tiny positions ignored

How It Works Now

Normal Trade Flow:

1. Signal arrives → Check risk (quality score, cooldown, duplicates)
2. Open position → Verify size matches expected
3. If size OK → Place exit orders, add to Position Manager
4. Monitor and exit normally

Phantom Trade Flow:

1. Signal arrives → Check risk ✅
2. Open position → Size mismatch detected! 🚨
3. Save phantom trade to database 💾
4. Return error, DO NOT add to Position Manager ❌
5. Tiny position on Drift ignored (will expire/auto-close)

Database Analysis Queries

-- Count phantom trades
SELECT COUNT(*) FROM "Trade" WHERE "isPhantom" = true;

-- Phantom trades by symbol
SELECT symbol, COUNT(*) as phantom_count, AVG("expectedSizeUSD") as avg_expected, AVG("actualSizeUSD") as avg_actual
FROM "Trade" 
WHERE "isPhantom" = true 
GROUP BY symbol;

-- Phantom trades by time of day (UTC)
SELECT EXTRACT(HOUR FROM "createdAt") as hour, COUNT(*) as phantom_count
FROM "Trade"
WHERE "isPhantom" = true
GROUP BY hour
ORDER BY hour;

-- Phantom trades with quality scores
SELECT "signalQualityScore", COUNT(*) as count, AVG("atrAtEntry") as avg_atr
FROM "Trade"
WHERE "isPhantom" = true
GROUP BY "signalQualityScore"
ORDER BY "signalQualityScore" DESC;

Expected Behavior

Telegram Notifications:

• If phantom detected, execute endpoint returns 500 error
• n8n workflow should catch this and send error notification
• User sees: "Trade failed: Phantom trade detected"
• NO "Position monitored" message

Dashboard:

• Phantom trades appear in database with status: 'phantom'
• Can be filtered out or analyzed separately
• Shows expected vs actual size for debugging

Position Manager:

• Phantom trades are NEVER added to Position Manager
• No monitoring, no false alarms
• No "closed externally" spam in logs

Prevention Strategy

Going forward, phantom trades should be rare because:

1. 1-minute cooldown prevents rapid flips during volatility
2. Quality score filtering blocks low-quality signals (which tend to occur during chaos)
3. Post-entry validation catches phantoms immediately
4. Database tracking allows us to analyze patterns and adjust

If phantom trades continue to occur frequently, we can:

• Increase cooldown for flips (2-3 minutes)
• Add ATR-based volatility check (block flips when ATR > threshold)
• Implement pre-entry oracle validation (add 2% discrepancy check before placing order)

Files Modified

• lib/drift/orders.ts - Added phantom detection in openPosition()
• app/api/trading/execute/route.ts - Added phantom handling after opening
• lib/database/trades.ts - Added phantom fields to CreateTradeParams
• prisma/schema.prisma - Added phantom trade fields to Trade model
• prisma/migrations/20251104091741_add_phantom_trade_fields/ - Database migration

Testing

To test phantom detection:

1. Modify openPosition() to simulate phantom (set actualSizeUSD = 10)
2. Send test trade signal
3. Verify:
   • Error returned from execute endpoint
   • Phantom trade saved to database with isPhantom: true
   • NO position added to Position Manager
   • Logs show "🚨 PHANTOM TRADE DETECTED"

Future Improvements

If phantom trades remain an issue:

1. Auto-retry with delay: Wait 5s for oracle to catch up, retry once
2. Oracle price validation: Check Pyth price vs Drift oracle before placing order
3. Volatility-based cooldown: Longer cooldown during high ATR periods
4. Symbol-specific thresholds: SOL might need different validation than ETH