# Phantom Trade Detection & Prevention

**Date:** November 4, 2025  
**Issue:** SOL-PERP SHORT position showed as opened in Telegram but no actual position existed on Drift

## Problem Description

When a SHORT signal arrived after a LONG position:
1. Bot closed LONG successfully (manual exit)
2. Bot attempted to open SHORT for $2,100
3. **Oracle price was $166.79 but actual market price was $158.51** (-5% discrepancy!)
4. Drift rejected or partially filled the order (only 0.05 SOL = $8 instead of 12.59 SOL = $2,100)
5. Position Manager detected size mismatch and marked as "phantom trade" with $0 P&L
6. No actual SHORT position existed on Drift

## Root Cause

**Oracle price lag during volatile market movement:**
- During signal flip, the market moved significantly
- Oracle price hadn't updated to reflect actual market price
- Drift rejected/partially filled order due to excessive price discrepancy
- Transaction was confirmed on-chain but position was tiny/nonexistent

## Solution Implemented

### 1. **Enhanced Post-Entry Position Validation** ✅

Modified `openPosition()` in `/lib/drift/orders.ts`:
- After position opens, verify actual size vs expected size
- Flag as "phantom" if actual size < 50% of expected
- Return `isPhantom` flag and `actualSizeUSD` in result

```typescript
export interface OpenPositionResult {
  success: boolean
  transactionSignature?: string
  fillPrice?: number
  fillSize?: number
  slippage?: number
  error?: string
  isPhantom?: boolean         // NEW: Position opened but size mismatch
  actualSizeUSD?: number      // NEW: Actual position size from Drift
}
```

### 2. **Phantom Trade Database Tracking** 📊

Added new fields to `Trade` model in Prisma schema:
```prisma
status           String   @default("open") // "open", "closed", "failed", "phantom"
isPhantom        Boolean  @default(false)
expectedSizeUSD  Float?
actualSizeUSD    Float?
phantomReason    String?  // "ORACLE_PRICE_MISMATCH", "PARTIAL_FILL", "ORDER_REJECTED"
```

**Why track phantom trades:**
- Measure how often this happens
- Analyze conditions that cause phantoms (volatility, time of day, etc.)
- Optimize entry logic based on data
- Provide transparency in trade history

### 3. **Immediate Phantom Detection in Execute Endpoint** 🚨

Modified `/app/api/trading/execute/route.ts`:
- After `openPosition()` returns, check `isPhantom` flag
- If phantom detected:
  - Save to database with `status: 'phantom'` and all metrics
  - Log detailed error with expected vs actual size
  - Return 500 error (prevents adding to Position Manager)
  - NO cleanup needed (tiny position ignored, will auto-close eventually)

```typescript
if (openResult.isPhantom) {
  console.error(`🚨 PHANTOM TRADE DETECTED - Not adding to Position Manager`)
  
  // Save for analysis
  await createTrade({
    ...params,
    status: 'phantom',
    isPhantom: true,
    expectedSizeUSD: positionSizeUSD,
    actualSizeUSD: openResult.actualSizeUSD,
    phantomReason: 'ORACLE_PRICE_MISMATCH',
  })
  
  return NextResponse.json({
    success: false,
    error: 'Phantom trade detected',
    message: 'Oracle price mismatch - position not opened correctly'
  }, { status: 500 })
}
```

### 4. **What We Did NOT Implement** ❌

Based on user preferences:

- ❌ **20-minute cooldown:** Too long, defeats purpose of flips
- ✅ **Keep 1-minute cooldown:** Already configured
- ✅ **Use quality score:** Already implemented in check-risk endpoint
- ❌ **Pre-entry oracle validation:** Not needed - post-entry detection is sufficient and catches the actual problem
- ❌ **Auto-close phantom positions:** Not needed - tiny positions ignored

## How It Works Now

### Normal Trade Flow:
1. Signal arrives → Check risk (quality score, cooldown, duplicates)
2. Open position → Verify size matches expected
3. If size OK → Place exit orders, add to Position Manager
4. Monitor and exit normally

### Phantom Trade Flow:
1. Signal arrives → Check risk ✅
2. Open position → Size mismatch detected! 🚨
3. Save phantom trade to database 💾
4. Return error, DO NOT add to Position Manager ❌
5. Tiny position on Drift ignored (will expire/auto-close)

## Database Analysis Queries

```sql
-- Count phantom trades
SELECT COUNT(*) FROM "Trade" WHERE "isPhantom" = true;

-- Phantom trades by symbol
SELECT symbol, COUNT(*) as phantom_count, AVG("expectedSizeUSD") as avg_expected, AVG("actualSizeUSD") as avg_actual
FROM "Trade" 
WHERE "isPhantom" = true 
GROUP BY symbol;

-- Phantom trades by time of day (UTC)
SELECT EXTRACT(HOUR FROM "createdAt") as hour, COUNT(*) as phantom_count
FROM "Trade"
WHERE "isPhantom" = true
GROUP BY hour
ORDER BY hour;

-- Phantom trades with quality scores
SELECT "signalQualityScore", COUNT(*) as count, AVG("atrAtEntry") as avg_atr
FROM "Trade"
WHERE "isPhantom" = true
GROUP BY "signalQualityScore"
ORDER BY "signalQualityScore" DESC;
```

## Expected Behavior

### Telegram Notifications:
- If phantom detected, execute endpoint returns 500 error
- n8n workflow should catch this and send error notification
- User sees: "Trade failed: Phantom trade detected"
- NO "Position monitored" message

### Dashboard:
- Phantom trades appear in database with `status: 'phantom'`
- Can be filtered out or analyzed separately
- Shows expected vs actual size for debugging

### Position Manager:
- Phantom trades are NEVER added to Position Manager
- No monitoring, no false alarms
- No "closed externally" spam in logs

## Prevention Strategy

Going forward, phantom trades should be rare because:
1. **1-minute cooldown** prevents rapid flips during volatility
2. **Quality score filtering** blocks low-quality signals (which tend to occur during chaos)
3. **Post-entry validation** catches phantoms immediately
4. **Database tracking** allows us to analyze patterns and adjust

If phantom trades continue to occur frequently, we can:
- Increase cooldown for flips (2-3 minutes)
- Add ATR-based volatility check (block flips when ATR > threshold)
- Implement pre-entry oracle validation (add 2% discrepancy check before placing order)

## Files Modified

- `lib/drift/orders.ts` - Added phantom detection in `openPosition()`
- `app/api/trading/execute/route.ts` - Added phantom handling after opening
- `lib/database/trades.ts` - Added phantom fields to CreateTradeParams
- `prisma/schema.prisma` - Added phantom trade fields to Trade model
- `prisma/migrations/20251104091741_add_phantom_trade_fields/` - Database migration

## Testing

To test phantom detection:
1. Modify `openPosition()` to simulate phantom (set actualSizeUSD = 10)
2. Send test trade signal
3. Verify:
   - Error returned from execute endpoint
   - Phantom trade saved to database with `isPhantom: true`
   - NO position added to Position Manager
   - Logs show "🚨 PHANTOM TRADE DETECTED"

## Future Improvements

If phantom trades remain an issue:
1. **Auto-retry with delay:** Wait 5s for oracle to catch up, retry once
2. **Oracle price validation:** Check Pyth price vs Drift oracle before placing order
3. **Volatility-based cooldown:** Longer cooldown during high ATR periods
4. **Symbol-specific thresholds:** SOL might need different validation than ETH