critical: Bug #93 - Three-layer entry price validation with oracle fallback
Root Cause: quoteAssetAmount/baseAssetAmount division producing garbage entry prices - Found 6 autosync records with impossible prices (.18, 6.30, 7.11, 116.24 for SOL) - Drift SDK values can be corrupted during state transitions Fix Layer 1 (lib/drift/client.ts): - Added per-asset price range validation (SOL: 0-000, BTC: 0k-00k, ETH: 00-0k) - Returns null for invalid prices Fix Layer 2 (lib/trading/sync-helper.ts): - Added validatedEntryPrice calculation with oracle fallback - Falls back to Pyth oracle when calculated price is garbage Fix Layer 3 (lib/trading/sync-helper.ts): - Trade creation uses validatedEntryPrice in all 4 price fields - entryPrice, peakPrice, maxFavorablePrice, maxAdversePrice Documentation: - Full Bug #93 added to COMMON_PITFALLS.md with code examples - Quick Reference table updated Cleaned: 6 garbage autosync records deleted from database
This commit is contained in:
mindesbunister
@@ -98,12 +98,11 @@ This document is the **comprehensive reference** for all documented pitfalls, bu
|
||||
| 70 | 🔴 CRITICAL | Smart Entry | Dec 3, 2025 | Smart Validation Queue rejected by execute endpoint |
|
||||
| 71 | 🔴 CRITICAL | Revenge System | Dec 3, 2025 | Revenge system missing external closure integration |
|
||||
| 72 | 🔴 CRITICAL | Telegram | Dec 4, 2025 | Telegram webhook conflicts with polling bot |
|
||||
| 91 | 🔴 CRITICAL | Orders | Jan 1, 2026 | Math.floor truncation in position close leaves fractional remnants |
|
||||
| 92 | 🔴 CRITICAL | Orders | Jan 6, 2026 | Exit order token sizing mismatch - TP/SL different sizes than position |
|
||||
| 89 | 🔴 CRITICAL | Drift Protocol | Dec 16, 2025 | Drift fractional position remnants after SL execution |
|
||||
| 90 | 🔴 CRITICAL | Drift Protocol | Dec 31, 2025 | placePerpOrder vs placeAndTakePerpOrder - MARKET orders not filling |
|
||||
| 91 | 🔴 CRITICAL | Drift Protocol | Jan 1, 2026 | Math.floor truncation leaves fractional position remnants |
|
||||
| 90 | 🔴 CRITICAL | Drift Protocol | Dec 31, 2025 | placePerpOrder only places orders, doesn't fill - use placeAndTakePerpOrder |
|
||||
| 91 | 🔴 CRITICAL | Orders | Jan 1, 2026 | Math.floor truncation leaves fractional position remnants |
|
||||
| 92 | 🔴 CRITICAL | Orders | Jan 6, 2026 | Exit order token sizing mismatch - TP/SL different sizes than position |
|
||||
| 93 | 🔴 CRITICAL | Data Integrity | Jan 6, 2026 | Autosync garbage entry prices from corrupted quoteAsset/baseAsset division |
|
||||
|
||||
---
|
||||
|
||||
@@ -1958,6 +1957,189 @@ docker logs -f trading-bot-v4 | grep "Exit order sizes"
|
||||
|
||||
---
|
||||
|
||||
## 93. CRITICAL: Autosync Garbage Entry Prices - quoteAsset/baseAsset Division Corruption (Jan 6, 2026)
|
||||
|
||||
**Symptom:** Autosync placeholder trades created with impossible entry prices that don't match actual market prices. Examples from SOL-PERP when actual price was ~$136:
|
||||
- Entry $1.18 (garbage)
|
||||
- Entry $16.30 (garbage)
|
||||
- Entry $57.11 (garbage)
|
||||
- Entry $4116.24 (garbage)
|
||||
|
||||
**User Report:** Database P&L didn't match Drift exchange UI. Investigation revealed 6 autosync records with impossible entry prices causing massive P&L calculation errors.
|
||||
|
||||
**Financial Impact:**
|
||||
- Corrupted P&L calculations in database
|
||||
- Can't trust database for financial reporting
|
||||
- Autosync "protection" actually creates worse problems than no protection
|
||||
- User can't reconcile trading performance with exchange data
|
||||
|
||||
**Real Incidents (Jan 6, 2026):**
|
||||
```sql
|
||||
-- Found garbage entry prices in autosync records:
|
||||
id | symbol | entryPrice | signalSource
|
||||
cmjpn... | SOL-PERP | 16.30 | autosync -- Should be ~$136
|
||||
cmjpo... | SOL-PERP | 1.18 | autosync -- Should be ~$136
|
||||
cmjpo... | SOL-PERP | 57.11 | autosync -- Should be ~$136
|
||||
cmjpv... | SOL-PERP | 4116.24 | autosync -- Should be ~$136
|
||||
cmjpw... | SOL-PERP | 16.30 | autosync -- Should be ~$136
|
||||
cmjpy... | SOL-PERP | 16.30 | autosync -- Should be ~$136
|
||||
```
|
||||
|
||||
**Root Cause:**
|
||||
* File: `lib/drift/client.ts` line 278
|
||||
* Code: `entryPrice = Math.abs(quoteAssetAmount / baseAssetAmount)`
|
||||
* Problem: When `quoteAssetAmount` or `baseAssetAmount` are:
|
||||
- Corrupted/stale from Drift state propagation delays
|
||||
- Near-zero (division produces huge numbers)
|
||||
- Mismatched timing (position data not yet settled)
|
||||
- The result is garbage entry prices
|
||||
* **Drift SDK doesn't always return reliable values** for quoteAssetAmount/baseAssetAmount during:
|
||||
- Position state transitions (opening/closing)
|
||||
- Partial fills
|
||||
- Network congestion
|
||||
- State propagation delays (5+ minutes documented)
|
||||
|
||||
**Why Entry Price Can't Be Trusted from Division:**
|
||||
```typescript
|
||||
// Drift SDK returns:
|
||||
interface PerpPosition {
|
||||
baseAssetAmount: BN // Token amount (can be stale/corrupted)
|
||||
quoteAssetAmount: BN // USD equivalent (can be stale/corrupted)
|
||||
}
|
||||
|
||||
// Division produces garbage when either value is wrong:
|
||||
// Case 1: baseAssetAmount near zero
|
||||
// 1000 / 0.0001 = 10,000,000 (impossibly high)
|
||||
|
||||
// Case 2: quoteAssetAmount stale
|
||||
// 1.0 / 0.5 = 2.0 (when actual price is $136)
|
||||
|
||||
// Case 3: Both stale but inconsistent
|
||||
// 100 / 6.13 = 16.30 (garbage)
|
||||
```
|
||||
|
||||
**THE FIX - Three-Layer Validation:**
|
||||
|
||||
**Layer 1: Primary validation in lib/drift/client.ts (lines 278-310):**
|
||||
```typescript
|
||||
// Calculate raw entry price
|
||||
const rawEntryPrice = Math.abs(quoteAssetAmount / baseAssetAmount)
|
||||
|
||||
// Validate against realistic ranges per asset
|
||||
const priceRanges: Record<string, { min: number; max: number }> = {
|
||||
'SOL-PERP': { min: 50, max: 1000 },
|
||||
'BTC-PERP': { min: 10000, max: 500000 },
|
||||
'ETH-PERP': { min: 500, max: 20000 },
|
||||
'DEFAULT': { min: 0.001, max: 1000000 }
|
||||
}
|
||||
|
||||
const range = priceRanges[symbol] || priceRanges['DEFAULT']
|
||||
if (rawEntryPrice < range.min || rawEntryPrice > range.max) {
|
||||
console.warn(`⚠️ Invalid entry price ${rawEntryPrice} for ${symbol} (valid range: ${range.min}-${range.max})`)
|
||||
return null // Signal that entry price is invalid
|
||||
}
|
||||
```
|
||||
|
||||
**Layer 2: Oracle fallback in lib/trading/sync-helper.ts (lines 130-150):**
|
||||
```typescript
|
||||
// If calculated entry price is garbage, fall back to oracle price
|
||||
let validatedEntryPrice = entryPrice
|
||||
|
||||
if (!entryPrice || entryPrice < 0.01 || entryPrice > 100000) {
|
||||
console.warn(`⚠️ Autosync: Entry price ${entryPrice} invalid, using oracle price`)
|
||||
const pythPriceMonitor = getPythPriceMonitor()
|
||||
const currentPrice = await pythPriceMonitor.getLatestPrice(driftPos.symbol)
|
||||
if (currentPrice && currentPrice > 0) {
|
||||
validatedEntryPrice = currentPrice
|
||||
console.log(`✅ Autosync: Using oracle price ${currentPrice} as entry price`)
|
||||
}
|
||||
}
|
||||
|
||||
// Use validatedEntryPrice for all calculations
|
||||
const tp1Price = calculateTp1Price(validatedEntryPrice, direction)
|
||||
const tp2Price = calculateTp2Price(validatedEntryPrice, direction)
|
||||
const stopLossPrice = calculateStopLossPrice(validatedEntryPrice, direction)
|
||||
```
|
||||
|
||||
**Layer 3: Trade creation uses validated price (lines 308-360):**
|
||||
```typescript
|
||||
const placeholderTrade = await prisma.trade.create({
|
||||
data: {
|
||||
// ... other fields
|
||||
entryPrice: validatedEntryPrice, // BUG #93 FIX: Use validated price
|
||||
peakPrice: validatedEntryPrice, // BUG #93 FIX: Use validated price
|
||||
maxFavorablePrice: validatedEntryPrice, // BUG #93 FIX: Use validated price
|
||||
maxAdversePrice: validatedEntryPrice, // BUG #93 FIX: Use validated price
|
||||
// ...
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
**Files Changed:**
|
||||
1. `lib/drift/client.ts`:
|
||||
- Added per-asset price range validation at line 278-295
|
||||
- Returns null for invalid prices (caller must handle)
|
||||
- Realistic ranges: SOL $50-$1000, BTC $10k-$500k, ETH $500-$20k
|
||||
|
||||
2. `lib/trading/sync-helper.ts`:
|
||||
- Added `validatedEntryPrice` calculation with oracle fallback (lines 130-150)
|
||||
- TP/SL calculations use validated price (lines 143-146)
|
||||
- Trade creation uses validated price in 4 fields (lines 308-360)
|
||||
|
||||
**Cleanup Performed:**
|
||||
```sql
|
||||
-- Deleted 6 garbage autosync records
|
||||
DELETE FROM "Trade"
|
||||
WHERE "signalSource" = 'autosync'
|
||||
AND ("entryPrice" < 50 OR "entryPrice" > 200) -- For SOL-PERP
|
||||
AND symbol = 'SOL-PERP';
|
||||
```
|
||||
|
||||
**Verification After Fix:**
|
||||
```bash
|
||||
# Check logs for validation messages
|
||||
docker logs -f trading-bot-v4 | grep -E "(Invalid entry price|Using oracle price)"
|
||||
|
||||
# Monitor autosync entries for realistic prices
|
||||
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c \
|
||||
"SELECT symbol, \"entryPrice\", \"signalSource\", \"createdAt\"
|
||||
FROM \"Trade\"
|
||||
WHERE \"signalSource\" = 'autosync'
|
||||
ORDER BY \"createdAt\" DESC LIMIT 5;"
|
||||
```
|
||||
|
||||
**Prevention Rules:**
|
||||
1. NEVER trust `quoteAssetAmount / baseAssetAmount` without validation
|
||||
2. ALWAYS validate calculated prices against realistic ranges
|
||||
3. ALWAYS have oracle price fallback for autosync operations
|
||||
4. ALWAYS use the validated price in ALL fields (entry, peak, MFE, MAE)
|
||||
5. Document per-asset price ranges and update when market conditions change
|
||||
6. Autosync is a safety net, not the source of truth - prefer TradingView data
|
||||
|
||||
**Red Flags Indicating This Bug:**
|
||||
- Database entry prices wildly different from current market price
|
||||
- `signalSource = 'autosync'` with extreme entry prices
|
||||
- P&L calculations in database don't match exchange UI
|
||||
- MFE/MAE showing impossible percentages (100x+ or negative when shouldn't be)
|
||||
- Health monitor creating placeholder trades with wrong prices
|
||||
|
||||
**Why This Matters:**
|
||||
- **Database is the source of truth** for P&L tracking and analytics
|
||||
- Garbage entry prices corrupt ALL derived calculations (P&L, MFE, MAE, win rate)
|
||||
- Can't trust historical performance data when autosync pollutes it
|
||||
- User can't reconcile trading performance with exchange data
|
||||
- Financial decisions based on wrong data = losing money
|
||||
|
||||
**Git Commit:** [PENDING] "critical: Bug #93 - Three-layer entry price validation with oracle fallback"
|
||||
|
||||
**Deployment:** [PENDING] - Requires docker restart
|
||||
|
||||
**Status:** ✅ CODE FIXED - Awaiting deployment
|
||||
|
||||
**Lesson Learned:** The Drift SDK `quoteAssetAmount / baseAssetAmount` calculation is unreliable during state transitions. The user said it best: "the bot simply needs to read the data from drift to get the proper data" - but even Drift's position data can be corrupted. Always validate against realistic ranges and fall back to oracle prices when the calculated value is garbage.
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Pattern Recognition
|
||||
|
||||
### Common Root Causes
|
||||
|
||||
Reference in New Issue
Block a user