docs: Add Common Pitfall #66 - Smart Entry Validation Queue symbol normalization bug
- Symptom: Abandonment notifications showing impossible prices (26 → 8.18 in 30s)
- Root cause: Symbol format mismatch (TradingView 'SOLUSDT' vs cache 'SOL-PERP')
- Fix: Added normalizeTradingViewSymbol() in check-risk endpoint before validation queue
- Impact: Cache lookup now succeeds, Telegram shows correct abandonment prices
- Files: check-risk/route.ts line 9 (import), lines 432-444 (normalization)
- Commit: 6cec2e8 deployed Dec 1, 2025
- Lesson: Always normalize symbols at integration boundaries, cache key mismatches fail silently
This commit is contained in:
mindesbunister
82
.github/copilot-instructions.md
vendored
82
.github/copilot-instructions.md
vendored
@@ -4839,6 +4839,88 @@ trade.realizedPnL += actualRealizedPnL // NOT: result.realizedPnL from SDK
|
||||
- **Impact:** Blocked 45 minutes of distributed work, wasted cluster compute, caught before Stage 1 analysis
|
||||
- **Files changed:** `cluster/distributed_worker.py` lines 67-77
|
||||
|
||||
66. **Smart Entry Validation Queue wrong price display (CRITICAL - Fixed Dec 1, 2025):**
|
||||
- **Symptom:** Abandonment notifications in Telegram showing incorrect/impossible prices (e.g., $126.00 → $98.18 = -22.08% drop in 30 seconds)
|
||||
- **User Report:** "the abandonend signal shows the wrong price. it never went that low"
|
||||
- **Root Cause:** Symbol format mismatch between validation queue storage and market data cache lookup
|
||||
* TradingView webhook sends: `"SOLUSDT"`, `"ETHUSDT"`, etc.
|
||||
* check-risk endpoint passed: `body.symbol` directly to validation queue without normalization
|
||||
* Validation queue stored: `signal.symbol = "SOLUSDT"`
|
||||
* Market data cache uses: Normalized Drift format `"SOL-PERP"`, `"ETH-PERP"`
|
||||
* Cache lookup: `marketDataCache.get("SOLUSDT")` returns `null` (key not found)
|
||||
* Result: No cached data, wrong/stale price used in Telegram abandonment notification
|
||||
- **Real Incident (Dec 1, 2025):**
|
||||
* Signal queued at $126.00
|
||||
* Abandonment notification showed $98.18
|
||||
* Reported drop: -22.08% in 30 seconds (physically impossible for SOL)
|
||||
* Actual: Cache lookup failed due to symbol format, displayed stale/wrong price
|
||||
- **Impact:** Every Smart Entry Validation Queue abandonment notification shows incorrect price, misleading user about system behavior and market conditions
|
||||
- **Data Flow Before Fix:**
|
||||
1. TradingView → check-risk: `{"symbol": "SOLUSDT", ...}`
|
||||
2. check-risk → validation queue: `symbol: "SOLUSDT"` (no normalization)
|
||||
3. Validation queue stores: `signal.symbol = "SOLUSDT"`
|
||||
4. Validation loop: `marketDataCache.get("SOLUSDT")` → **null** (key mismatch)
|
||||
5. Telegram notification: Shows wrong/stale price
|
||||
- **Data Flow After Fix:**
|
||||
1. TradingView → check-risk: `{"symbol": "SOLUSDT", ...}`
|
||||
2. check-risk normalizes: `const normalizedSymbol = normalizeTradingViewSymbol("SOLUSDT")` → `"SOL-PERP"`
|
||||
3. check-risk → validation queue: `symbol: "SOL-PERP"` (normalized)
|
||||
4. Validation loop: `marketDataCache.get("SOL-PERP")` → **success** (key matches)
|
||||
5. Telegram notification: Shows correct current price from cache
|
||||
- **Fix (app/api/trading/check-risk/route.ts):**
|
||||
```typescript
|
||||
// Line 9: Added import
|
||||
import { normalizeTradingViewSymbol } from '@/config/trading'
|
||||
|
||||
// Lines 432-444: Normalize symbol before validation queue
|
||||
// CRITICAL FIX (Dec 1, 2025): Normalize TradingView symbol format to Drift format
|
||||
// Bug: Market data cache uses "SOL-PERP" but TradingView sends "SOLUSDT"
|
||||
// Without normalization, validation queue can't find matching price data
|
||||
// Result: Wrong/stale price shown in Telegram abandonment notifications
|
||||
const normalizedSymbol = normalizeTradingViewSymbol(body.symbol)
|
||||
|
||||
const queued = await validationQueue.addSignal({
|
||||
blockReason: 'QUALITY_SCORE_TOO_LOW',
|
||||
symbol: normalizedSymbol, // Use normalized format for cache lookup
|
||||
direction: body.direction,
|
||||
originalPrice: currentPrice,
|
||||
qualityScore: qualityScore.score,
|
||||
// ... other parameters
|
||||
})
|
||||
|
||||
// Line 458: Updated console.log
|
||||
console.log(`🧠 Signal queued for smart validation: ${normalizedSymbol} ${body.direction}...`)
|
||||
```
|
||||
- **Files Changed:**
|
||||
* `app/api/trading/check-risk/route.ts` (line 9 import, lines 432-444 normalization, line 458 log)
|
||||
* Total: 6 insertions including comment block
|
||||
- **Related Components (No Changes Needed):**
|
||||
* `lib/trading/smart-validation-queue.ts` - Receives normalized symbol, validation logic correct
|
||||
* `lib/trading/market-data-cache.ts` - Uses normalized format "SOL-PERP" (line 11 comment)
|
||||
* `lib/notifications/telegram.ts` - Displays whatever price passed, formatter correct
|
||||
* `app/api/trading/market-data/route.ts` - Already normalizes correctly (reference implementation)
|
||||
* `config/trading.ts` - Contains `normalizeTradingViewSymbol()` function at line 238
|
||||
- **Git Commit:** 6cec2e8 "critical: Fix Smart Entry Validation Queue wrong price display"
|
||||
- **Deployed:** Dec 1, 2025, 23:45:21 +0100 (container trading-bot-v4 restarted with fix)
|
||||
- **Verification Required:**
|
||||
* Wait for next marginal quality signal (quality 50-89) to be queued
|
||||
* Monitor validation queue logs: Should show normalized symbol format ("SOL-PERP" not "SOLUSDT")
|
||||
* Check Telegram abandonment: Should show correct price matching market reality
|
||||
* Verify cache lookup succeeds: Console logs "✅ Using fresh TradingView data for SOL-PERP"
|
||||
- **Lessons Learned:**
|
||||
1. **Symbol format consistency is CRITICAL across all system components** - One endpoint without normalization breaks entire data flow
|
||||
2. **Cache key mismatches fail silently** - `marketDataCache.get("WRONG-KEY")` returns `null`, no error thrown
|
||||
3. **Always normalize at integration boundaries** - External webhooks (TradingView) → Internal systems (validation queue, cache) require format conversion
|
||||
4. **Market data cache uses Drift format as standard** - "SOL-PERP", "ETH-PERP", "BTC-PERP" (not TradingView "SOLUSDT", "ETHUSDT")
|
||||
5. **ALL endpoints feeding validation queue must normalize** - check-risk, execute, market-data all need consistent symbol handling
|
||||
6. **Verify data flow end-to-end** - Trace from webhook → storage → lookup → display to catch format mismatches
|
||||
7. **Silent failures cause data corruption downstream** - Wrong price displayed misleads user about system behavior
|
||||
- **Why This Matters:**
|
||||
* Smart Entry Validation Queue is profit recovery system for marginal quality signals (expected +$1,823/month)
|
||||
* Wrong abandonment prices mislead user about system decisions (appears system abandoned profitable setups)
|
||||
* In real money trading, user relies on accurate notifications for manual intervention decisions
|
||||
* Cache lookups are critical path for all validation queue price checks (every 30 seconds)
|
||||
|
||||
## File Conventions
|
||||
|
||||
- **API routes:** `app/api/[feature]/[action]/route.ts` (Next.js 15 App Router)
|
||||
|
||||
Reference in New Issue
Block a user