feat: Implement re-entry analytics system with fresh TradingView data

- Add market data cache service (5min expiry) for storing TradingView metrics - Create /api/trading/market-data webhook endpoint for continuous data updates - Add /api/analytics/reentry-check endpoint for validating manual trades - Update execute endpoint to auto-cache metrics from incoming signals - Enhance Telegram bot with pre-execution analytics validation - Support --force flag to override analytics blocks - Use fresh ADX/ATR/RSI data when available, fallback to historical - Apply performance modifiers: -20 for losing streaks, +10 for winning - Minimum re-entry score 55 (vs 60 for new signals) - Fail-open design: proceeds if analytics unavailable - Show data freshness and source in Telegram responses - Add comprehensive setup guide in docs/guides/REENTRY_ANALYTICS_QUICKSTART.md Phase 1 implementation for smart manual trade validation.
2025-11-07 20:40:07 +01:00
parent 6d5991172a
commit 9b767342dc
14 changed files with 1150 additions and 568 deletions
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -30,6 +30,13 @@

 **Manual Trading via Telegram:** Send plain-text messages like `long sol`, `short eth`, `long btc` to open positions instantly (bypasses n8n, calls `/api/trading/execute` directly with preset healthy metrics).

+**Re-Entry Analytics System:** Manual trades are validated before execution using fresh TradingView data:
+- Market data cached from TradingView signals (5min expiry)
+- `/api/analytics/reentry-check` scores re-entry based on fresh metrics + recent performance
+- Telegram bot blocks low-quality re-entries unless `--force` flag used
+- Uses real TradingView ADX/ATR/RSI when available, falls back to historical data
+- Penalty for recent losing trades, bonus for winning streaks
+
 ## Critical Components

 ### 1. Signal Quality Scoring (`lib/trading/signal-quality.ts`)
@@ -87,18 +94,23 @@ await positionManager.addTrade(activeTrade)
 **Manual trade commands via plain text:**
 ```python
 # User sends plain text message (not slash commands)
-"long sol"   → Opens SOL-PERP long position
-"short eth"  → Opens ETH-PERP short position
-"long btc"   → Opens BTC-PERP long position
+"long sol"          → Validates via analytics, then opens SOL-PERP long
+"short eth"         → Validates via analytics, then opens ETH-PERP short
+"long btc --force"  → Skips analytics validation, opens BTC-PERP long immediately
 ```

 **Key behaviors:**
 - MessageHandler processes all text messages (not just commands)
 - Maps user-friendly symbols (sol, eth, btc) to Drift format (SOL-PERP, etc.)
- Calls `/api/trading/execute` directly with preset healthy metrics (ATR=1.0, ADX=25, RSI=50, volumeRatio=1.2)
+- **Analytics validation:** Calls `/api/analytics/reentry-check` before execution
+  - Blocks trades with score <55 unless `--force` flag used
+  - Uses fresh TradingView data (<5min old) when available
+  - Falls back to historical metrics with penalty
+  - Considers recent trade performance (last 3 trades)
+- Calls `/api/trading/execute` directly with preset healthy metrics (ATR=0.45, ADX=32, RSI=58/42)
 - Bypasses n8n workflow and TradingView requirements
 - 60-second timeout for API calls
- Responds with trade confirmation or error message
+- Responds with trade confirmation or analytics rejection message

 **Status command:**
 ```python
@@ -218,13 +230,15 @@ const driftSymbol = normalizeTradingViewSymbol(body.symbol)
 7. Add to Position Manager if applicable

 **Key endpoints:**
- `/api/trading/execute` - Main entry point from n8n (production, requires auth)
+- `/api/trading/execute` - Main entry point from n8n (production, requires auth), **auto-caches market data**
 - `/api/trading/check-risk` - Pre-execution validation (duplicate check, quality score, **per-symbol cooldown**, rate limits, **symbol enabled check**)
 - `/api/trading/test` - Test trades from settings UI (no auth required, **respects symbol enable/disable**)
 - `/api/trading/close` - Manual position closing
 - `/api/trading/positions` - Query open positions from Drift
+- `/api/trading/market-data` - Webhook for TradingView market data updates (GET for debug, POST for data)
 - `/api/settings` - Get/update config (writes to .env file, **includes per-symbol settings**)
 - `/api/analytics/last-trade` - Fetch most recent trade details for dashboard (includes quality score)
+- `/api/analytics/reentry-check` - **Validate manual re-entry** with fresh TradingView data + recent performance
 - `/api/analytics/version-comparison` - Compare performance across signal quality logic versions (v1/v2/v3)
 - `/api/restart` - Create restart flag for watch-restart.sh script

@@ -442,6 +456,70 @@ trade.realizedPnL += actualRealizedPnL  // NOT: result.realizedPnL from SDK
 - **Types:** Define interfaces in same file as implementation (not separate types directory)
 - **Console logs:** Use emojis for visual scanning: 🎯 🚀 ✅ ❌ 💰 📊 🛡️

+## Re-Entry Analytics System (Phase 1)
+
+**Purpose:** Validate manual Telegram trades using fresh TradingView data + recent performance analysis
+
+**Components:**
+1. **Market Data Cache** (`lib/trading/market-data-cache.ts`)
+   - Singleton service storing TradingView metrics
+   - 5-minute expiry on cached data
+   - Tracks: ATR, ADX, RSI, volume ratio, price position, timeframe
+
+2. **Market Data Webhook** (`app/api/trading/market-data/route.ts`)
+   - Receives TradingView alerts every 1-5 minutes
+   - POST: Updates cache with fresh metrics
+   - GET: View cached data (debugging)
+
+3. **Re-Entry Check Endpoint** (`app/api/analytics/reentry-check/route.ts`)
+   - Validates manual trade requests
+   - Uses fresh TradingView data if available (<5min old)
+   - Falls back to historical metrics from last trade
+   - Scores signal quality + applies performance modifiers:
+     - **-20 points** if last 3 trades lost money (avgPnL < -5%)
+     - **+10 points** if last 3 trades won (avgPnL > +5%, WR >= 66%)
+     - **-5 points** for stale data, **-10 points** for no data
+   - Minimum score: 55 (vs 60 for new signals)
+
+4. **Auto-Caching** (`app/api/trading/execute/route.ts`)
+   - Every trade signal from TradingView auto-caches metrics
+   - Ensures fresh data available for manual re-entries
+
+5. **Telegram Integration** (`telegram_command_bot.py`)
+   - Calls `/api/analytics/reentry-check` before executing manual trades
+   - Shows data freshness ("✅ FRESH 23s old" vs "⚠️ Historical")
+   - Blocks low-quality re-entries unless `--force` flag used
+   - Fail-open: Proceeds if analytics check fails
+
+**User Flow:**
+```
+User: "long sol"
+  ↓ Check cache for SOL-PERP
+  ↓ Fresh data? → Use real TradingView metrics
+  ↓ Stale/missing? → Use historical + penalty
+  ↓ Score quality + recent performance
+  ↓ Score >= 55? → Execute
+  ↓ Score < 55? → Block (unless --force)
+```
+
+**TradingView Setup:**
+Create alerts that fire every 1-5 minutes with this webhook message:
+```json
+{
+  "action": "market_data",
+  "symbol": "{{ticker}}",
+  "timeframe": "{{interval}}",
+  "atr": {{ta.atr(14)}},
+  "adx": {{ta.dmi(14, 14)}},
+  "rsi": {{ta.rsi(14)}},
+  "volumeRatio": {{volume / ta.sma(volume, 20)}},
+  "pricePosition": {{(close - ta.lowest(low, 100)) / (ta.highest(high, 100) - ta.lowest(low, 100)) * 100}},
+  "currentPrice": {{close}}
+}
+```
+
+Webhook URL: `https://your-domain.com/api/trading/market-data`
+
 ## Per-Symbol Trading Controls

 **Purpose:** Independent enable/disable toggles and position sizing for SOL and ETH to support different trading strategies (e.g., ETH for data collection at minimal size, SOL for profit generation).