From 356b4ed578cb23a9aa2dbedcdf8eef3c8fb21be1 Mon Sep 17 00:00:00 2001 From: mindesbunister Date: Tue, 11 Nov 2025 11:52:24 +0100 Subject: [PATCH] docs: update AI agent instructions with blocked signals tracking - Added BlockedSignal to database models list - Updated signalQualityVersion to v4 (current) - Added blocked signals tracking functions to database section - Updated check-risk endpoint description - Added Signal Quality Optimization Roadmap reference - Documented blocked signals analysis workflow - Added reference to BLOCKED_SIGNALS_TRACKING.md This ensures AI agents understand the new data collection system. --- .github/copilot-instructions.md | 44 +++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 8 deletions(-) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 5032733..0002fac 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -208,7 +208,7 @@ Without this, order cancellations fail silently during TP1→breakeven order upd ### 5. Database (`lib/database/trades.ts` + `prisma/schema.prisma`) **Purpose:** PostgreSQL via Prisma ORM for trade history and analytics -**Models:** Trade, PriceUpdate, SystemEvent, DailyStats +**Models:** Trade, PriceUpdate, SystemEvent, DailyStats, BlockedSignal **Singleton pattern:** Use `getPrismaClient()` - never instantiate PrismaClient directly @@ -218,19 +218,31 @@ Without this, order cancellations fail silently during TP1→breakeven order upd - `addPriceUpdate()` - Track price movements (called by Position Manager) - `getTradeStats()` - Win rate, profit factor, avg win/loss - `getLastTrade()` - Fetch most recent trade for analytics dashboard +- `createBlockedSignal()` - Save blocked signals for data-driven optimization analysis +- `getRecentBlockedSignals()` - Query recent blocked signals +- `getBlockedSignalsForAnalysis()` - Fetch signals needing price analysis (future automation) **Important fields:** - `signalQualityScore` (Int?) - 0-100 score for data-driven optimization -- `signalQualityVersion` (String?) - Tracks which scoring logic was used ('v1', 'v2', 'v3') +- `signalQualityVersion` (String?) - Tracks which scoring logic was used ('v1', 'v2', 'v3', 'v4') - v1: Original logic (price position < 5% threshold) - v2: Added volume compensation for low ADX (2025-11-07) - - v3: CURRENT - Stricter breakdown requirements: positions < 15% require (ADX > 18 AND volume > 1.2x) OR (RSI < 35 for shorts / RSI > 60 for longs) + - v3: Stricter breakdown requirements: positions < 15% require (ADX > 18 AND volume > 1.2x) OR (RSI < 35 for shorts / RSI > 60 for longs) + - v4: CURRENT - Blocked signals tracking enabled for data-driven threshold optimization (2025-11-11) - All new trades tagged with current version for comparative analysis - `maxFavorableExcursion` / `maxAdverseExcursion` - Track best/worst P&L during trade lifetime - `maxFavorablePrice` / `maxAdversePrice` - Track prices at MFE/MAE points - `configSnapshot` (Json) - Stores Position Manager state for crash recovery - `atr`, `adx`, `rsi`, `volumeRatio`, `pricePosition` - Context metrics from TradingView +**BlockedSignal model fields (NEW):** +- Signal metrics: `atr`, `adx`, `rsi`, `volumeRatio`, `pricePosition`, `timeframe` +- Quality scoring: `signalQualityScore`, `signalQualityVersion`, `scoreBreakdown` (JSON), `minScoreRequired` +- Block tracking: `blockReason` (QUALITY_SCORE_TOO_LOW, COOLDOWN_PERIOD, HOURLY_TRADE_LIMIT, etc.), `blockDetails` +- Future analysis: `priceAfter1/5/15/30Min`, `wouldHitTP1/TP2/SL`, `analysisComplete` +- Automatically saved by check-risk endpoint when signals are blocked +- Enables data-driven optimization: collect 10-20 blocked signals → analyze patterns → adjust thresholds + **Per-symbol functions:** - `getLastTradeTimeForSymbol(symbol)` - Get last trade time for specific coin (enables per-symbol cooldown) - Each coin (SOL/ETH/BTC) has independent cooldown timer to avoid missed opportunities @@ -272,7 +284,7 @@ const driftSymbol = normalizeTradingViewSymbol(body.symbol) **Key endpoints:** - `/api/trading/execute` - Main entry point from n8n (production, requires auth), **auto-caches market data** -- `/api/trading/check-risk` - Pre-execution validation (duplicate check, quality score, **per-symbol cooldown**, rate limits, **symbol enabled check**) +- `/api/trading/check-risk` - Pre-execution validation (duplicate check, quality score, **per-symbol cooldown**, rate limits, **symbol enabled check**, **saves blocked signals automatically**) - `/api/trading/test` - Test trades from settings UI (no auth required, **respects symbol enable/disable**) - `/api/trading/close` - Manual position closing (requires symbol normalization) - `/api/trading/cancel-orders` - **Manual order cleanup** (for stuck/ghost orders after rate limit failures) @@ -281,7 +293,7 @@ const driftSymbol = normalizeTradingViewSymbol(body.symbol) - `/api/settings` - Get/update config (writes to .env file, **includes per-symbol settings**) - `/api/analytics/last-trade` - Fetch most recent trade details for dashboard (includes quality score) - `/api/analytics/reentry-check` - **Validate manual re-entry** with fresh TradingView data + recent performance -- `/api/analytics/version-comparison` - Compare performance across signal quality logic versions (v1/v2/v3) +- `/api/analytics/version-comparison` - Compare performance across signal quality logic versions (v1/v2/v3/v4) - `/api/restart` - Create restart flag for watch-restart.sh script ## Critical Workflows @@ -609,7 +621,14 @@ if (!enabled) { ## Development Roadmap -See `POSITION_SCALING_ROADMAP.md` for planned optimizations: +See `SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md` for systematic signal quality improvements: +- **Phase 1 (🔄 IN PROGRESS):** Collect 10-20 blocked signals with quality scores (1-2 weeks) +- **Phase 2 (🔜 NEXT):** Analyze patterns and make data-driven threshold decisions +- **Phase 3 (🎯 FUTURE):** Implement dual-threshold system or other optimizations based on data +- **Phase 4 (🤖 FUTURE):** Automated price analysis for blocked signals +- **Phase 5 (🧠 DISTANT):** ML-based scoring weight optimization + +See `POSITION_SCALING_ROADMAP.md` for planned position management optimizations: - **Phase 1 (✅ COMPLETE):** Collect data with quality scores (20-50 trades needed) - **Phase 2:** ATR-based dynamic targets (adapt to volatility) - **Phase 3:** Signal quality-based scaling (high quality = larger runners) @@ -619,13 +638,22 @@ See `POSITION_SCALING_ROADMAP.md` for planned optimizations: **Recent Implementation:** TP2-as-runner system provides 5x larger runner (25% vs 5%) for better profit capture on extended moves. When TP2 price is hit, trailing stop activates on full remaining position instead of closing partial amount. +**Blocked Signals Tracking (Nov 11, 2025):** System now automatically saves all blocked signals to database for data-driven optimization. See `BLOCKED_SIGNALS_TRACKING.md` for SQL queries and analysis workflows. + **Data-driven approach:** Each phase requires validation through SQL analysis before implementation. No premature optimization. **Signal Quality Version Tracking:** Database tracks `signalQualityVersion` field to compare algorithm performance: - Analytics dashboard shows version comparison: trades, win rate, P&L, extreme position stats -- Focus on extreme positions (< 15% range) - v3 aims to reduce losses from weak ADX entries +- v4 (current) includes blocked signals tracking for data-driven optimization +- Focus on extreme positions (< 15% range) - v3 aimed to reduce losses from weak ADX entries - SQL queries in `docs/analysis/SIGNAL_QUALITY_VERSION_ANALYSIS.sql` for deep-dive analysis -- Need 20+ v3 trades before meaningful comparison vs v1/v2 data +- Need 20+ trades per version before meaningful comparison + +**Blocked Signals Analysis:** See `BLOCKED_SIGNALS_TRACKING.md` for: +- SQL queries to analyze blocked signal patterns +- Score distribution and metric analysis +- Comparison with executed trades at similar quality levels +- Future automation of price tracking (would TP1/TP2/SL have hit?) ## Integration Points