diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 998b10c..ad04684 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -414,6 +414,104 @@ docker system df --- +## Multi-Timeframe Price Tracking System (Nov 19, 2025) + +**Purpose:** Automated data collection and analysis for signals across multiple timeframes (5min, 15min, 1H, 4H, Daily) to determine which timeframe produces the best trading results. + +**Architecture:** +- **5min signals:** Execute trades (production) +- **15min/1H/4H/Daily signals:** Save to BlockedSignal table with `blockReason='DATA_COLLECTION_ONLY'` +- **Background tracker:** Runs every 5 minutes, monitors price movements for 30 minutes +- **Analysis:** After 50+ signals per timeframe, compare win rates and profit potential + +**Components:** + +1. **BlockedSignalTracker** (`lib/analysis/blocked-signal-tracker.ts`) + - Background job running every 5 minutes + - Tracks price at 1min, 5min, 15min, 30min intervals + - Detects if TP1/TP2/SL would have been hit using ATR-based targets + - Records max favorable/adverse excursion (MFE/MAE) + - Auto-completes after 30 minutes (`analysisComplete=true`) + - Singleton pattern: Use `getBlockedSignalTracker()` or `startBlockedSignalTracking()` + +2. **Database Schema** (BlockedSignal table) + ```sql + entryPrice FLOAT -- Price at signal time (baseline) + priceAfter1Min FLOAT? -- Price 1 minute after + priceAfter5Min FLOAT? -- Price 5 minutes after + priceAfter15Min FLOAT? -- Price 15 minutes after + priceAfter30Min FLOAT? -- Price 30 minutes after + wouldHitTP1 BOOLEAN? -- Would TP1 have been hit? + wouldHitTP2 BOOLEAN? -- Would TP2 have been hit? + wouldHitSL BOOLEAN? -- Would SL have been hit? + maxFavorablePrice FLOAT? -- Price at max profit + maxAdversePrice FLOAT? -- Price at max loss + maxFavorableExcursion FLOAT? -- Best profit % during 30min + maxAdverseExcursion FLOAT? -- Worst loss % during 30min + analysisComplete BOOLEAN -- Tracking finished (30min elapsed) + ``` + +3. **API Endpoints** + - `GET /api/analytics/signal-tracking` - View tracking status, metrics, recent signals + - `POST /api/analytics/signal-tracking` - Manually trigger tracking update (auth required) + +4. **Integration Points** + - Execute endpoint: Captures entry price when saving DATA_COLLECTION_ONLY signals + - Startup: Auto-starts tracker via `initializePositionManagerOnStartup()` + - Check-risk endpoint: Bypasses quality checks for non-5min signals (lines 147-159) + +**How It Works:** +1. TradingView sends 15min/1H/4H/Daily signal → n8n → `/api/trading/execute` +2. Execute endpoint detects `timeframe !== '5'` +3. Gets current price from Pyth, saves to BlockedSignal with `entryPrice` +4. Background tracker wakes every 5 minutes +5. Queries current price, calculates profit % based on direction +6. Checks if TP1 (~0.86%), TP2 (~1.72%), or SL (~1.29%) would have hit +7. Updates price fields at appropriate intervals (1/5/15/30 min) +8. Tracks MFE/MAE throughout 30-minute window +9. After 30 minutes, marks `analysisComplete=true` + +**Analysis Queries (After 50+ signals per timeframe):** +```sql +-- Compare win rates across timeframes +SELECT + timeframe, + COUNT(*) as total_signals, + COUNT(CASE WHEN wouldHitTP1 = true THEN 1 END) as tp1_wins, + COUNT(CASE WHEN wouldHitSL = true THEN 1 END) as sl_losses, + ROUND(100.0 * COUNT(CASE WHEN wouldHitTP1 = true THEN 1 END) / COUNT(*), 1) as win_rate, + ROUND(AVG(maxFavorableExcursion), 2) as avg_mfe, + ROUND(AVG(maxAdverseExcursion), 2) as avg_mae +FROM "BlockedSignal" +WHERE analysisComplete = true + AND blockReason = 'DATA_COLLECTION_ONLY' +GROUP BY timeframe +ORDER BY win_rate DESC; +``` + +**Decision Making:** +After sufficient data collected: +- Compare: 5min vs 15min vs 1H vs 4H vs Daily win rates +- Evaluate: Signal frequency (trades/day) vs win rate trade-off +- Identify: Which timeframe has best TP1 hit rate with acceptable MAE +- Action: Switch production timeframe if higher timeframe shows significantly better results + +**Key Features:** +- **Autonomous:** No manual work needed, runs in background +- **Accurate:** Uses same TP/SL calculations as live trades (ATR-based) +- **Risk-free:** Data collection only, no money at risk +- **Comprehensive:** Tracks best/worst case scenarios (MFE/MAE) +- **API accessible:** Check status anytime via `/api/analytics/signal-tracking` + +**Current Status (Nov 19, 2025):** +- ✅ System deployed and running +- ✅ 2 signals tracked (15min and 1H from earlier today) +- ✅ TradingView alerts configured for 15min and 1H +- 📋 Pending: Set up 4H and Daily alerts +- 📊 Target: Collect 50+ signals per timeframe for statistical analysis + +--- + ## Critical Components ### 1. Phantom Trade Auto-Closure System @@ -2882,14 +2980,33 @@ if (!enabled) { ``` - Types: `feat:` (feature), `fix:` (bug fix), `docs:` (documentation), `refactor:` (code restructure) - This is NOT optional - code exists only when committed and pushed -13. **NEXTCLOUD DECK SYNC (MANDATORY):** After completing phases or making significant roadmap progress: +13. **DOCKER MAINTENANCE (AFTER BUILDS):** Clean up accumulated cache to prevent disk full: + ```bash + # Remove dangling images (old builds) + docker image prune -f + + # Remove build cache (biggest space hog - 40+ GB typical) + docker builder prune -f + + # Optional: Remove dangling volumes (if no important data) + docker volume prune -f + + # Check space saved + docker system df + ``` + - **When to run:** After successful deployments, weekly if building frequently, when disk warnings appear + - **Space freed:** Dangling images (2-5 GB), Build cache (40-50 GB), Dangling volumes (0.5-1 GB) + - **Safe to delete:** `` tagged images, build cache (recreated on next build), dangling volumes + - **Keep:** Named volumes (`trading-bot-postgres`), active containers, tagged images in use + - **Why critical:** Docker builds create 1.3+ GB per build, cache accumulates to 40-50 GB without cleanup +14. **NEXTCLOUD DECK SYNC (MANDATORY):** After completing phases or making significant roadmap progress: - Update roadmap markdown files with new status (🔄 IN PROGRESS, ✅ COMPLETE, 🔜 NEXT) - Run sync to update Deck cards: `python3 scripts/sync-roadmap-to-deck.py --init` - Move cards between stacks in Nextcloud Deck UI to reflect progress visually - Backlog (📥) → Planning (📋) → In Progress (🚀) → Complete (✅) - Keep Deck in sync with actual work - it's the visual roadmap tracker - Documentation: `docs/NEXTCLOUD_DECK_SYNC.md` -14. **UPDATE COPILOT-INSTRUCTIONS.MD (MANDATORY):** After implementing ANY significant feature or system change: +15. **UPDATE COPILOT-INSTRUCTIONS.MD (MANDATORY):** After implementing ANY significant feature or system change: - Document new database fields and their purpose - Add filtering requirements (e.g., manual vs TradingView trades) - Update "Important fields" sections with new schema changes