diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 42f2311..2db5625 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -2817,7 +2817,75 @@ This section contains the **TOP 10 MOST CRITICAL** pitfalls that every AI agent 📚 **Full Documentation:** `docs/COMMON_PITFALLS.md` (72 pitfalls with code examples, git commits, deployment dates) -72. **CRITICAL SECURITY: .env file tracked in git (CRITICAL - Fixed Dec 5, 2025 - PR #3):** +72. **CRITICAL: MFE Data Unit Mismatch - ALWAYS Filter by Date (CRITICAL - Dec 5, 2025):** + - **Symptom:** SQL analysis shows "20%+ average MFE" but TP1 (0.6% target) never hits + - **Root Cause:** Old Trade records stored MFE/MAE in DOLLARS, new records store PERCENTAGES + - **Data Corruption Examples:** + * Entry $126.51, Peak $128.21 = 1.35% actual move + * But stored as maxFavorableExcursion = 90.73 (dollars, not percent) + * SQL AVG() returns meaningless mix: (1.35 + 90.73 + 0.85 + 87.22) / 4 = 45.04 + - **Incident (Dec 5, 2025):** + * Agent analyzed blocked vs executed signals + * SQL showed executed signals: 20.15% avg MFE (appeared AMAZING) + * Implemented "optimizations": tighter targets, higher TP1 close, 5× leverage + * User questioned: "tp1 barely hits that has nothing to do with our software monitoring does it?" + * Investigation revealed: Only 2/11 trades reached TP1 price + * TRUE MFE after filtering: 0.76% (long), 1.20% (short) - NOT 20%! + * 26× inflation due to unit mismatch in old data + - **MANDATORY SQL Pattern:** + ```sql + -- WRONG: Includes corrupted old data + SELECT AVG("maxFavorableExcursion") FROM "Trade" + WHERE "signalQualityScore" >= 90; + + -- CORRECT: Filter to after Nov 23, 2025 fix + SELECT AVG("maxFavorableExcursion") FROM "Trade" + WHERE "signalQualityScore" >= 90 + AND "createdAt" >= '2025-11-23'; -- After MFE fix + + -- OR: Recalculate from prices (always correct) + SELECT AVG( + CASE + WHEN direction = 'long' THEN + (("maxFavorablePrice" - "entryPrice") / "entryPrice") * 100 + ELSE + (("entryPrice" - "maxFavorablePrice") / "entryPrice") * 100 + END + ) FROM "Trade" WHERE "signalQualityScore" >= 90; + ``` + - **Why This Matters:** + * **This is a REAL MONEY system** - wrong analysis = wrong trades = financial losses + * MFE/MAE used for exit timing optimization, trade analysis, quality validation + * Agent made "data-driven" decisions based on 26× inflated numbers + * Optimizations REVERTED via commits f65aae5 and a67a338 (Dec 5, 2025) + - **Verification Before Any MFE/MAE Analysis:** + ```sql + -- Check if data is percentages or dollars + SELECT + "entryPrice", + "maxFavorablePrice", + "maxFavorableExcursion" as stored, + CASE + WHEN direction = 'long' THEN + (("maxFavorablePrice" - "entryPrice") / "entryPrice") * 100 + ELSE + (("entryPrice" - "maxFavorablePrice") / "entryPrice") * 100 + END as calculated_pct + FROM "Trade" + WHERE "exitReason" IS NOT NULL + ORDER BY "createdAt" DESC + LIMIT 5; + + -- If stored ≠ calculated_pct → OLD DATA, use date filter + ``` + - **See Also:** + * Common Pitfall #54 - MFE/MAE stored as dollars (supposedly fixed Nov 23) + * Revert commits: a15f17f "revert: Undo exit strategy optimization based on corrupted MFE data" + * Original bug commits: a67a338 (code), f65aae5 (docs) + - **Git commits:** a15f17f (revert), a67a338 (incorrect optimization), f65aae5 (incorrect docs) + - **Status:** ✅ Fixed - Analysis methodology documented, incorrect changes reverted + +73. **CRITICAL SECURITY: .env file tracked in git (CRITICAL - Fixed Dec 5, 2025 - PR #3):** - **Symptom:** Sensitive credentials exposed in git repository history - **Credentials exposed:** * Database connection strings (PostgreSQL)