docs: Add Common Pitfall #72 - MFE data unit mismatch
CRITICAL LESSON LEARNED (Dec 5, 2025): Document the data analysis disaster caused by MFE/MAE stored in mixed units. What Happened: - Analyzed blocked vs executed signals to improve win rate - SQL showed executed signals: 20.15% avg MFE (appeared excellent) - Implemented "optimizations" based on this data: * Tighter ATR multipliers (2.0→1.5, 4.0→3.0) * Higher TP1 close (60%→75%) * Increased leverage (1×→5×) - User questioned: Why doesn't TP1 hit if it's 20% MFE? - Investigation: Only 2/11 trades reached TP1 price target - Root cause: Old records stored MFE in DOLLARS, new in PERCENTAGES - TRUE MFE: 0.76% (long), 1.20% (short) - NOT 20%! - 26× inflation due to unit mismatch Why This Matters: - This is a REAL MONEY system - wrong analysis = wrong trades = losses - MFE/MAE used for critical decisions (exit timing, quality validation) - Agent made "data-driven" optimizations on 26× inflated data - All changes had to be reverted (commitsa67a338,f65aae5) MANDATORY SQL Pattern: - ALWAYS filter by createdAt >= '2025-11-23' for MFE/MAE queries - OR recalculate from prices (maxFavorablePrice - entryPrice) - NEVER trust raw AVG(maxFavorableExcursion) without date filter Prevention: - Verify stored vs calculated values before ANY MFE/MAE analysis - Check sample of recent vs old records to detect unit changes - Document data format changes in Common Pitfalls immediately Related: - Common Pitfall #54: Original MFE/MAE units bug (Nov 23, 2025) - Revert commit:a15f17f- Incorrect optimization:a67a338,f65aae5
This commit is contained in:
70
.github/copilot-instructions.md
vendored
70
.github/copilot-instructions.md
vendored
@@ -2817,7 +2817,75 @@ This section contains the **TOP 10 MOST CRITICAL** pitfalls that every AI agent
|
||||
|
||||
📚 **Full Documentation:** `docs/COMMON_PITFALLS.md` (72 pitfalls with code examples, git commits, deployment dates)
|
||||
|
||||
72. **CRITICAL SECURITY: .env file tracked in git (CRITICAL - Fixed Dec 5, 2025 - PR #3):**
|
||||
72. **CRITICAL: MFE Data Unit Mismatch - ALWAYS Filter by Date (CRITICAL - Dec 5, 2025):**
|
||||
- **Symptom:** SQL analysis shows "20%+ average MFE" but TP1 (0.6% target) never hits
|
||||
- **Root Cause:** Old Trade records stored MFE/MAE in DOLLARS, new records store PERCENTAGES
|
||||
- **Data Corruption Examples:**
|
||||
* Entry $126.51, Peak $128.21 = 1.35% actual move
|
||||
* But stored as maxFavorableExcursion = 90.73 (dollars, not percent)
|
||||
* SQL AVG() returns meaningless mix: (1.35 + 90.73 + 0.85 + 87.22) / 4 = 45.04
|
||||
- **Incident (Dec 5, 2025):**
|
||||
* Agent analyzed blocked vs executed signals
|
||||
* SQL showed executed signals: 20.15% avg MFE (appeared AMAZING)
|
||||
* Implemented "optimizations": tighter targets, higher TP1 close, 5× leverage
|
||||
* User questioned: "tp1 barely hits that has nothing to do with our software monitoring does it?"
|
||||
* Investigation revealed: Only 2/11 trades reached TP1 price
|
||||
* TRUE MFE after filtering: 0.76% (long), 1.20% (short) - NOT 20%!
|
||||
* 26× inflation due to unit mismatch in old data
|
||||
- **MANDATORY SQL Pattern:**
|
||||
```sql
|
||||
-- WRONG: Includes corrupted old data
|
||||
SELECT AVG("maxFavorableExcursion") FROM "Trade"
|
||||
WHERE "signalQualityScore" >= 90;
|
||||
|
||||
-- CORRECT: Filter to after Nov 23, 2025 fix
|
||||
SELECT AVG("maxFavorableExcursion") FROM "Trade"
|
||||
WHERE "signalQualityScore" >= 90
|
||||
AND "createdAt" >= '2025-11-23'; -- After MFE fix
|
||||
|
||||
-- OR: Recalculate from prices (always correct)
|
||||
SELECT AVG(
|
||||
CASE
|
||||
WHEN direction = 'long' THEN
|
||||
(("maxFavorablePrice" - "entryPrice") / "entryPrice") * 100
|
||||
ELSE
|
||||
(("entryPrice" - "maxFavorablePrice") / "entryPrice") * 100
|
||||
END
|
||||
) FROM "Trade" WHERE "signalQualityScore" >= 90;
|
||||
```
|
||||
- **Why This Matters:**
|
||||
* **This is a REAL MONEY system** - wrong analysis = wrong trades = financial losses
|
||||
* MFE/MAE used for exit timing optimization, trade analysis, quality validation
|
||||
* Agent made "data-driven" decisions based on 26× inflated numbers
|
||||
* Optimizations REVERTED via commits f65aae5 and a67a338 (Dec 5, 2025)
|
||||
- **Verification Before Any MFE/MAE Analysis:**
|
||||
```sql
|
||||
-- Check if data is percentages or dollars
|
||||
SELECT
|
||||
"entryPrice",
|
||||
"maxFavorablePrice",
|
||||
"maxFavorableExcursion" as stored,
|
||||
CASE
|
||||
WHEN direction = 'long' THEN
|
||||
(("maxFavorablePrice" - "entryPrice") / "entryPrice") * 100
|
||||
ELSE
|
||||
(("entryPrice" - "maxFavorablePrice") / "entryPrice") * 100
|
||||
END as calculated_pct
|
||||
FROM "Trade"
|
||||
WHERE "exitReason" IS NOT NULL
|
||||
ORDER BY "createdAt" DESC
|
||||
LIMIT 5;
|
||||
|
||||
-- If stored ≠ calculated_pct → OLD DATA, use date filter
|
||||
```
|
||||
- **See Also:**
|
||||
* Common Pitfall #54 - MFE/MAE stored as dollars (supposedly fixed Nov 23)
|
||||
* Revert commits: a15f17f "revert: Undo exit strategy optimization based on corrupted MFE data"
|
||||
* Original bug commits: a67a338 (code), f65aae5 (docs)
|
||||
- **Git commits:** a15f17f (revert), a67a338 (incorrect optimization), f65aae5 (incorrect docs)
|
||||
- **Status:** ✅ Fixed - Analysis methodology documented, incorrect changes reverted
|
||||
|
||||
73. **CRITICAL SECURITY: .env file tracked in git (CRITICAL - Fixed Dec 5, 2025 - PR #3):**
|
||||
- **Symptom:** Sensitive credentials exposed in git repository history
|
||||
- **Credentials exposed:**
|
||||
* Database connection strings (PostgreSQL)
|
||||
|
||||
Reference in New Issue
Block a user