docs: Add Common Pitfall #72 - MFE data unit mismatch

CRITICAL LESSON LEARNED (Dec 5, 2025):
Document the data analysis disaster caused by MFE/MAE stored in mixed units.

What Happened:
- Analyzed blocked vs executed signals to improve win rate
- SQL showed executed signals: 20.15% avg MFE (appeared excellent)
- Implemented "optimizations" based on this data:
  * Tighter ATR multipliers (2.0→1.5, 4.0→3.0)
  * Higher TP1 close (60%→75%)
  * Increased leverage (1×→5×)
- User questioned: Why doesn't TP1 hit if it's 20% MFE?
- Investigation: Only 2/11 trades reached TP1 price target
- Root cause: Old records stored MFE in DOLLARS, new in PERCENTAGES
- TRUE MFE: 0.76% (long), 1.20% (short) - NOT 20%!
- 26× inflation due to unit mismatch

Why This Matters:
- This is a REAL MONEY system - wrong analysis = wrong trades = losses
- MFE/MAE used for critical decisions (exit timing, quality validation)
- Agent made "data-driven" optimizations on 26× inflated data
- All changes had to be reverted (commits a67a338, f65aae5)

MANDATORY SQL Pattern:
- ALWAYS filter by createdAt >= '2025-11-23' for MFE/MAE queries
- OR recalculate from prices (maxFavorablePrice - entryPrice)
- NEVER trust raw AVG(maxFavorableExcursion) without date filter

Prevention:
- Verify stored vs calculated values before ANY MFE/MAE analysis
- Check sample of recent vs old records to detect unit changes
- Document data format changes in Common Pitfalls immediately

Related:
- Common Pitfall #54: Original MFE/MAE units bug (Nov 23, 2025)
- Revert commit: a15f17f
- Incorrect optimization: a67a338, f65aae5
This commit is contained in:
mindesbunister
2025-12-05 10:07:06 +01:00
parent a15f17f489
commit 8ec4eb0782

View File

@@ -2817,7 +2817,75 @@ This section contains the **TOP 10 MOST CRITICAL** pitfalls that every AI agent
📚 **Full Documentation:** `docs/COMMON_PITFALLS.md` (72 pitfalls with code examples, git commits, deployment dates)
72. **CRITICAL SECURITY: .env file tracked in git (CRITICAL - Fixed Dec 5, 2025 - PR #3):**
72. **CRITICAL: MFE Data Unit Mismatch - ALWAYS Filter by Date (CRITICAL - Dec 5, 2025):**
- **Symptom:** SQL analysis shows "20%+ average MFE" but TP1 (0.6% target) never hits
- **Root Cause:** Old Trade records stored MFE/MAE in DOLLARS, new records store PERCENTAGES
- **Data Corruption Examples:**
* Entry $126.51, Peak $128.21 = 1.35% actual move
* But stored as maxFavorableExcursion = 90.73 (dollars, not percent)
* SQL AVG() returns meaningless mix: (1.35 + 90.73 + 0.85 + 87.22) / 4 = 45.04
- **Incident (Dec 5, 2025):**
* Agent analyzed blocked vs executed signals
* SQL showed executed signals: 20.15% avg MFE (appeared AMAZING)
* Implemented "optimizations": tighter targets, higher TP1 close, 5× leverage
* User questioned: "tp1 barely hits that has nothing to do with our software monitoring does it?"
* Investigation revealed: Only 2/11 trades reached TP1 price
* TRUE MFE after filtering: 0.76% (long), 1.20% (short) - NOT 20%!
* 26× inflation due to unit mismatch in old data
- **MANDATORY SQL Pattern:**
```sql
-- WRONG: Includes corrupted old data
SELECT AVG("maxFavorableExcursion") FROM "Trade"
WHERE "signalQualityScore" >= 90;
-- CORRECT: Filter to after Nov 23, 2025 fix
SELECT AVG("maxFavorableExcursion") FROM "Trade"
WHERE "signalQualityScore" >= 90
AND "createdAt" >= '2025-11-23'; -- After MFE fix
-- OR: Recalculate from prices (always correct)
SELECT AVG(
CASE
WHEN direction = 'long' THEN
(("maxFavorablePrice" - "entryPrice") / "entryPrice") * 100
ELSE
(("entryPrice" - "maxFavorablePrice") / "entryPrice") * 100
END
) FROM "Trade" WHERE "signalQualityScore" >= 90;
```
- **Why This Matters:**
* **This is a REAL MONEY system** - wrong analysis = wrong trades = financial losses
* MFE/MAE used for exit timing optimization, trade analysis, quality validation
* Agent made "data-driven" decisions based on 26× inflated numbers
* Optimizations REVERTED via commits f65aae5 and a67a338 (Dec 5, 2025)
- **Verification Before Any MFE/MAE Analysis:**
```sql
-- Check if data is percentages or dollars
SELECT
"entryPrice",
"maxFavorablePrice",
"maxFavorableExcursion" as stored,
CASE
WHEN direction = 'long' THEN
(("maxFavorablePrice" - "entryPrice") / "entryPrice") * 100
ELSE
(("entryPrice" - "maxFavorablePrice") / "entryPrice") * 100
END as calculated_pct
FROM "Trade"
WHERE "exitReason" IS NOT NULL
ORDER BY "createdAt" DESC
LIMIT 5;
-- If stored ≠ calculated_pct → OLD DATA, use date filter
```
- **See Also:**
* Common Pitfall #54 - MFE/MAE stored as dollars (supposedly fixed Nov 23)
* Revert commits: a15f17f "revert: Undo exit strategy optimization based on corrupted MFE data"
* Original bug commits: a67a338 (code), f65aae5 (docs)
- **Git commits:** a15f17f (revert), a67a338 (incorrect optimization), f65aae5 (incorrect docs)
- **Status:** ✅ Fixed - Analysis methodology documented, incorrect changes reverted
73. **CRITICAL SECURITY: .env file tracked in git (CRITICAL - Fixed Dec 5, 2025 - PR #3):**
- **Symptom:** Sensitive credentials exposed in git repository history
- **Credentials exposed:**
* Database connection strings (PostgreSQL)