docs: Major documentation reorganization + ENV variable reference
**Documentation Structure:** - Created docs/ subdirectory organization (analysis/, architecture/, bugs/, cluster/, deployments/, roadmaps/, setup/, archived/) - Moved 68 root markdown files to appropriate categories - Root directory now clean (only README.md remains) - Total: 83 markdown files now organized by purpose **New Content:** - Added comprehensive Environment Variable Reference to copilot-instructions.md - 100+ ENV variables documented with types, defaults, purpose, notes - Organized by category: Required (Drift/RPC/Pyth), Trading Config (quality/ leverage/sizing), ATR System, Runner System, Risk Limits, Notifications, etc. - Includes usage examples (correct vs wrong patterns) **File Distribution:** - docs/analysis/ - Performance analyses, blocked signals, profit projections - docs/architecture/ - Adaptive leverage, ATR trailing, indicator tracking - docs/bugs/ - CRITICAL_*.md, FIXES_*.md bug reports (7 files) - docs/cluster/ - EPYC setup, distributed computing docs (3 files) - docs/deployments/ - *_COMPLETE.md, DEPLOYMENT_*.md status (12 files) - docs/roadmaps/ - All *ROADMAP*.md strategic planning files (7 files) - docs/setup/ - TradingView guides, signal quality, n8n setup (8 files) - docs/archived/2025_pre_nov/ - Obsolete verification checklist (1 file) **Key Improvements:** - ENV variable reference: Single source of truth for all configuration - Common Pitfalls #68-71: Already complete, verified during audit - Better findability: Category-based navigation vs 68 files in root - Preserves history: All files git mv (rename), not copy/delete - Zero broken functionality: Only documentation moved, no code changes **Verification:** - 83 markdown files now in docs/ subdirectories - Root directory cleaned: 68 files → 0 files (except README.md) - Git history preserved for all moved files - Container running: trading-bot-v4 (no restart needed) **Next Steps:** - Create README.md files in each docs subdirectory - Add navigation index - Update main README.md with new structure - Consolidate duplicate deployment docs - Archive truly obsolete files (old SQL backups) See: docs/analysis/CLEANUP_PLAN.md for complete reorganization strategy
This commit is contained in:
mindesbunister
104
docs/analysis/DIAGNOSTIC_RESULTS_SUMMARY.md
Normal file
104
docs/analysis/DIAGNOSTIC_RESULTS_SUMMARY.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# V9 Diagnostic Results Summary
|
||||
**Date:** November 29, 2025
|
||||
**Data:** 95,617 5-minute bars (SOLUSDT, Jan-Nov 2024)
|
||||
|
||||
## 🚨 CRITICAL FINDINGS
|
||||
|
||||
### 1. **BASELINE IS LOSING MONEY**
|
||||
- **Baseline PnL:** -$1,532.30 (1,663 trades)
|
||||
- **Win Rate:** 0.6% (essentially all losses!)
|
||||
- **This explains the "parameter insensitivity"** - when strategy loses on EVERY trade, parameters don't matter much
|
||||
|
||||
### 2. **momentum_min_adx Parameter BROKEN**
|
||||
```
|
||||
momentum_min_adx=18.0: 1663 trades, $-1532.30 PnL
|
||||
momentum_min_adx=21.0: 1663 trades, $-1532.30 PnL ← IDENTICAL
|
||||
momentum_min_adx=24.0: 1663 trades, $-1532.30 PnL ← IDENTICAL
|
||||
momentum_min_adx=27.0: 1663 trades, $-1532.30 PnL ← IDENTICAL
|
||||
```
|
||||
**Status:** 🔴 NO EFFECT - Parameter is NOT being applied or is overridden
|
||||
|
||||
### 3. **Other Parameters Show Minimal Effect**
|
||||
- **flip_threshold:** 1662-1663 trades (0.1% variation), PnL: -$1,185 to -$1,532
|
||||
- **cooldown_bars:** 1660-1664 trades (0.2% variation), PnL: -$1,408 to -$1,859
|
||||
- **ma_gap_threshold:** 1662-1663 trades (0.1% variation), PnL: -$1,185 to -$1,532
|
||||
|
||||
**Signal counts barely change** - most parameters have almost zero effect on trade generation.
|
||||
|
||||
## 📊 Comparison to Sweep Results
|
||||
|
||||
**Exhaustive Sweep (EPYC):**
|
||||
- Best Result: $498.12 PnL, 568 trades, 61.09% WR
|
||||
- Configuration: Different from baseline
|
||||
|
||||
**Diagnostic Test (Local):**
|
||||
- Baseline: -$1,532.30 PnL, 1,663 trades, 0.6% WR
|
||||
- Best: -$1,514.75 PnL, 1,663 trades, 0.6% WR
|
||||
|
||||
## 🤔 Why The Discrepancy?
|
||||
|
||||
### Hypothesis 1: Data Mismatch
|
||||
- **EPYC used:** Aug 1 - Nov 28, 2024 (34,273 candles - mentioned in DUAL_SWEEP_README.md)
|
||||
- **Local used:** Jan 1 - Nov 28, 2024 (95,617 candles - full year)
|
||||
- **Impact:** Different time periods = different market conditions = different results
|
||||
|
||||
### Hypothesis 2: Configuration Mismatch
|
||||
- EPYC sweep might be using different TradeConfig settings
|
||||
- Position size, max bars per trade, or other simulator settings might differ
|
||||
|
||||
### Hypothesis 3: Strategy Implementation Difference
|
||||
- Backtester `simulate_money_line()` might not match live v9 indicator
|
||||
- Parameters might not map correctly between TradingView and Python
|
||||
|
||||
## 🎯 Action Items
|
||||
|
||||
### IMMEDIATE (Before Any Optimization):
|
||||
|
||||
1. **✅ VERIFY DATA ALIGNMENT**
|
||||
```bash
|
||||
# Download exact same date range as EPYC
|
||||
python3 scripts/export_binance_ohlcv.py \
|
||||
--symbol SOLUSDT --interval 5m \
|
||||
--start 2024-08-01 --end 2024-11-28 \
|
||||
--output backtester/data/solusdt_5m_aug_nov.csv
|
||||
|
||||
# Re-run diagnostics on matched dataset
|
||||
./run_comprehensive_diagnostics.sh backtester/data/solusdt_5m_aug_nov.csv
|
||||
```
|
||||
|
||||
2. **VERIFY SIMULATOR SETTINGS**
|
||||
- Check if EPYC sweep uses different position_size or max_bars_per_trade
|
||||
- Compare TradeConfig between sweep script and diagnostic scripts
|
||||
|
||||
3. **FIX momentum_min_adx BUG**
|
||||
- Investigate money_line_signals() to find why ADX parameter is ignored
|
||||
- This is likely why all sweep configs produced similar results
|
||||
|
||||
4. **FIX EXTREME BUGS**
|
||||
- Fix load_csv() call in test_extreme_configs() (missing symbol/timeframe)
|
||||
- Fix SimulatedTrade.pnl attribute access in trade_analysis.py
|
||||
|
||||
### AFTER VERIFICATION:
|
||||
|
||||
5. **If Data Mismatch Confirmed:**
|
||||
- Use Aug-Nov 2024 dataset for all future analysis
|
||||
- Understand why Q1-Q3 2024 was so terrible (bear market?)
|
||||
|
||||
6. **If Simulator Bug Confirmed:**
|
||||
- Fix Python backtester to match TradingView v9 exactly
|
||||
- Validate against known live trades
|
||||
|
||||
7. **Parameter Optimization:**
|
||||
- Only optimize AFTER baseline is profitable on test data
|
||||
- No point optimizing if strategy loses money fundamentally
|
||||
|
||||
## 💡 Key Insight
|
||||
|
||||
**You can't optimize a fundamentally losing strategy.**
|
||||
|
||||
If v9 baseline loses $1,532 on full-year data but makes $498 on Aug-Nov subset, either:
|
||||
- A) Aug-Nov was a favorable period (cherry-picked results)
|
||||
- B) Jan-Jul market was unfavorable for momentum strategies (bear market)
|
||||
- C) Backtester doesn't match production v9 indicator
|
||||
|
||||
**Must resolve this before any parameter tuning!**
|
||||
Reference in New Issue
Block a user