docs: Complete v9 parameter optimization & backtesting documentation
- v10 removal background (Nov 28, 2025) - v9 baseline performance (05.88, 569 trades, 60.98% WR) - Adaptive leverage implementation (5x high quality, 1x borderline) - Parameter sweep strategy (8 parameters, 65,536 combinations) - EPYC exhaustive sweep status (24 workers, ~17h remaining) - Backtesting infrastructure details - Expected outcomes and analysis plan - Key lessons learned from v10 failure
This commit is contained in:
mindesbunister
142
.github/copilot-instructions.md
vendored
142
.github/copilot-instructions.md
vendored
@@ -5026,6 +5026,148 @@ const tracker = new StopHuntTracker() // ❌ Don't do this
|
|||||||
- Fixed import paths, added error handling, removed debug logs
|
- Fixed import paths, added error handling, removed debug logs
|
||||||
- Full system operational, monitoring active
|
- Full system operational, monitoring active
|
||||||
|
|
||||||
|
## v9 Parameter Optimization & Backtesting (Nov 28-29, 2025)
|
||||||
|
|
||||||
|
**Purpose:** Comprehensive parameter sweep to optimize v9 Money Line indicator for maximum profitability while maintaining quality standards.
|
||||||
|
|
||||||
|
**Background - v10 Removal (Nov 28, 2025):**
|
||||||
|
- **v10 Status:** FULLY REMOVED - discovered to be "garbage" during initial backtest analysis
|
||||||
|
- **v10 Problems Discovered:**
|
||||||
|
1. **Parameter insensitivity:** 72 different configurations produced identical $498.12 P&L
|
||||||
|
2. **Bug in penalty logic:** Price position penalty incorrectly applied to 18.9% position (should only apply to 40-60% chop zone)
|
||||||
|
3. **No edge over v9:** Despite added complexity, no performance improvement
|
||||||
|
- **Removal Actions (Nov 28, 2025):**
|
||||||
|
* Removed moneyline_v10_adaptive_position_scoring.pinescript
|
||||||
|
* Removed v10-specific code from backtester modules
|
||||||
|
* Updated all documentation to remove v10 references
|
||||||
|
* Docker rebuild completed successfully
|
||||||
|
* Git commit: 5f77024 "remove: Complete v10 indicator removal - proven garbage"
|
||||||
|
- **Lesson:** Parameter insensitivity = no real edge, just noise. Simpler is better.
|
||||||
|
|
||||||
|
**v9 Baseline Performance:**
|
||||||
|
- **Data:** Nov 2024 - Nov 2025 SOLUSDT 5-minute OHLCV (139,678 rows)
|
||||||
|
- **Default Parameters:** flip_threshold=0.6, ma_gap=0.35, momentum_adx=23, long_pos=70, short_pos=25, cooldown_bars=2, momentum_spacing=3, momentum_cooldown=2
|
||||||
|
- **Results:** $405.88 PnL, 569 trades, 60.98% WR, 1.022 PF, -$1,360.58 max DD
|
||||||
|
- **Baseline established:** Nov 28, 2025
|
||||||
|
|
||||||
|
**Adaptive Leverage Implementation (Nov 28, 2025):**
|
||||||
|
- **Purpose:** Increase profit potential while maintaining risk management
|
||||||
|
- **Configuration:**
|
||||||
|
```bash
|
||||||
|
USE_ADAPTIVE_LEVERAGE=true
|
||||||
|
HIGH_QUALITY_LEVERAGE=5 # 5x for quality ≥95 LONGs, ≥90 SHORTs
|
||||||
|
LOW_QUALITY_LEVERAGE=1 # 1x for borderline quality (Q90-94 LONGs, Q80-89 SHORTs)
|
||||||
|
QUALITY_LEVERAGE_THRESHOLD=95
|
||||||
|
```
|
||||||
|
- **Rationale:** User chose "option 1" (quality-based adaptive leverage) with 5x instead of 6x for "less risk"
|
||||||
|
- **Direction-Specific Thresholds:**
|
||||||
|
* LONGs: Quality ≥95 → 5x, Quality 90-94 → 1x
|
||||||
|
* SHORTs: Quality ≥90 → 5x, Quality 80-89 → 1x
|
||||||
|
* Lower quality than thresholds → blocked by execute endpoint
|
||||||
|
- **Expected Impact:** 5× profit on high-quality signals without increasing risk on borderline setups
|
||||||
|
- **Status:** ✅ ACTIVE in production (container restarted Nov 28, 2025)
|
||||||
|
- **See:** `ADAPTIVE_LEVERAGE_SYSTEM.md` for implementation details
|
||||||
|
|
||||||
|
**Parameter Sweep Strategy:**
|
||||||
|
- **8 Parameters to Optimize:**
|
||||||
|
1. **flip_threshold:** 0.4, 0.5, 0.6, 0.7 (4 values) - EMA flip confirmation threshold
|
||||||
|
2. **ma_gap:** 0.20, 0.30, 0.40, 0.50 (4 values) - MA50-MA200 convergence bonus
|
||||||
|
3. **momentum_adx:** 18, 21, 24, 27 (4 values) - ADX requirement for momentum filter
|
||||||
|
4. **momentum_long_pos:** 60, 65, 70, 75 (4 values) - Price position for LONG momentum entry
|
||||||
|
5. **momentum_short_pos:** 20, 25, 30, 35 (4 values) - Price position for SHORT momentum entry
|
||||||
|
6. **cooldown_bars:** 1, 2, 3, 4 (4 values) - Bars between signals
|
||||||
|
7. **momentum_spacing:** 2, 3, 4, 5 (4 values) - Bars between momentum confirmations
|
||||||
|
8. **momentum_cooldown:** 1, 2, 3, 4 (4 values) - Momentum-specific cooldown
|
||||||
|
- **Total Combinations:** 4^8 = 65,536 exhaustive search
|
||||||
|
- **Grid Design:** 4 values per parameter = balanced between granularity and computation time
|
||||||
|
|
||||||
|
**Sweep Results - Narrow Grid (27 combinations):**
|
||||||
|
- **Date:** Nov 28, 2025 (killed early to port to EPYC)
|
||||||
|
- **Top Result:** $496.41 PnL (22% improvement over baseline)
|
||||||
|
- **Key Finding:** Parameter insensitivity observed again
|
||||||
|
* Multiple different configurations produced identical results
|
||||||
|
* Suggests v9 edge comes from core EMA logic, not parameter tuning
|
||||||
|
* Similar pattern to v10 (but v9 has proven baseline edge)
|
||||||
|
- **Decision:** Proceed with exhaustive 65,536 combo search on EPYC to confirm pattern
|
||||||
|
|
||||||
|
**EPYC Server Exhaustive Sweep (Nov 28-29, 2025):**
|
||||||
|
- **Hardware:** AMD EPYC 7282 16-Core Processor, Debian 12 Bookworm
|
||||||
|
- **Configuration:** 24 workers, 1.60s per combo (4× faster than local 6 workers)
|
||||||
|
- **Total Combinations:** 65,536 (full 4^8 grid)
|
||||||
|
- **Duration:** ~29 hours estimated
|
||||||
|
- **Output:** Top 100 results saved to sweep_v9_exhaustive_epyc.csv
|
||||||
|
- **Setup:**
|
||||||
|
* Package: backtest_v9_sweep.tar.gz (1.1MB compressed)
|
||||||
|
* Contents: data/solusdt_5m.csv (1.9MB), backtester modules, sweep scripts
|
||||||
|
* Python env: 3.11.2 with pandas 2.3.3, numpy 2.3.5
|
||||||
|
* Virtual environment: /home/backtest/.venv/
|
||||||
|
- **Status:** ✅ RUNNING (started Nov 28, 2025 ~17:00 UTC, ~17h remaining as of Nov 29)
|
||||||
|
- **Critical Fixes Applied:**
|
||||||
|
1. Added `source .venv/bin/activate` to run script (fixes ModuleNotFoundError)
|
||||||
|
2. Kept `--top 100` limit (tests all 65,536, saves top 100 to CSV)
|
||||||
|
3. Proper output naming: sweep_v9_exhaustive_epyc.csv
|
||||||
|
|
||||||
|
**Backtesting Infrastructure:**
|
||||||
|
- **Location:** `/home/icke/traderv4/backtester/` and `/home/backtest/` (EPYC)
|
||||||
|
- **Modules:**
|
||||||
|
* `backtester_core.py` - Core backtesting engine with ATR-based TP/SL
|
||||||
|
* `v9_moneyline_ma_gap.py` - v9 indicator logic implementation
|
||||||
|
* `moneyline_core.py` - Shared EMA/signal detection logic
|
||||||
|
- **Data:** `data/solusdt_5m.csv` - Nov 2024 to Nov 2025 OHLCV (139,678 5-min bars)
|
||||||
|
- **Sweep Script:** `scripts/run_backtest_sweep.py` - Multiprocessing parameter grid search
|
||||||
|
* Progress bar shows hours/minutes (not seconds) for long-running sweeps
|
||||||
|
* Supports --top N to limit output file size
|
||||||
|
* Uses multiprocessing.Pool for parallel execution
|
||||||
|
- **Python Environments:**
|
||||||
|
* Local: Python 3.7.3 with .venv (pandas/numpy)
|
||||||
|
* EPYC: Python 3.11.2 with .venv (pandas 2.3.3, numpy 2.3.5)
|
||||||
|
- **Setup Scripts:**
|
||||||
|
* `setup_epyc.sh` - Installs python3-venv, creates .venv, installs pandas/numpy
|
||||||
|
* `run_sweep_epyc.sh` - Executes parameter sweep with proper venv activation
|
||||||
|
|
||||||
|
**Expected Outcomes:**
|
||||||
|
1. **If parameter insensitivity persists:** v9 edge is in core EMA logic, not tuning
|
||||||
|
- Action: Use baseline parameters in production
|
||||||
|
- Conclusion: v9 works because of momentum filter logic, not specific values
|
||||||
|
2. **If clear winners emerge:** Optimize production parameters
|
||||||
|
- Action: Update .pinescript with optimal values
|
||||||
|
- Validation: Confirm via forward testing (50-100 trades)
|
||||||
|
3. **If quality thresholds need adjustment:**
|
||||||
|
- SHORT threshold 80 may be too strict (could be missing profitable setups)
|
||||||
|
- Analyze win rate distribution around thresholds
|
||||||
|
|
||||||
|
**Post-Sweep Analysis Plan:**
|
||||||
|
1. Review top 100 results for parameter clustering
|
||||||
|
2. Check if top performers share common characteristics
|
||||||
|
3. Identify "stability zones" (parameters that consistently perform well)
|
||||||
|
4. Compare exhaustive results to baseline ($405.88) and narrow sweep ($496.41)
|
||||||
|
5. Make production parameter recommendations
|
||||||
|
6. Consider if SHORT quality threshold (80) needs lowering based on blocked signals analysis
|
||||||
|
|
||||||
|
**Key Files:**
|
||||||
|
- `workflows/trading/moneyline_v9_ma_gap.pinescript` - Production v9 indicator
|
||||||
|
- `backtester/v9_moneyline_ma_gap.py` - Python implementation for backtesting
|
||||||
|
- `scripts/run_backtest_sweep.py` - Parameter sweep orchestration
|
||||||
|
- `run_sweep_epyc.sh` - EPYC execution script (24 workers, venv activation)
|
||||||
|
- `ADAPTIVE_LEVERAGE_SYSTEM.md` - Adaptive leverage implementation docs
|
||||||
|
- `INDICATOR_V9_MA_GAP_ROADMAP.md` - v9 development roadmap
|
||||||
|
|
||||||
|
**Current Production State (Nov 28-29, 2025):**
|
||||||
|
- **Indicator:** v9 Money Line with MA Gap + Momentum SHORT Filter
|
||||||
|
- **Quality Thresholds:** LONG ≥90, SHORT ≥80
|
||||||
|
- **Adaptive Leverage:** ACTIVE (5x high quality, 1x borderline)
|
||||||
|
- **Capital:** $540 USDC at 100% health
|
||||||
|
- **Expected Profit Boost:** 5× on high-quality signals with adaptive leverage
|
||||||
|
- **Backtesting:** Exhaustive parameter sweep in progress (17h remaining)
|
||||||
|
|
||||||
|
**Lessons Learned:**
|
||||||
|
1. **Parameter insensitivity indicates overfitting:** When many configs give identical results, the edge isn't in parameters
|
||||||
|
2. **Simpler is better:** v10 added complexity but no edge → removed completely
|
||||||
|
3. **Quality-based leverage scales winners:** 5x on Q95+ signals amplifies edge without increasing borderline risk
|
||||||
|
4. **Exhaustive search validates findings:** 65,536 combos confirm if pattern is real or sampling artifact
|
||||||
|
5. **Python environments matter:** Always activate venv before running backtests on remote servers
|
||||||
|
6. **Portable packages enable distributed computing:** 1.1MB tar.gz enables 16-core EPYC utilization
|
||||||
|
|
||||||
## Integration Points
|
## Integration Points
|
||||||
|
|
||||||
- **n8n:** Expects exact response format from `/api/trading/execute` (see n8n-complete-workflow.json)
|
- **n8n:** Expects exact response format from `/api/trading/execute` (see n8n-complete-workflow.json)
|
||||||
|
|||||||
Reference in New Issue
Block a user