docs: Add ASCII architecture diagram for v11 test sweep system
Co-authored-by: mindesbunister <32161838+mindesbunister@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
216
cluster/ARCHITECTURE.txt
Normal file
216
cluster/ARCHITECTURE.txt
Normal file
@@ -0,0 +1,216 @@
|
||||
V11 Test Parameter Sweep - System Architecture
|
||||
================================================
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ LOCAL MACHINE │
|
||||
│ /home/icke/traderv4/cluster/ │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 1. deploy_v11_test.sh │
|
||||
│ └─> Syncs files to EPYC cluster │
|
||||
│ │
|
||||
│ Files deployed: │
|
||||
│ • v11_test_coordinator.py │
|
||||
│ • v11_test_worker.py │
|
||||
│ • run_v11_test_sweep.sh │
|
||||
│ • backtester/v11_moneyline_all_filters.py │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ rsync via SSH
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ WORKER 1 (pve-nu-monitor01) │
|
||||
│ root@10.10.254.106:/home/comprehensive_sweep │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 2. run_v11_test_sweep.sh │
|
||||
│ └─> Initializes database │
|
||||
│ └─> Launches coordinator │
|
||||
│ │
|
||||
│ 3. v11_test_coordinator.py (COORDINATOR) │
|
||||
│ ├─> Checks Worker 2 availability (office hours) │
|
||||
│ ├─> Creates 2 chunks (128 combos each) │
|
||||
│ ├─> Assigns chunks to workers │
|
||||
│ └─> Monitors completion │
|
||||
│ │ │
|
||||
│ ├── Worker 1 Assignment │
|
||||
│ │ └─> SSH: python3 v11_test_worker.py │
|
||||
│ │ data/solusdt_5m.csv v11_test_chunk_0000 0 │
|
||||
│ │ │
|
||||
│ └── Worker 2 Assignment (if available) │
|
||||
│ └─> SSH via hop: python3 v11_test_worker.py │
|
||||
│ data/solusdt_5m.csv v11_test_chunk_0001 128 │
|
||||
│ │
|
||||
│ Database: exploration.db │
|
||||
│ ├── v11_test_chunks (2 chunks) │
|
||||
│ └── v11_test_strategies (256 strategies) │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
│ │
|
||||
│ SSH direct │ SSH via hop
|
||||
│ 24/7 availability │ Office hours aware
|
||||
▼ ▼
|
||||
┌─────────────────────────┐ ┌──────────────────────────────────────┐
|
||||
│ WORKER 1 PROCESSING │ │ WORKER 2 PROCESSING │
|
||||
│ (Always Available) │ │ (Office Hours Aware) │
|
||||
├─────────────────────────┤ ├──────────────────────────────────────┤
|
||||
│ │ │ │
|
||||
│ 4. v11_test_worker.py │ │ 4. v11_test_worker.py │
|
||||
│ ├─> 27-core MP │ │ ├─> 27-core MP │
|
||||
│ └─> Chunk 0000 │ │ └─> Chunk 0001 │
|
||||
│ (0-127) │ │ (128-255) │
|
||||
│ │ │ │ │ │
|
||||
│ ▼ │ │ ▼ │
|
||||
│ 5. v11 Indicator │ │ 5. v11 Indicator │
|
||||
│ (For each combo) │ │ (For each combo) │
|
||||
│ ├─> Load data │ │ ├─> Load data │
|
||||
│ ├─> Generate signals │ │ ├─> Generate signals │
|
||||
│ └─> Backtest │ │ └─> Backtest │
|
||||
│ │ │ │ │ │
|
||||
│ ▼ │ │ ▼ │
|
||||
│ Output: │ │ Output: │
|
||||
│ • CSV results │ │ • CSV results │
|
||||
│ • 128 strategies │ │ • 128 strategies │
|
||||
│ │ │ │
|
||||
│ Availability: │ │ Availability: │
|
||||
│ • 24/7 │ │ • Mon-Fri: 6PM-8AM │
|
||||
│ • No restrictions │ │ • Sat-Sun: 24/7 │
|
||||
│ │ │ • Office hours: DISABLED │
|
||||
└─────────────────────────┘ └──────────────────────────────────────┘
|
||||
│ │
|
||||
└───────────────┬───────────────────────┘
|
||||
│ Results returned
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ RESULTS AGGREGATION │
|
||||
│ Worker 1:/home/comprehensive_sweep/ │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ v11_test_results/ │
|
||||
│ ├── v11_test_chunk_0000_results.csv (128 rows) │
|
||||
│ └── v11_test_chunk_0001_results.csv (128 rows) │
|
||||
│ │
|
||||
│ exploration.db │
|
||||
│ └── v11_test_strategies (256 total strategies) │
|
||||
│ ├── params (JSON) │
|
||||
│ ├── pnl (REAL) │
|
||||
│ ├── win_rate (REAL) │
|
||||
│ ├── profit_factor (REAL) │
|
||||
│ ├── max_drawdown (REAL) │
|
||||
│ └── total_trades (INTEGER) │
|
||||
│ │
|
||||
│ Telegram Notifications: │
|
||||
│ ├── Start: Worker count, configuration │
|
||||
│ ├── Progress: Chunk completions │
|
||||
│ └── Completion: Duration, results location │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
|
||||
DATA FLOW
|
||||
=========
|
||||
|
||||
Market Data (solusdt_5m.csv)
|
||||
↓
|
||||
v11_moneyline_all_filters.py
|
||||
├─> Calculate Money Line (supertrend)
|
||||
├─> Detect trend flips
|
||||
├─> Apply ALL filters:
|
||||
│ ├─> ADX minimum (trend strength)
|
||||
│ ├─> Entry buffer (price beyond line)
|
||||
│ ├─> Price position (don't chase extremes)
|
||||
│ ├─> Volume ratio (avoid dead/overheated)
|
||||
│ └─> RSI boundaries (momentum confirmation)
|
||||
└─> Generate signals (ONLY when ALL filters pass)
|
||||
↓
|
||||
Simple Backtest Logic
|
||||
├─> For each signal:
|
||||
│ ├─> Look ahead 100 bars
|
||||
│ ├─> Check if TP1 hit (+0.86%)
|
||||
│ └─> Check if SL hit (-1.29%)
|
||||
└─> Track equity curve
|
||||
↓
|
||||
Results
|
||||
├─> Total P&L
|
||||
├─> Win rate
|
||||
├─> Profit factor
|
||||
├─> Max drawdown
|
||||
└─> Total trades
|
||||
|
||||
|
||||
PARAMETER SPACE
|
||||
===============
|
||||
|
||||
256 Combinations = 2^8 (2 values per parameter)
|
||||
|
||||
Parameters being optimized:
|
||||
1. flip_threshold [0.5, 0.6] % price movement to flip
|
||||
2. adx_min [18, 21] Minimum ADX for trend
|
||||
3. long_pos_max [75, 80] Max price position for longs
|
||||
4. short_pos_min [20, 25] Min price position for shorts
|
||||
5. vol_min [0.8, 1.0] Minimum volume ratio
|
||||
6. entry_buffer_atr [0.15, 0.20] ATR buffer beyond line
|
||||
7. rsi_long_min [35, 40] RSI minimum for longs
|
||||
8. rsi_short_max [65, 70] RSI maximum for shorts
|
||||
|
||||
Fixed parameters (not being optimized in test):
|
||||
- confirm_bars: 0 (immediate signals)
|
||||
- cooldown_bars: 3 (prevent overtrading)
|
||||
- atr_period: 12 (5-minute chart default)
|
||||
- multiplier: 3.8 (ATR band multiplier)
|
||||
- adx_length: 16
|
||||
- rsi_length: 14
|
||||
- vol_max: 3.5
|
||||
- rsi_long_max: 70
|
||||
- rsi_short_min: 30
|
||||
|
||||
|
||||
TIMING
|
||||
======
|
||||
|
||||
Expected Runtime:
|
||||
├─> Worker 1 only (weekday daytime): ~25 minutes
|
||||
└─> Both workers (nights/weekends): ~12-15 minutes
|
||||
|
||||
Optimal Start Times:
|
||||
├─> Fastest: Weekend anytime (both workers)
|
||||
├─> Good: Weekday after 6 PM (both workers)
|
||||
└─> Slowest: Weekday 8am-6pm (Worker 1 only)
|
||||
|
||||
|
||||
VERIFICATION CHECKLIST
|
||||
======================
|
||||
|
||||
After completion, verify:
|
||||
□ Coordinator log shows "V11 TEST SWEEP COMPLETE!"
|
||||
□ 2 CSV files exist in v11_test_results/
|
||||
□ Each CSV has 128 rows (256 total)
|
||||
□ Database has 256 entries in v11_test_strategies
|
||||
□ PnL values are varied (NOT all zeros)
|
||||
□ Top result shows PnL > $0 and trades > 0
|
||||
□ Telegram received completion notification
|
||||
|
||||
|
||||
CRITICAL DIFFERENCE FROM V9
|
||||
============================
|
||||
|
||||
v9 Bug (filters calculated but not applied):
|
||||
if flip_long:
|
||||
adx_ok = ...
|
||||
volume_ok = ...
|
||||
# BUT: Signal fires regardless
|
||||
signals.append(...) ❌
|
||||
|
||||
v11 Fix (ALL filters must pass):
|
||||
if flip_long:
|
||||
adx_ok = ...
|
||||
volume_ok = ...
|
||||
rsi_ok = ...
|
||||
pos_ok = ...
|
||||
entry_buffer_ok = ...
|
||||
# Signal ONLY fires when ALL pass
|
||||
if adx_ok and volume_ok and rsi_ok and pos_ok and entry_buffer_ok:
|
||||
signals.append(...) ✅
|
||||
|
||||
This is why v9 showed "no data" - broken filters allowed garbage signals.
|
||||
Reference in New Issue
Block a user