Files
trading_bot_v4/app/api/roadmap/route.ts
mindesbunister cecdb8290c docs: Add 1-minute data enhancements roadmap
DOCUMENTATION:
- Created 1MIN_DATA_ENHANCEMENTS_ROADMAP.md (comprehensive 7-phase plan)
- Copied to docs/ folder for permanent documentation
- Updated website roadmap API with Phase 7 items

PHASE 7 FOUNDATION  COMPLETE (Nov 27, 2025):
- 1-minute data collection working (verified)
- Revenge system ADX validation deployed
- Market data cache updates every 60 seconds
- Foundation for 6 future enhancements

PLANNED ENHANCEMENTS:
1. Smart Entry Timing (0.2-0.5% better entries)
2. Signal Quality Real-Time Validation (block degraded signals)
3. Stop-Hunt Early Warning System (predictive revenge)
4. Dynamic Position Sizing (ADX momentum-based leverage)
5. Re-Entry Analytics Momentum Filters (trend strength)
6. Dynamic Trailing Stop Optimization (adaptive trail width)

EXPECTED IMPACT:
- Entry improvement: $1,600-4,000 over 100 trades
- Block 5-10% degraded signals
- Revenge success rate: +10-15%
- Runner profitability: +10-20%
- Better risk-adjusted returns across all systems

User requested: "put that on every documentation. it has to go on the websites roadmap as well"
All locations updated 
2025-11-27 10:38:48 +01:00

416 lines
17 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { NextResponse } from 'next/server'
export async function GET() {
const roadmap = [
// Phase 1: Foundation (COMPLETE)
{
phase: 'Phase 1',
title: 'Core Trading System',
status: 'complete',
description: 'Drift Protocol integration, Position Manager, TP/SL automation, database tracking',
impact: 'Fully automated trading with position monitoring and exit management',
completed: 'Nov 2024',
items: [
'Drift SDK integration with transaction confirmation',
'Position Manager with 2-second monitoring loop',
'Database persistence with Prisma ORM',
'On-chain TP/SL order placement',
'Telegram notifications for trade events',
'n8n webhook integration for TradingView signals'
]
},
{
phase: 'Phase 1',
title: 'TP2-as-Runner System',
status: 'complete',
description: 'Configurable TP1 (60% default) + Runner (40%) with trailing stop',
impact: '5x larger runner position capturing extended trend moves',
completed: 'Nov 11, 2025',
items: [
'Configurable TP1 close percentage (TAKE_PROFIT_1_SIZE_PERCENT)',
'TP2 activates trailing stop instead of closing',
'Dynamic runner size calculation (100% - TP1%)',
'ATR-based trailing stop with profit acceleration',
'Settings UI shows dynamic runner percentage'
]
},
{
phase: 'Phase 1',
title: 'ATR-Based Risk Management',
status: 'complete',
description: 'Regime-agnostic TP/SL system adapting to market volatility',
impact: 'Auto-adjusts targets: bull markets expand, bear markets tighten',
completed: 'Nov 17, 2025',
items: [
'ATR × multipliers for TP1 (2.0x), TP2 (4.0x), SL (3.0x)',
'Safety bounds prevent extreme targets',
'Asset-agnostic (SOL vs BTC different ATR)',
'Fallback to fixed % when ATR unavailable',
'TradingView integration passes ATR field'
]
},
{
phase: 'Phase 1',
title: 'ADX-Based Adaptive Systems',
status: 'complete',
description: 'Trend-strength aware runner SL and trailing stop multipliers',
impact: 'Strong trends get wider trail, weak trends protect capital',
completed: 'Nov 19, 2025',
items: [
'Runner SL: ADX <20 = breakeven, 20-25 = -0.3%, >25 = -0.55%',
'Trailing stop multipliers: ADX >30 = 1.5×, 25-30 = 1.25×, <25 = 1.0×',
'Profit acceleration: >2% profit adds 1.3× multiplier',
'Backward compatible (trades without ADX use base values)',
'Graduated approach prevents binary decisions'
]
},
// Phase 2: Quality Filtering (COMPLETE)
{
phase: 'Phase 2',
title: 'Signal Quality Scoring System',
status: 'complete',
description: 'Multi-factor signal evaluation (ATR, ADX, RSI, Volume, Price Position)',
impact: '0-100 score filtering low-quality setups, data-driven optimization',
completed: 'Nov 11, 2025',
items: [
'Timeframe-aware scoring (5min vs daily thresholds)',
'Component breakdown stored in database',
'BlockedSignal table tracks filtered signals',
'Version tracking for scoring algorithm evolution',
'Anti-chop filter (-20 points for extreme sideways)'
]
},
{
phase: 'Phase 2',
title: 'Direction-Specific Quality Thresholds',
status: 'complete',
description: 'LONG 90+, SHORT 95+ based on historical win rate analysis',
impact: 'Blocks $847 in losses while missing only $210 in wins (+$637 net)',
completed: 'Nov 23, 2025',
items: [
'Historical analysis: Quality 90-94 shorts = 28.6% WR (-$553.76)',
'Historical analysis: Quality 90-94 longs = 71.4% WR (+$44.77)',
'Direction-specific ENV variables',
'Top 2 biggest losses (-$524.97) blocked by 95+ threshold',
'3:1 loss avoidance ratio validated (Nov 24 analysis)'
]
},
{
phase: 'Phase 2',
title: 'Adaptive Leverage System',
status: 'complete',
description: 'Quality-based position sizing (95+ = 15x, 90-94 = 10x)',
impact: 'Risk-adjusted positions: borderline signals get 33% less exposure',
completed: 'Nov 24, 2025',
items: [
'Quality threshold at 95 (configurable)',
'High quality: 15x leverage, Low quality: 10x leverage',
'Early quality calculation before position sizing',
'Logging shows adaptive leverage decisions',
'ENV configuration: USE_ADAPTIVE_LEVERAGE=true'
]
},
// Phase 3: Robustness (COMPLETE)
{
phase: 'Phase 3',
title: '5-Layer Database Protection',
status: 'complete',
description: 'Persistent logging, retry logic, orphan detection, auto-recovery',
impact: 'Zero data loss guarantee - all trades tracked even on failures',
completed: 'Nov 21, 2025',
items: [
'Persistent file logger (survives container restarts)',
'Retry + verification logic (3 attempts with backoff)',
'Orphan position detection on every startup',
'Retroactive record creation for missing trades',
'Telegram alerts for recovered positions'
]
},
{
phase: 'Phase 3',
title: 'Ghost Position Protection',
status: 'complete',
description: '3-layer ghost detection preventing unmonitored positions',
impact: 'Self-healing system catches state drift within 5 minutes',
completed: 'Nov 15, 2025',
items: [
'Layer 1: Drift API validation (every 5 minutes)',
'Layer 2: Death spiral detector (20+ failed close attempts)',
'Layer 3: Monitoring loop checks (every 40 seconds)',
'Automatic ghost cleanup with P&L calculation',
'No manual intervention required'
]
},
{
phase: 'Phase 3',
title: 'Rate Limit Handling',
status: 'complete',
description: 'Exponential backoff retry logic for Solana RPC operations',
impact: '99% of transient failures auto-recover, prevents missed trades',
completed: 'Nov 14, 2025',
items: [
'Retry with exponential backoff (5s → 10s → 20s)',
'Applies to all order operations (open, close, cancel)',
'Database logging of rate limit events',
'Analytics endpoint for monitoring patterns',
'Position Manager keeps monitoring on 429 errors'
]
},
// Phase 4: Automated Price Tracking (COMPLETE)
{
phase: 'Phase 4',
title: 'Multi-Timeframe Analysis',
status: 'complete',
description: 'Parallel data collection: 5min (execute) vs 15min/1H/4H/Daily (track)',
impact: 'Determine which timeframe has best win rate for live trading',
completed: 'Nov 19, 2025',
items: [
'BlockedSignalTracker monitors non-5min signals',
'Tracks price movements at 1/5/15/30 minute intervals',
'Detects if TP1/TP2/SL would have been hit',
'Records MFE/MAE for 30-minute window',
'Target: 50+ signals per timeframe before analysis',
'✅ Currently tracking: 21 signals (19 complete)'
]
},
{
phase: 'Phase 4',
title: 'Quality Threshold Validation',
status: 'complete',
description: 'Track quality-blocked signals to validate 90/95 thresholds',
impact: 'Data-driven proof if thresholds filter winners or losers',
completed: 'Nov 22, 2025',
items: [
'BlockedSignalTracker monitors QUALITY_SCORE_TOO_LOW signals',
'Same price tracking as multi-timeframe analysis',
'Target: 20-30 blocked signals over 2-4 weeks',
'Compare win rate vs executed trades in same range',
'Adjust thresholds if blocked signals >50% win rate',
'✅ Currently tracking: 13 signals (6 complete)',
'✅ First result: Quality 80 blocked would have won +0.52%'
]
},
// Phase 5: Enhancement (PLANNED)
{
phase: 'Phase 5',
title: 'Stop Hunt Revenge System',
status: 'complete',
description: 'Auto re-entry after high-quality stop-outs when price reverses',
impact: 'Capture reversals after stop hunts on quality 85+ signals',
completed: 'Nov 20, 2025',
items: [
'4-hour revenge window monitoring',
'Quality 85+ signals eligible',
'1.0× position size (same as original)',
'One revenge per stop hunt',
'30-second price checks for active stop hunts',
'Database tracking with StopHunt table'
]
},
{
phase: 'Phase 5',
title: 'MAE/MFE Analysis Dashboard',
status: 'complete',
description: 'Visualize Maximum Adverse/Favorable Excursion for TP/SL optimization',
impact: 'Data-driven adjustment of target distances based on actual price action',
items: [
'✅ Position Manager tracks MFE/MAE every 2 seconds',
'✅ Database stores maxFavorableExcursion and maxAdverseExcursion',
'✅ Analytics dashboard displays avg MFE/MAE per indicator version',
'✅ Version comparison shows MFE/MAE trends',
'✅ Optimization API analyzes MFE vs TP1 rate',
'⏳ Future: Distribution charts for optimal target identification'
]
},
{
phase: 'Phase 5',
title: 'Market Regime Detection',
status: 'planned',
description: 'Dynamic quality threshold adjustment based on ADX trends',
impact: 'Adapt to market conditions: raise thresholds in chop, lower in trends',
items: [
'Multi-timeframe ADX analysis (1H + 4H)',
'If ADX <20 on both: raise all thresholds +5 points',
'If ADX >25 on both: lower thresholds -5 points',
'Separate filters for range-bound vs trending markets',
'Telegram alerts when regime changes detected'
]
},
// Phase 6: Scaling (FUTURE)
{
phase: 'Phase 6',
title: 'Multi-Symbol Support',
status: 'planned',
description: 'Trade SOL, ETH, BTC simultaneously with independent configs',
impact: 'Diversification reduces single-asset risk, 3× opportunity frequency',
items: [
'Per-symbol enable/disable toggles',
'Independent position sizing and leverage',
'Separate quality thresholds per symbol',
'Symbol-specific cooldown periods',
'Drift market config for each asset'
]
},
{
phase: 'Phase 6',
title: 'High Availability Setup',
status: 'in-progress',
description: 'Hot standby with Hostinger operational (Nov 25), Patroni PostgreSQL cluster pending Oracle approval',
impact: 'App-level HA: 99.9% uptime validated, zero-downtime failover working. Database HA: In progress',
items: [
'✅ OPERATIONAL: Hostinger hot standby (72.62.39.24) with PostgreSQL replica',
'✅ OPERATIONAL: DNS failover with INWX API (automatic switching in 90s)',
'✅ OPERATIONAL: Health monitoring (30-second checks, systemd service)',
'✅ VALIDATED: Live failover test (Nov 25) - 0s downtime, automatic failback',
'✅ OPERATIONAL: Primary srvdocker02 (95.216.52.28) with PostgreSQL streaming replication',
'⏳ WAITING: Oracle Cloud free tier approval for Patroni nodes',
'⏳ PLANNED: 3-node Patroni cluster (primary + 2x Oracle Cloud for true DB HA)',
'⏳ PLANNED: Automatic PostgreSQL leader election with Patroni + etcd'
]
},
// Phase 7: 1-Minute Data Enhancements (FOUNDATION COMPLETE - Nov 27, 2025)
{
phase: 'Phase 7',
title: '1-Minute Market Data Foundation',
status: 'complete',
description: 'Real-time data collection and revenge system ADX validation',
impact: 'Fresh market context every 60 seconds enables intelligent optimizations',
completed: 'Nov 27, 2025',
items: [
'✅ 1-minute data collection via TradingView alerts',
'✅ Bot filters timeframe="1" → saves to BlockedSignal',
'✅ Market data cache updates every 60 seconds',
'✅ Revenge system ADX validation (blocks if ADX < 20)',
'✅ Telegram notifications show validation results',
'✅ 2+ signals collected per minute, 0 unintended trades',
'✅ Foundation for 6 future enhancements'
]
},
{
phase: 'Phase 7',
title: 'Smart Entry Timing',
status: 'planned',
description: 'Wait 1-2 minutes after signal for optimal entry conditions',
impact: '0.2-0.5% better entry price per trade = $1,600-4,000 over 100 trades',
items: [
'Queue signals for delayed execution (up to 2 minutes)',
'Monitor for pullback (0.2-0.3% favorable move)',
'Volume confirmation (>1.5x average)',
'ADX validation (must hold or increase)',
'Execute when 2+ conditions met or timeout',
'Track improvement: signalPrice vs actualEntryPrice',
'Expected: 60-75% of trades find favorable conditions'
]
},
{
phase: 'Phase 7',
title: 'Signal Quality Real-Time Validation',
status: 'planned',
description: 'Cross-check 5-minute signals against latest 1-minute data',
impact: 'Block 5-10% of degraded signals, prevent stale entry losses',
items: [
'ADX degradation check (current < signal - 5 points)',
'Volume collapse check (current < 0.5x signal)',
'RSI reversal detection (oversold/overbought shifts)',
'Price position shift detection (chasing extremes)',
'Block or reduce position size on degradation',
'Track blocked signals that would have won/lost'
]
},
{
phase: 'Phase 7',
title: 'Stop-Hunt Early Warning System',
status: 'planned',
description: 'Predictive revenge activation before stop loss hit',
impact: 'Revenge entry 5-10s vs 90s, +10-15% success rate',
items: [
'Detect price within 0.3% of SL on quality 85+ trades',
'Pre-validate ADX conditions before stop-out',
'Immediate revenge execution on SL (no 90s delay)',
'Disable revenge if ADX weak before stop',
'Avoid whipsaw losses from weak trend stops',
'Better timing = better entry price'
]
},
{
phase: 'Phase 7',
title: 'Dynamic Position Sizing',
status: 'planned',
description: 'Adjust leverage ±20% based on ADX momentum',
impact: 'Larger positions in strengthening trends, smaller in weakening',
items: [
'Track ADX over last 3 minutes (trend momentum)',
'ADX rising >10%: boost leverage by 1.2× (10x → 12x)',
'ADX falling >10%: reduce leverage by 0.8× (10x → 8x)',
'Safety limits: ±20% max adjustment, 5-20x range',
'Better risk-adjusted returns',
'Smoother equity curve'
]
},
{
phase: 'Phase 7',
title: 'Re-Entry Analytics Momentum Filters',
status: 'planned',
description: 'Add trend momentum to manual trade validation',
impact: '+5-10% win rate on manual re-entries',
items: [
'Analyze ADX trend over last 3 minutes',
'RSI momentum toward/away from extremes',
'Bonus points for strengthening trends',
'Penalty points for weakening trends',
'Block re-entries into deteriorating conditions',
'Favor momentum confirmation'
]
},
{
phase: 'Phase 7',
title: 'Dynamic Trailing Stop Optimization',
status: 'planned',
description: 'Adjust trail width based on real-time ADX changes',
impact: '+10-20% runner profitability from adaptive trailing',
items: [
'Monitor ADX changes during trade lifetime',
'ADX spiking >5 points: widen trail (1.5× → 1.8×)',
'ADX dropping >5 points: tighten trail (1.5× → 1.2×)',
'Capture more profit in accelerating trends',
'Lock profit when trend weakens',
'Reduce premature stops in strong moves'
]
},
// Phase 8: Intelligence (DISTANT FUTURE)
{
phase: 'Phase 8',
title: 'ML-Based Quality Scoring',
status: 'planned',
description: 'Machine learning model optimizes component weights based on outcomes',
impact: 'Automatic adaptation to changing market conditions',
items: [
'Collect 500+ trades with outcomes for training data',
'Feature engineering: ADX, ATR, RSI, Volume, Price Position',
'Train gradient boosting model (XGBoost/LightGBM)',
'Predict win probability instead of fixed score',
'Retrain monthly on new data',
'A/B test ML scores vs rule-based scores'
]
},
]
return NextResponse.json({
success: true,
roadmap,
stats: {
total: roadmap.length,
complete: roadmap.filter(item => item.status === 'complete').length,
inProgress: roadmap.filter(item => item.status === 'in-progress').length,
planned: roadmap.filter(item => item.status === 'planned').length,
}
})
}