trading_bot_v4/SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md

Signal Quality Optimization Roadmap

Goal: Optimize signal quality thresholds and scoring logic using data-driven analysis

Current Status: Phase 1 - Data Collection (Active)
Last Updated: November 11, 2025

---

Overview

This roadmap guides the systematic improvement of signal quality filtering. We follow a data-first approach: collect evidence, analyze patterns, then make changes. No premature optimization.

Current System

• Quality Score Threshold: 65 points (recently raised from 60)
• Executed Trades: 157 total (155 closed, 2 open)
• Performance: +$3.43 total P&L, 44.5% win rate
• Score Distribution:
  • 80-100 (Excellent): 49 trades, +$46.48, 46.9% WR
  • 70-79 (Good): 15 trades, -$2.20, 40.0% WR ⚠️
  • 65-69 (Pass): 13 trades, +$28.28, 53.8% WR ✅
  • 60-64 (Just Below): 2 trades, +$45.78, 100% WR 🔥
  • 0-49 (Very Weak): 13 trades, -$127.89, 30.8% WR 💀

---

Phase 1: Data Collection (CURRENT) ✅ IN PROGRESS

Status: Infrastructure complete, collecting data
Started: November 11, 2025
Target: Collect 10-20 blocked signals (1-2 weeks)

Completed (Nov 11, 2025)

• Created BlockedSignal database table
• Implemented automatic saving in check-risk endpoint
• Deployed to production (trading-bot-v4 container)
• Created tracking documentation (BLOCKED_SIGNALS_TRACKING.md)

What's Being Tracked

Every blocked signal captures:

• Metrics: ATR, ADX, RSI, volume ratio, price position, timeframe
• Score: Quality score (0-100), version, detailed breakdown
• Block Reason: Quality score, cooldown, hourly limit, daily drawdown
• Context: Symbol, direction, price at signal time, timestamp

What We're Looking For

1. How many signals score 60-64 (just below threshold)?
2. What are their characteristics (ADX, ATR, price position)?
3. Are there patterns (extreme positions, specific timeframes)?
4. Do they cluster around specific block reasons?

Phase 1 Completion Criteria

• Minimum 10 blocked signals with quality scores 55-64
• At least 2 signals in 60-64 range (close calls)
• Mix of block reasons (not all quality score)
• Data spans multiple market conditions (trending, choppy, volatile)

SQL Queries for Phase 1

-- Check progress
SELECT COUNT(*) as total_blocked 
FROM "BlockedSignal";

-- Score distribution
SELECT 
  CASE 
    WHEN signalQualityScore >= 60 THEN '60-64 (Close)'
    WHEN signalQualityScore >= 55 THEN '55-59 (Marginal)'
    WHEN signalQualityScore >= 50 THEN '50-54 (Weak)'
    ELSE '0-49 (Very Weak)'
  END as tier,
  COUNT(*) as count
FROM "BlockedSignal"
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
GROUP BY tier
ORDER BY MIN(signalQualityScore) DESC;

---

Phase 2: Pattern Analysis 🔜 NEXT

Prerequisites: 10-20 blocked signals collected
Estimated Duration: 2-3 days
Owner: Manual analysis + SQL queries

Analysis Tasks

2.1: Score Distribution Analysis

-- Analyze blocked signals by score range
SELECT 
  CASE 
    WHEN signalQualityScore >= 60 THEN '60-64'
    WHEN signalQualityScore >= 55 THEN '55-59'
    ELSE '50-54'
  END as score_range,
  COUNT(*) as count,
  ROUND(AVG(atr)::numeric, 2) as avg_atr,
  ROUND(AVG(adx)::numeric, 1) as avg_adx,
  ROUND(AVG(pricePosition)::numeric, 1) as avg_price_pos,
  ROUND(AVG(volumeRatio)::numeric, 2) as avg_volume
FROM "BlockedSignal"
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
GROUP BY score_range
ORDER BY MIN(signalQualityScore) DESC;

2.2: Compare with Executed Trades

-- Find executed trades with similar scores to blocked signals
SELECT 
  'Executed' as type,
  signalQualityScore,
  COUNT(*) as trades,
  ROUND(AVG(realizedPnL)::numeric, 2) as avg_pnl,
  ROUND(100.0 * SUM(CASE WHEN realizedPnL > 0 THEN 1 ELSE 0 END) / COUNT(*)::numeric, 1) as win_rate
FROM "Trade"
WHERE exitReason IS NOT NULL
  AND signalQualityScore BETWEEN 60 AND 69
GROUP BY signalQualityScore
ORDER BY signalQualityScore;

2.3: ADX Pattern Analysis

Key finding from existing data: ADX 20-25 is a trap zone!

-- ADX distribution in blocked signals
SELECT 
  CASE 
    WHEN adx >= 25 THEN 'Strong (25+)'
    WHEN adx >= 20 THEN 'Moderate (20-25)'
    WHEN adx >= 15 THEN 'Weak (15-20)'
    ELSE 'Very Weak (<15)'
  END as adx_tier,
  COUNT(*) as count,
  ROUND(AVG(signalQualityScore)::numeric, 1) as avg_score
FROM "BlockedSignal"
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
  AND adx IS NOT NULL
GROUP BY adx_tier
ORDER BY MIN(adx) DESC;

2.4: Extreme Position Analysis

Test hypothesis: Extremes (<10% or >90%) need different thresholds

-- Blocked signals at range extremes
SELECT 
  direction,
  signalQualityScore,
  ROUND(pricePosition::numeric, 1) as pos,
  ROUND(adx::numeric, 1) as adx,
  ROUND(volumeRatio::numeric, 2) as vol
FROM "BlockedSignal"
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
  AND (pricePosition < 10 OR pricePosition > 90)
ORDER BY signalQualityScore DESC;

Phase 2 Deliverables

• Score distribution report
• ADX pattern analysis
• Extreme position analysis
• Comparison with executed trades
• DECISION: Keep threshold at 65, lower to 60, or implement dual-threshold system

---

Phase 3: Implementation (Conditional) 🎯 FUTURE

Trigger: Analysis shows clear pattern worth exploiting
Prerequisites: Phase 2 complete + statistical significance (15+ blocked signals)

Option A: Dual-Threshold System (Recommended)

IF data shows extreme positions (price <10% or >90%) with scores 60-64 are profitable:

Implementation:

// In check-risk endpoint
const isExtremePosition = pricePosition < 10 || pricePosition > 90
const requiredScore = isExtremePosition ? 60 : 65

if (qualityScore.score < requiredScore) {
  // Block signal
}

Changes Required:

• app/api/trading/check-risk/route.ts - Add dual threshold logic
• lib/trading/signal-quality.ts - Add isExtremePosition helper
• config/trading.ts - Add minScoreForExtremes config option
• Update AI instructions with new logic

Option B: ADX-Based Gates (Alternative)

IF data shows strong ADX trends (25+) with lower scores are profitable:

Implementation:

const requiredScore = adx >= 25 ? 60 : 65

Changes Required:

• Similar to Option A but based on ADX threshold

Option C: Keep Current (If No Clear Pattern)

IF data shows no consistent profit opportunity in blocked signals:

• No changes needed
• Continue monitoring
• Revisit in 20 more trades

Phase 3 Checklist

• Decision made based on Phase 2 analysis
• Code changes implemented
• Updated signalQualityVersion to 'v5' in database
• AI instructions updated
• Tested with historical blocked signals
• Deployed to production
• Monitoring for 10 trades to validate improvement

---

Phase 4: Price Analysis Automation 🤖 FUTURE

Goal: Automatically track if blocked signals would have been profitable
Complexity: Medium - requires price monitoring job
Prerequisites: Phase 3 complete OR 50+ blocked signals collected

Architecture

Monitoring Job (runs every 30 min)
  ↓
  Fetch BlockedSignal records where:
    - analysisComplete = false
    - createdAt > 30 minutes ago
  ↓
  For each signal:
    - Get price history from Pyth/Drift
    - Calculate if TP1/TP2/SL would have been hit
    - Update priceAfter1Min/5Min/15Min/30Min
    - Set wouldHitTP1/TP2/SL flags
    - Mark analysisComplete = true
  ↓
  Save results back to database

Implementation Tasks

• Create price history fetching service
• Implement TP/SL hit calculation logic
• Create cron job or Next.js API route with scheduler
• Add monitoring dashboard for blocked signal outcomes
• Generate weekly reports on missed opportunities

Success Metrics

• X% of blocked signals would have hit SL (blocks were correct)
• Y% would have hit TP1/TP2 (missed opportunities)
• Overall P&L of hypothetical blocked trades

---

Phase 5: ML-Based Optimization 🧠 DISTANT FUTURE

Goal: Use machine learning to optimize scoring weights
Prerequisites: 200+ trades with quality scores, 100+ blocked signals
Complexity: High

Approach

1. Extract features: ATR, ADX, RSI, volume, price position, timeframe
2. Train model on: executed trades (outcome = P&L)
3. Validate on: blocked signals (if price analysis complete)
4. Generate: Optimal scoring weights for each feature
5. Implement: Dynamic threshold adjustment based on market conditions

Not Implemented Yet

This is a future consideration only. Current data-driven approach is sufficient.

---

Key Principles

1. Data Before Action

• Minimum 10 samples before any decision
• Prefer 20+ for statistical confidence
• No changes based on 1-2 outliers

2. Incremental Changes

• Change one variable at a time
• Test for 10-20 trades after each change
• Revert if performance degrades

3. Version Tracking

• Every scoring logic change gets new version (v4 → v5)
• Store version with each trade/blocked signal
• Enables A/B testing and rollback

4. Document Everything

• Update this roadmap after each phase
• Record decisions and rationale
• Link to SQL queries and analysis

---

Progress Tracking

Milestones

• Nov 11, 2025: Phase 1 infrastructure complete
• Target: ~Nov 20-25, 2025: Phase 1 complete (10-20 blocked signals)
• Target: ~Nov 25-30, 2025: Phase 2 analysis complete
• TBD: Phase 3 implementation (conditional)

Metrics to Watch

• Blocked signals collected: 0/10 minimum
• Close calls (60-64 score): 0/2 minimum
• Days of data collection: 0/7 minimum
• Market conditions covered: 0/3 (trending, choppy, volatile)

Review Schedule

• Weekly: Check blocked signal count
• After 10 blocked: Run Phase 2 analysis
• After Phase 2: Decide on Phase 3 implementation
• Monthly: Review overall system performance

---

Questions to Answer

Phase 1 Questions

• How many signals get blocked per day?
• What's the score distribution of blocked signals?
• Are most blocks from quality score or other reasons?

Phase 2 Questions

• Do blocked signals at 60-64 have common characteristics?
• Would lowering threshold to 60 improve performance?
• Do extreme positions need different treatment?
• Is ADX pattern valid in blocked signals?

Phase 3 Questions

• Did the change improve win rate?
• Did it increase profitability?
• Any unintended side effects?

---

Appendix: Historical Context

Why This Roadmap Exists

Date: November 11, 2025

Situation: Three TradingView signals fired:

1. SHORT at 05:15 - Executed (score likely 65+) → Losing trade
2. LONG at 05:20 - Executed (score likely 65+) → Losing trade
3. SHORT at 05:30 - BLOCKED (score 45) → Would have been profitable

User Question: "What can we do about this?"

Analysis Findings:

• Only 2 historical trades scored 60-64 (both winners +$45.78)
• Sample size too small for confident decision
• ADX 20-25 is a trap zone (-$23.41 in 23 trades)
• Low volume (<0.8x) outperforms high volume (counterintuitive!)

Decision: Build data collection system instead of changing thresholds prematurely

This Roadmap: Systematic approach to optimization with proper data backing

---

Remember: The goal isn't to catch every winning trade. The goal is to optimize the risk-adjusted return by catching more winners than losers at each threshold level. Sometimes blocking a potential winner is correct if it also blocks 3 losers.