trading_bot_v4/SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md

# Signal Quality Optimization Roadmap

**Goal:** Optimize signal quality thresholds and scoring logic using data-driven analysis

**Current Status:** Phase 1 - Data Collection (Active)  
**Last Updated:** November 11, 2025

---

## Overview

This roadmap guides the systematic improvement of signal quality filtering. We follow a **data-first approach**: collect evidence, analyze patterns, then make changes. No premature optimization.

### Current System
- **Quality Score Threshold:** 65 points (recently raised from 60)
- **Executed Trades:** 157 total (155 closed, 2 open)
- **Performance:** +$3.43 total P&L, 44.5% win rate
- **Score Distribution:**
  - 80-100 (Excellent): 49 trades, +$46.48, 46.9% WR
  - 70-79 (Good): 15 trades, -$2.20, 40.0% WR ⚠️
  - 65-69 (Pass): 13 trades, +$28.28, 53.8% WR ✅
  - 60-64 (Just Below): 2 trades, +$45.78, **100% WR** 🔥
  - 0-49 (Very Weak): 13 trades, -$127.89, 30.8% WR 💀

---

## Phase 1: Data Collection (CURRENT) ✅ IN PROGRESS

**Status:** Infrastructure complete, collecting data  
**Started:** November 11, 2025  
**Target:** Collect 10-20 blocked signals (1-2 weeks)

### Completed (Nov 11, 2025)
- [x] Created `BlockedSignal` database table
- [x] Implemented automatic saving in check-risk endpoint
- [x] Deployed to production (trading-bot-v4 container)
- [x] Created tracking documentation (BLOCKED_SIGNALS_TRACKING.md)

### What's Being Tracked
Every blocked signal captures:
- **Metrics:** ATR, ADX, RSI, volume ratio, price position, timeframe
- **Score:** Quality score (0-100), version, detailed breakdown
- **Block Reason:** Quality score, cooldown, hourly limit, daily drawdown
- **Context:** Symbol, direction, price at signal time, timestamp

### What We're Looking For
1. How many signals score 60-64 (just below threshold)?
2. What are their characteristics (ADX, ATR, price position)?
3. Are there patterns (extreme positions, specific timeframes)?
4. Do they cluster around specific block reasons?

### Phase 1 Completion Criteria
- [ ] Minimum 10 blocked signals with quality scores 55-64
- [ ] At least 2 signals in 60-64 range (close calls)
- [ ] Mix of block reasons (not all quality score)
- [ ] Data spans multiple market conditions (trending, choppy, volatile)

### SQL Queries for Phase 1
```sql
-- Check progress
SELECT COUNT(*) as total_blocked 
FROM "BlockedSignal";

-- Score distribution
SELECT 
  CASE 
    WHEN signalQualityScore >= 60 THEN '60-64 (Close)'
    WHEN signalQualityScore >= 55 THEN '55-59 (Marginal)'
    WHEN signalQualityScore >= 50 THEN '50-54 (Weak)'
    ELSE '0-49 (Very Weak)'
  END as tier,
  COUNT(*) as count
FROM "BlockedSignal"
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
GROUP BY tier
ORDER BY MIN(signalQualityScore) DESC;
```

---

## Phase 2: Pattern Analysis 🔜 NEXT

**Prerequisites:** 10-20 blocked signals collected  
**Estimated Duration:** 2-3 days  
**Owner:** Manual analysis + SQL queries

### Analysis Tasks

#### 2.1: Score Distribution Analysis
```sql
-- Analyze blocked signals by score range
SELECT 
  CASE 
    WHEN signalQualityScore >= 60 THEN '60-64'
    WHEN signalQualityScore >= 55 THEN '55-59'
    ELSE '50-54'
  END as score_range,
  COUNT(*) as count,
  ROUND(AVG(atr)::numeric, 2) as avg_atr,
  ROUND(AVG(adx)::numeric, 1) as avg_adx,
  ROUND(AVG(pricePosition)::numeric, 1) as avg_price_pos,
  ROUND(AVG(volumeRatio)::numeric, 2) as avg_volume
FROM "BlockedSignal"
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
GROUP BY score_range
ORDER BY MIN(signalQualityScore) DESC;
```

#### 2.2: Compare with Executed Trades
```sql
-- Find executed trades with similar scores to blocked signals
SELECT 
  'Executed' as type,
  signalQualityScore,
  COUNT(*) as trades,
  ROUND(AVG(realizedPnL)::numeric, 2) as avg_pnl,
  ROUND(100.0 * SUM(CASE WHEN realizedPnL > 0 THEN 1 ELSE 0 END) / COUNT(*)::numeric, 1) as win_rate
FROM "Trade"
WHERE exitReason IS NOT NULL
  AND signalQualityScore BETWEEN 60 AND 69
GROUP BY signalQualityScore
ORDER BY signalQualityScore;
```

#### 2.3: ADX Pattern Analysis
Key finding from existing data: ADX 20-25 is a trap zone!
```sql
-- ADX distribution in blocked signals
SELECT 
  CASE 
    WHEN adx >= 25 THEN 'Strong (25+)'
    WHEN adx >= 20 THEN 'Moderate (20-25)'
    WHEN adx >= 15 THEN 'Weak (15-20)'
    ELSE 'Very Weak (<15)'
  END as adx_tier,
  COUNT(*) as count,
  ROUND(AVG(signalQualityScore)::numeric, 1) as avg_score
FROM "BlockedSignal"
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
  AND adx IS NOT NULL
GROUP BY adx_tier
ORDER BY MIN(adx) DESC;
```

#### 2.4: Extreme Position Analysis
Test hypothesis: Extremes (<10% or >90%) need different thresholds
```sql
-- Blocked signals at range extremes
SELECT 
  direction,
  signalQualityScore,
  ROUND(pricePosition::numeric, 1) as pos,
  ROUND(adx::numeric, 1) as adx,
  ROUND(volumeRatio::numeric, 2) as vol
FROM "BlockedSignal"
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
  AND (pricePosition < 10 OR pricePosition > 90)
ORDER BY signalQualityScore DESC;
```

### Phase 2 Deliverables
- [ ] Score distribution report
- [ ] ADX pattern analysis
- [ ] Extreme position analysis
- [ ] Comparison with executed trades
- [ ] **DECISION:** Keep threshold at 65, lower to 60, or implement dual-threshold system

---

## Phase 3: Implementation (Conditional) 🎯 FUTURE

**Trigger:** Analysis shows clear pattern worth exploiting  
**Prerequisites:** Phase 2 complete + statistical significance (15+ blocked signals)

### Option A: Dual-Threshold System (Recommended)
**IF** data shows extreme positions (price <10% or >90%) with scores 60-64 are profitable:

**Implementation:**
```typescript
// In check-risk endpoint
const isExtremePosition = pricePosition < 10 || pricePosition > 90
const requiredScore = isExtremePosition ? 60 : 65

if (qualityScore.score < requiredScore) {
  // Block signal
}
```

**Changes Required:**
- `app/api/trading/check-risk/route.ts` - Add dual threshold logic
- `lib/trading/signal-quality.ts` - Add `isExtremePosition` helper
- `config/trading.ts` - Add `minScoreForExtremes` config option
- Update AI instructions with new logic

### Option B: ADX-Based Gates (Alternative)
**IF** data shows strong ADX trends (25+) with lower scores are profitable:

**Implementation:**
```typescript
const requiredScore = adx >= 25 ? 60 : 65
```

**Changes Required:**
- Similar to Option A but based on ADX threshold

### Option C: Keep Current (If No Clear Pattern)
**IF** data shows no consistent profit opportunity in blocked signals:
- No changes needed
- Continue monitoring
- Revisit in 20 more trades

### Phase 3 Checklist
- [ ] Decision made based on Phase 2 analysis
- [ ] Code changes implemented
- [ ] Updated signalQualityVersion to 'v5' in database
- [ ] AI instructions updated
- [ ] Tested with historical blocked signals
- [ ] Deployed to production
- [ ] Monitoring for 10 trades to validate improvement

---

## Phase 4: Price Analysis Automation 🤖 FUTURE

**Goal:** Automatically track if blocked signals would have been profitable  
**Complexity:** Medium - requires price monitoring job  
**Prerequisites:** Phase 3 complete OR 50+ blocked signals collected

### Architecture
```
Monitoring Job (runs every 30 min)
  ↓
  Fetch BlockedSignal records where:
    - analysisComplete = false
    - createdAt > 30 minutes ago
  ↓
  For each signal:
    - Get price history from Pyth/Drift
    - Calculate if TP1/TP2/SL would have been hit
    - Update priceAfter1Min/5Min/15Min/30Min
    - Set wouldHitTP1/TP2/SL flags
    - Mark analysisComplete = true
  ↓
  Save results back to database
```

### Implementation Tasks
- [ ] Create price history fetching service
- [ ] Implement TP/SL hit calculation logic
- [ ] Create cron job or Next.js API route with scheduler
- [ ] Add monitoring dashboard for blocked signal outcomes
- [ ] Generate weekly reports on missed opportunities

### Success Metrics
- X% of blocked signals would have hit SL (blocks were correct)
- Y% would have hit TP1/TP2 (missed opportunities)
- Overall P&L of hypothetical blocked trades

---

## Phase 5: ML-Based Optimization 🧠 DISTANT FUTURE

**Goal:** Use machine learning to optimize scoring weights  
**Prerequisites:** 200+ trades with quality scores, 100+ blocked signals  
**Complexity:** High

### Approach
1. Extract features: ATR, ADX, RSI, volume, price position, timeframe
2. Train model on: executed trades (outcome = P&L)
3. Validate on: blocked signals (if price analysis complete)
4. Generate: Optimal scoring weights for each feature
5. Implement: Dynamic threshold adjustment based on market conditions

### Not Implemented Yet
This is a future consideration only. Current data-driven approach is sufficient.

---

## Key Principles

### 1. Data Before Action
- Minimum 10 samples before any decision
- Prefer 20+ for statistical confidence
- No changes based on 1-2 outliers

### 2. Incremental Changes
- Change one variable at a time
- Test for 10-20 trades after each change
- Revert if performance degrades

### 3. Version Tracking
- Every scoring logic change gets new version (v4 → v5)
- Store version with each trade/blocked signal
- Enables A/B testing and rollback

### 4. Document Everything
- Update this roadmap after each phase
- Record decisions and rationale
- Link to SQL queries and analysis

---

## Progress Tracking

### Milestones
- [x] Nov 11, 2025: Phase 1 infrastructure complete
- [ ] Target: ~Nov 20-25, 2025: Phase 1 complete (10-20 blocked signals)
- [ ] Target: ~Nov 25-30, 2025: Phase 2 analysis complete
- [ ] TBD: Phase 3 implementation (conditional)

### Metrics to Watch
- **Blocked signals collected:** 0/10 minimum
- **Close calls (60-64 score):** 0/2 minimum
- **Days of data collection:** 0/7 minimum
- **Market conditions covered:** 0/3 (trending, choppy, volatile)

### Review Schedule
- **Weekly:** Check blocked signal count
- **After 10 blocked:** Run Phase 2 analysis
- **After Phase 2:** Decide on Phase 3 implementation
- **Monthly:** Review overall system performance

---

## Questions to Answer

### Phase 1 Questions
- [ ] How many signals get blocked per day?
- [ ] What's the score distribution of blocked signals?
- [ ] Are most blocks from quality score or other reasons?

### Phase 2 Questions
- [ ] Do blocked signals at 60-64 have common characteristics?
- [ ] Would lowering threshold to 60 improve performance?
- [ ] Do extreme positions need different treatment?
- [ ] Is ADX pattern valid in blocked signals?

### Phase 3 Questions
- [ ] Did the change improve win rate?
- [ ] Did it increase profitability?
- [ ] Any unintended side effects?

---

## Appendix: Historical Context

### Why This Roadmap Exists
**Date:** November 11, 2025

**Situation:** Three TradingView signals fired:
1. SHORT at 05:15 - Executed (score likely 65+) → Losing trade
2. LONG at 05:20 - Executed (score likely 65+) → Losing trade
3. SHORT at 05:30 - **BLOCKED** (score 45) → Would have been profitable

**User Question:** "What can we do about this?"

**Analysis Findings:**
- Only 2 historical trades scored 60-64 (both winners +$45.78)
- Sample size too small for confident decision
- ADX 20-25 is a trap zone (-$23.41 in 23 trades)
- Low volume (<0.8x) outperforms high volume (counterintuitive!)

**Decision:** Build data collection system instead of changing thresholds prematurely

**This Roadmap:** Systematic approach to optimization with proper data backing

---

**Remember:** The goal isn't to catch every winning trade. The goal is to optimize the **risk-adjusted return** by catching more winners than losers at each threshold level. Sometimes blocking a potential winner is correct if it also blocks 3 losers.