# Quality Threshold Validation Plan

## Overview
System currently blocks signals with quality score <91. First validation shows potential false negative (quality 80 signal would have profited +0.52%). Need data to determine if threshold is optimal or too restrictive.

## Discovery (Nov 22, 2025)

**User Observation:** "Green dots shot up" - TradingView Money Line signals blocked by system moved favorably

**Signal 1 Analysis (Nov 21, 16:50):**
- **Blocked:** Quality 80 (threshold 91)
- **Primary reason:** ADX 16.6 (weak trend, needs 18+)
- **Entry:** $126.20
- **Peak:** $126.86 within 1 minute
- **Profit:** +0.52% (+$43 on $8,300 position)
- **TP1 Target:** +1.51% (would NOT have hit)
- **Result:** FALSE NEGATIVE - blocked a profitable signal

**Signal 2 (Nov 21, 23:00):**
- **Blocked:** Quality 70 (threshold 91)
- **Primary reason:** RSI 25.6 oversold on SHORT
- **Status:** Data collection in progress
- **Entry price:** Bug (0 value, needs investigation)

## Data Collection Plan

### Phase 1: Collect 20-30 Quality-Blocked Signals (2-4 weeks)

**System Enhancement (Nov 22, 2025):**
- BlockedSignalTracker now tracks `QUALITY_SCORE_TOO_LOW` signals
- Price movement captured at 1min, 5min, 15min, 30min intervals
- TP1/TP2/SL hit detection using ATR-based targets
- Max favorable/adverse excursion tracked

**SQL Query for Progress:**
```sql
-- Check data collection progress
SELECT 
  COUNT(*) as total_blocked,
  SUM(CASE WHEN "analysisComplete" THEN 1 ELSE 0 END) as complete,
  ROUND(AVG("signalQualityScore")::numeric, 1) as avg_quality,
  MIN("signalQualityScore") as min_quality,
  MAX("signalQualityScore") as max_quality
FROM "BlockedSignal"
WHERE "blockReason" = 'QUALITY_SCORE_TOO_LOW'
  AND "createdAt" >= NOW() - INTERVAL '30 days';
```

### Phase 2: Analysis (After 20+ Complete Signals)

**Win Rate Comparison:**
```sql
-- Compare blocked signals vs executed trades
WITH blocked_stats AS (
  SELECT 
    COUNT(*) as total,
    SUM(CASE WHEN "wouldHitTP1" THEN 1 ELSE 0 END) as winners,
    SUM(CASE WHEN "wouldHitSL" THEN 1 ELSE 0 END) as losers,
    ROUND(AVG("maxFavorableExcursion")::numeric, 2) as avg_mfe,
    ROUND(AVG("maxAdverseExcursion")::numeric, 2) as avg_mae
  FROM "BlockedSignal"
  WHERE "blockReason" = 'QUALITY_SCORE_TOO_LOW'
    AND "analysisComplete" = true
),
executed_stats AS (
  SELECT 
    COUNT(*) as total,
    SUM(CASE WHEN "realizedPnL" > 0 THEN 1 ELSE 0 END) as winners,
    SUM(CASE WHEN "realizedPnL" < 0 THEN 1 ELSE 0 END) as losers,
    ROUND(AVG("maxFavorableExcursion")::numeric, 2) as avg_mfe,
    ROUND(AVG("maxAdverseExcursion")::numeric, 2) as avg_mae
  FROM "Trade"
  WHERE "indicatorVersion" = 'v8'
    AND "exitReason" IS NOT NULL
)
SELECT 
  'Blocked Signals' as category,
  b.total,
  b.winners,
  b.losers,
  ROUND(100.0 * b.winners / NULLIF(b.total, 0), 1) as win_rate,
  b.avg_mfe,
  b.avg_mae
FROM blocked_stats b
UNION ALL
SELECT 
  'Executed Trades',
  e.total,
  e.winners,
  e.losers,
  ROUND(100.0 * e.winners / NULLIF(e.total, 0), 1) as win_rate,
  e.avg_mfe,
  e.avg_mae
FROM executed_stats e;
```

**Quality Score Distribution:**
```sql
-- Analyze blocked signal performance by quality tier
SELECT 
  CASE 
    WHEN "signalQualityScore" >= 85 THEN '85-90 (Near Threshold)'
    WHEN "signalQualityScore" >= 80 THEN '80-84 (Marginal)'
    WHEN "signalQualityScore" >= 75 THEN '75-79 (Weak)'
    ELSE '70-74 (Very Weak)'
  END as quality_tier,
  COUNT(*) as count,
  SUM(CASE WHEN "wouldHitTP1" THEN 1 ELSE 0 END) as tp1_hits,
  SUM(CASE WHEN "wouldHitSL" THEN 1 ELSE 0 END) as sl_hits,
  ROUND(100.0 * SUM(CASE WHEN "wouldHitTP1" THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate,
  ROUND(AVG("maxFavorableExcursion")::numeric, 2) as avg_mfe
FROM "BlockedSignal"
WHERE "blockReason" = 'QUALITY_SCORE_TOO_LOW'
  AND "analysisComplete" = true
GROUP BY quality_tier
ORDER BY MIN("signalQualityScore") DESC;
```

**ADX/RSI Patterns:**
```sql
-- Check if specific metrics predict blocked signal outcomes
SELECT 
  CASE 
    WHEN adx >= 20 THEN 'Strong Trend (ADX 20+)'
    WHEN adx >= 18 THEN 'Moderate Trend (ADX 18-20)'
    ELSE 'Weak Trend (ADX <18)'
  END as trend_strength,
  COUNT(*) as count,
  SUM(CASE WHEN "wouldHitTP1" THEN 1 ELSE 0 END) as winners,
  ROUND(100.0 * SUM(CASE WHEN "wouldHitTP1" THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate
FROM "BlockedSignal"
WHERE "blockReason" = 'QUALITY_SCORE_TOO_LOW'
  AND "analysisComplete" = true
  AND adx IS NOT NULL
GROUP BY trend_strength
ORDER BY MIN(adx) DESC;
```

## Decision Framework

### Scenario 1: Blocked Signals Are Losers (<40% Win Rate)
**Action:** Keep threshold at 91
**Reasoning:** System correctly filtering low-quality setups
**Conclusion:** Current false negative (quality 80 +0.52%) was statistical outlier

### Scenario 2: Blocked Signals Are Winners (50%+ Win Rate)
**Action:** Lower threshold to 85
**Reasoning:** Too restrictive, missing profitable opportunities
**Expected Impact:** +3-5 trades/week, potential +10-15% monthly P&L improvement

### Scenario 3: Quality 80-84 Wins, 85-90 Loses
**Action:** Adjust threshold to 85, focus ADX/RSI weight optimization
**Reasoning:** Sharp cutoff exists, but threshold slightly misplaced
**Next Step:** Refine scoring weights to push marginal setups above 91

### Scenario 4: Pattern-Specific (e.g., Low ADX Wins in Strong Trends)
**Action:** Add context filters instead of lowering threshold
**Example:** Allow ADX 16-18 IF volume >1.2x AND price position mid-range
**Reasoning:** Preserve quality 91 standard but add intelligent exceptions

## Timeline

- **Week 1-2:** Collect 10-15 blocked signals
- **Week 3-4:** Collect 10-15 more blocked signals (total 20-30)
- **End of Week 4:** Run analysis queries, determine action
- **Week 5:** Implement changes if needed, document results

## Success Metrics

**Before Optimization:**
- Quality 91 threshold
- v8: 8 trades, 57.1% WR, +$262.70 total
- 1 confirmed false negative (quality 80, +0.52% missed)

**After Optimization Target:**
- Maintain/improve win rate (≥55%)
- Increase trade frequency if lowering threshold (+3-5/week)
- Reduce false negatives (<5% of blocked signals profitable)
- Data-validated threshold (not guesswork)

## Notes

- **User requirement:** "data is king" - no changes without validation
- **Conservative approach:** Collect full dataset before any threshold adjustments
- **Financial impact:** Quality 80 signal = $43 missed profit on $8,300 position
- **Risk:** Lowering threshold too much could increase losses more than gains
- **Key insight:** Perfect quality separation at ≥95 still holds (5/5 trades won)