feat: implement blocked signals tracking system
- Add BlockedSignal table with 25 fields for comprehensive signal analysis - Track all blocked signals with metrics (ATR, ADX, RSI, volume, price position) - Store quality scores, block reasons, and detailed breakdowns - Include future fields for automated price analysis (priceAfter1/5/15/30Min) - Restore signalQualityVersion field to Trade table Database changes: - New table: BlockedSignal with indexes on symbol, createdAt, score, blockReason - Fixed schema drift from manual changes API changes: - Modified check-risk endpoint to save blocked signals automatically - Fixed hasContextMetrics variable scope (moved to line 209) - Save blocks for: quality score too low, cooldown period, hourly limit - Use config.minSignalQualityScore instead of hardcoded 60 Database helpers: - Added createBlockedSignal() function with try/catch safety - Added getRecentBlockedSignals(limit) for queries - Added getBlockedSignalsForAnalysis(olderThanMinutes) for automation Documentation: - Created BLOCKED_SIGNALS_TRACKING.md with SQL queries and analysis workflow - Created SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md with 5-phase plan - Documented data-first approach: collect 10-20 signals before optimization Rationale: Only 2 historical trades scored 60-64 (insufficient sample size for threshold decision). Building data collection infrastructure before making premature optimizations. Phase 1 (current): Collect blocked signals for 1-2 weeks Phase 2 (next): Analyze patterns and make data-driven threshold decision Phase 3-5 (future): Automation and ML optimization
This commit is contained in:
mindesbunister
214
BLOCKED_SIGNALS_TRACKING.md
Normal file
214
BLOCKED_SIGNALS_TRACKING.md
Normal file
@@ -0,0 +1,214 @@
|
||||
# Blocked Signals Tracking System
|
||||
|
||||
**Date Implemented:** November 11, 2025
|
||||
**Status:** ✅ ACTIVE
|
||||
|
||||
## Overview
|
||||
|
||||
Automatically tracks all signals that get blocked by the trading bot's risk checks. This data allows us to analyze whether blocked signals would have been profitable, helping optimize the signal quality thresholds over time.
|
||||
|
||||
## What Gets Tracked
|
||||
|
||||
Every time a signal is blocked, the system saves:
|
||||
|
||||
### Signal Metrics
|
||||
- Symbol (e.g., SOL-PERP)
|
||||
- Direction (long/short)
|
||||
- Timeframe (5min, 15min, 1H, etc.)
|
||||
- Price at signal time
|
||||
- ATR, ADX, RSI, volume ratio, price position
|
||||
|
||||
### Quality Score
|
||||
- Calculated score (0-100)
|
||||
- Score version (v4 = current)
|
||||
- Detailed breakdown of scoring reasons
|
||||
- Minimum score required (currently 65)
|
||||
|
||||
### Block Reason
|
||||
- `QUALITY_SCORE_TOO_LOW` - Score below threshold
|
||||
- `COOLDOWN_PERIOD` - Too soon after last trade
|
||||
- `HOURLY_TRADE_LIMIT` - Too many trades in last hour
|
||||
- `DAILY_DRAWDOWN_LIMIT` - Max daily loss reached
|
||||
|
||||
### Future Analysis Fields (NOT YET IMPLEMENTED)
|
||||
- `priceAfter1Min`, `priceAfter5Min`, `priceAfter15Min`, `priceAfter30Min`
|
||||
- `wouldHitTP1`, `wouldHitTP2`, `wouldHitSL`
|
||||
- `analysisComplete`
|
||||
|
||||
These will be filled by a monitoring job that tracks what happened after each blocked signal.
|
||||
|
||||
## Database Table
|
||||
|
||||
```sql
|
||||
Table: BlockedSignal
|
||||
- id (PK)
|
||||
- createdAt (timestamp)
|
||||
- symbol, direction, timeframe
|
||||
- signalPrice, atr, adx, rsi, volumeRatio, pricePosition
|
||||
- signalQualityScore, signalQualityVersion, scoreBreakdown (JSON)
|
||||
- minScoreRequired, blockReason, blockDetails
|
||||
- priceAfter1Min/5Min/15Min/30Min (for future analysis)
|
||||
- wouldHitTP1/TP2/SL, analysisComplete
|
||||
```
|
||||
|
||||
## Query Examples
|
||||
|
||||
### Recent Blocked Signals
|
||||
```sql
|
||||
SELECT
|
||||
symbol,
|
||||
direction,
|
||||
signalQualityScore as score,
|
||||
minScoreRequired as threshold,
|
||||
blockReason,
|
||||
createdAt
|
||||
FROM "BlockedSignal"
|
||||
ORDER BY createdAt DESC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
### Blocked by Quality Score (60-64 range)
|
||||
```sql
|
||||
SELECT
|
||||
symbol,
|
||||
direction,
|
||||
signalQualityScore,
|
||||
ROUND(atr::numeric, 2) as atr,
|
||||
ROUND(adx::numeric, 1) as adx,
|
||||
ROUND(rsi::numeric, 1) as rsi,
|
||||
ROUND(pricePosition::numeric, 1) as pos,
|
||||
blockDetails
|
||||
FROM "BlockedSignal"
|
||||
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
|
||||
AND signalQualityScore >= 60
|
||||
AND signalQualityScore < 65
|
||||
ORDER BY createdAt DESC;
|
||||
```
|
||||
|
||||
### Breakdown by Block Reason
|
||||
```sql
|
||||
SELECT
|
||||
blockReason,
|
||||
COUNT(*) as count,
|
||||
ROUND(AVG(signalQualityScore)::numeric, 1) as avg_score,
|
||||
MIN(signalQualityScore) as min_score,
|
||||
MAX(signalQualityScore) as max_score
|
||||
FROM "BlockedSignal"
|
||||
GROUP BY blockReason
|
||||
ORDER BY count DESC;
|
||||
```
|
||||
|
||||
### Today's Blocked Signals
|
||||
```sql
|
||||
SELECT
|
||||
TO_CHAR(createdAt, 'HH24:MI:SS') as time,
|
||||
symbol,
|
||||
direction,
|
||||
signalQualityScore,
|
||||
blockReason
|
||||
FROM "BlockedSignal"
|
||||
WHERE createdAt >= CURRENT_DATE
|
||||
ORDER BY createdAt DESC;
|
||||
```
|
||||
|
||||
## Analysis Workflow
|
||||
|
||||
### Step 1: Collect Data (Current Phase)
|
||||
- Bot automatically saves blocked signals
|
||||
- Wait for 10-20 blocked signals to accumulate
|
||||
- No action needed - runs automatically
|
||||
|
||||
### Step 2: Manual Analysis (When Ready)
|
||||
```sql
|
||||
-- Check how many blocked signals we have
|
||||
SELECT COUNT(*) FROM "BlockedSignal";
|
||||
|
||||
-- Analyze score distribution
|
||||
SELECT
|
||||
CASE
|
||||
WHEN signalQualityScore >= 60 THEN '60-64 (Close Call)'
|
||||
WHEN signalQualityScore >= 55 THEN '55-59 (Marginal)'
|
||||
WHEN signalQualityScore >= 50 THEN '50-54 (Weak)'
|
||||
ELSE '0-49 (Very Weak)'
|
||||
END as score_tier,
|
||||
COUNT(*) as count,
|
||||
ROUND(AVG(atr)::numeric, 2) as avg_atr,
|
||||
ROUND(AVG(adx)::numeric, 1) as avg_adx
|
||||
FROM "BlockedSignal"
|
||||
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
|
||||
GROUP BY score_tier
|
||||
ORDER BY MIN(signalQualityScore) DESC;
|
||||
```
|
||||
|
||||
### Step 3: Future Automation (Not Yet Built)
|
||||
Create a monitoring job that:
|
||||
1. Fetches `BlockedSignal` records where `analysisComplete = false` and `createdAt` > 30min ago
|
||||
2. Gets price history for those timestamps
|
||||
3. Calculates if TP1/TP2/SL would have been hit
|
||||
4. Updates the record with analysis results
|
||||
5. Sets `analysisComplete = true`
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Code Files Modified
|
||||
1. `prisma/schema.prisma` - Added `BlockedSignal` model
|
||||
2. `lib/database/trades.ts` - Added `createBlockedSignal()` function
|
||||
3. `app/api/trading/check-risk/route.ts` - Saves blocked signals
|
||||
|
||||
### Where Blocking Happens
|
||||
- Quality score check (line ~311-350)
|
||||
- Cooldown period check (line ~281-303)
|
||||
- Hourly trade limit (line ~235-258)
|
||||
- Daily drawdown limit (line ~211-223)
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Phase 1: Data Collection (CURRENT)
|
||||
- ✅ Database table created
|
||||
- ✅ Automatic saving implemented
|
||||
- ✅ Bot deployed and running
|
||||
- ⏳ Collect 10-20 blocked signals (wait ~1-2 weeks)
|
||||
|
||||
### Phase 2: Analysis
|
||||
- Query blocked signal patterns
|
||||
- Identify "close calls" (score 60-64)
|
||||
- Compare with executed trades that had similar scores
|
||||
- Determine if threshold adjustment is warranted
|
||||
|
||||
### Phase 3: Automation (Future)
|
||||
- Build price monitoring job
|
||||
- Auto-calculate would-be outcomes
|
||||
- Generate reports on missed opportunities
|
||||
- Feed data into threshold optimization algorithm
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Data-Driven Decisions** - No guessing, only facts
|
||||
2. **Prevents Over-Optimization** - Wait for statistically significant sample
|
||||
3. **Tracks All Block Reasons** - Not just quality score
|
||||
4. **Historical Record** - Can review past decisions
|
||||
5. **Continuous Improvement** - System learns from what it blocks
|
||||
|
||||
## Important Notes
|
||||
|
||||
⚠️ **Don't change thresholds prematurely!**
|
||||
- 2 trades is NOT enough data
|
||||
- Wait for 10-20 blocked signals minimum
|
||||
- Analyze patterns before making changes
|
||||
|
||||
✅ **System is working correctly if:**
|
||||
- Blocked signals appear in database
|
||||
- Each has metrics (ATR, ADX, RSI, etc.)
|
||||
- Block reason is recorded
|
||||
- Timestamp is correct
|
||||
|
||||
❌ **Troubleshooting:**
|
||||
- If no blocked signals appear: Check bot is receiving TradingView alerts with metrics
|
||||
- If missing metrics: Ensure TradingView webhook includes ATR/ADX/RSI/volume/pricePosition
|
||||
- If database errors: Check Prisma client is regenerated after schema changes
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** November 11, 2025
|
||||
**Version:** 1.0
|
||||
**Maintained By:** Trading Bot v4 Development Team
|
||||
369
SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md
Normal file
369
SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md
Normal file
@@ -0,0 +1,369 @@
|
||||
# Signal Quality Optimization Roadmap
|
||||
|
||||
**Goal:** Optimize signal quality thresholds and scoring logic using data-driven analysis
|
||||
|
||||
**Current Status:** Phase 1 - Data Collection (Active)
|
||||
**Last Updated:** November 11, 2025
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This roadmap guides the systematic improvement of signal quality filtering. We follow a **data-first approach**: collect evidence, analyze patterns, then make changes. No premature optimization.
|
||||
|
||||
### Current System
|
||||
- **Quality Score Threshold:** 65 points (recently raised from 60)
|
||||
- **Executed Trades:** 157 total (155 closed, 2 open)
|
||||
- **Performance:** +$3.43 total P&L, 44.5% win rate
|
||||
- **Score Distribution:**
|
||||
- 80-100 (Excellent): 49 trades, +$46.48, 46.9% WR
|
||||
- 70-79 (Good): 15 trades, -$2.20, 40.0% WR ⚠️
|
||||
- 65-69 (Pass): 13 trades, +$28.28, 53.8% WR ✅
|
||||
- 60-64 (Just Below): 2 trades, +$45.78, **100% WR** 🔥
|
||||
- 0-49 (Very Weak): 13 trades, -$127.89, 30.8% WR 💀
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Data Collection (CURRENT) ✅ IN PROGRESS
|
||||
|
||||
**Status:** Infrastructure complete, collecting data
|
||||
**Started:** November 11, 2025
|
||||
**Target:** Collect 10-20 blocked signals (1-2 weeks)
|
||||
|
||||
### Completed (Nov 11, 2025)
|
||||
- [x] Created `BlockedSignal` database table
|
||||
- [x] Implemented automatic saving in check-risk endpoint
|
||||
- [x] Deployed to production (trading-bot-v4 container)
|
||||
- [x] Created tracking documentation (BLOCKED_SIGNALS_TRACKING.md)
|
||||
|
||||
### What's Being Tracked
|
||||
Every blocked signal captures:
|
||||
- **Metrics:** ATR, ADX, RSI, volume ratio, price position, timeframe
|
||||
- **Score:** Quality score (0-100), version, detailed breakdown
|
||||
- **Block Reason:** Quality score, cooldown, hourly limit, daily drawdown
|
||||
- **Context:** Symbol, direction, price at signal time, timestamp
|
||||
|
||||
### What We're Looking For
|
||||
1. How many signals score 60-64 (just below threshold)?
|
||||
2. What are their characteristics (ADX, ATR, price position)?
|
||||
3. Are there patterns (extreme positions, specific timeframes)?
|
||||
4. Do they cluster around specific block reasons?
|
||||
|
||||
### Phase 1 Completion Criteria
|
||||
- [ ] Minimum 10 blocked signals with quality scores 55-64
|
||||
- [ ] At least 2 signals in 60-64 range (close calls)
|
||||
- [ ] Mix of block reasons (not all quality score)
|
||||
- [ ] Data spans multiple market conditions (trending, choppy, volatile)
|
||||
|
||||
### SQL Queries for Phase 1
|
||||
```sql
|
||||
-- Check progress
|
||||
SELECT COUNT(*) as total_blocked
|
||||
FROM "BlockedSignal";
|
||||
|
||||
-- Score distribution
|
||||
SELECT
|
||||
CASE
|
||||
WHEN signalQualityScore >= 60 THEN '60-64 (Close)'
|
||||
WHEN signalQualityScore >= 55 THEN '55-59 (Marginal)'
|
||||
WHEN signalQualityScore >= 50 THEN '50-54 (Weak)'
|
||||
ELSE '0-49 (Very Weak)'
|
||||
END as tier,
|
||||
COUNT(*) as count
|
||||
FROM "BlockedSignal"
|
||||
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
|
||||
GROUP BY tier
|
||||
ORDER BY MIN(signalQualityScore) DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Pattern Analysis 🔜 NEXT
|
||||
|
||||
**Prerequisites:** 10-20 blocked signals collected
|
||||
**Estimated Duration:** 2-3 days
|
||||
**Owner:** Manual analysis + SQL queries
|
||||
|
||||
### Analysis Tasks
|
||||
|
||||
#### 2.1: Score Distribution Analysis
|
||||
```sql
|
||||
-- Analyze blocked signals by score range
|
||||
SELECT
|
||||
CASE
|
||||
WHEN signalQualityScore >= 60 THEN '60-64'
|
||||
WHEN signalQualityScore >= 55 THEN '55-59'
|
||||
ELSE '50-54'
|
||||
END as score_range,
|
||||
COUNT(*) as count,
|
||||
ROUND(AVG(atr)::numeric, 2) as avg_atr,
|
||||
ROUND(AVG(adx)::numeric, 1) as avg_adx,
|
||||
ROUND(AVG(pricePosition)::numeric, 1) as avg_price_pos,
|
||||
ROUND(AVG(volumeRatio)::numeric, 2) as avg_volume
|
||||
FROM "BlockedSignal"
|
||||
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
|
||||
GROUP BY score_range
|
||||
ORDER BY MIN(signalQualityScore) DESC;
|
||||
```
|
||||
|
||||
#### 2.2: Compare with Executed Trades
|
||||
```sql
|
||||
-- Find executed trades with similar scores to blocked signals
|
||||
SELECT
|
||||
'Executed' as type,
|
||||
signalQualityScore,
|
||||
COUNT(*) as trades,
|
||||
ROUND(AVG(realizedPnL)::numeric, 2) as avg_pnl,
|
||||
ROUND(100.0 * SUM(CASE WHEN realizedPnL > 0 THEN 1 ELSE 0 END) / COUNT(*)::numeric, 1) as win_rate
|
||||
FROM "Trade"
|
||||
WHERE exitReason IS NOT NULL
|
||||
AND signalQualityScore BETWEEN 60 AND 69
|
||||
GROUP BY signalQualityScore
|
||||
ORDER BY signalQualityScore;
|
||||
```
|
||||
|
||||
#### 2.3: ADX Pattern Analysis
|
||||
Key finding from existing data: ADX 20-25 is a trap zone!
|
||||
```sql
|
||||
-- ADX distribution in blocked signals
|
||||
SELECT
|
||||
CASE
|
||||
WHEN adx >= 25 THEN 'Strong (25+)'
|
||||
WHEN adx >= 20 THEN 'Moderate (20-25)'
|
||||
WHEN adx >= 15 THEN 'Weak (15-20)'
|
||||
ELSE 'Very Weak (<15)'
|
||||
END as adx_tier,
|
||||
COUNT(*) as count,
|
||||
ROUND(AVG(signalQualityScore)::numeric, 1) as avg_score
|
||||
FROM "BlockedSignal"
|
||||
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
|
||||
AND adx IS NOT NULL
|
||||
GROUP BY adx_tier
|
||||
ORDER BY MIN(adx) DESC;
|
||||
```
|
||||
|
||||
#### 2.4: Extreme Position Analysis
|
||||
Test hypothesis: Extremes (<10% or >90%) need different thresholds
|
||||
```sql
|
||||
-- Blocked signals at range extremes
|
||||
SELECT
|
||||
direction,
|
||||
signalQualityScore,
|
||||
ROUND(pricePosition::numeric, 1) as pos,
|
||||
ROUND(adx::numeric, 1) as adx,
|
||||
ROUND(volumeRatio::numeric, 2) as vol
|
||||
FROM "BlockedSignal"
|
||||
WHERE blockReason = 'QUALITY_SCORE_TOO_LOW'
|
||||
AND (pricePosition < 10 OR pricePosition > 90)
|
||||
ORDER BY signalQualityScore DESC;
|
||||
```
|
||||
|
||||
### Phase 2 Deliverables
|
||||
- [ ] Score distribution report
|
||||
- [ ] ADX pattern analysis
|
||||
- [ ] Extreme position analysis
|
||||
- [ ] Comparison with executed trades
|
||||
- [ ] **DECISION:** Keep threshold at 65, lower to 60, or implement dual-threshold system
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Implementation (Conditional) 🎯 FUTURE
|
||||
|
||||
**Trigger:** Analysis shows clear pattern worth exploiting
|
||||
**Prerequisites:** Phase 2 complete + statistical significance (15+ blocked signals)
|
||||
|
||||
### Option A: Dual-Threshold System (Recommended)
|
||||
**IF** data shows extreme positions (price <10% or >90%) with scores 60-64 are profitable:
|
||||
|
||||
**Implementation:**
|
||||
```typescript
|
||||
// In check-risk endpoint
|
||||
const isExtremePosition = pricePosition < 10 || pricePosition > 90
|
||||
const requiredScore = isExtremePosition ? 60 : 65
|
||||
|
||||
if (qualityScore.score < requiredScore) {
|
||||
// Block signal
|
||||
}
|
||||
```
|
||||
|
||||
**Changes Required:**
|
||||
- `app/api/trading/check-risk/route.ts` - Add dual threshold logic
|
||||
- `lib/trading/signal-quality.ts` - Add `isExtremePosition` helper
|
||||
- `config/trading.ts` - Add `minScoreForExtremes` config option
|
||||
- Update AI instructions with new logic
|
||||
|
||||
### Option B: ADX-Based Gates (Alternative)
|
||||
**IF** data shows strong ADX trends (25+) with lower scores are profitable:
|
||||
|
||||
**Implementation:**
|
||||
```typescript
|
||||
const requiredScore = adx >= 25 ? 60 : 65
|
||||
```
|
||||
|
||||
**Changes Required:**
|
||||
- Similar to Option A but based on ADX threshold
|
||||
|
||||
### Option C: Keep Current (If No Clear Pattern)
|
||||
**IF** data shows no consistent profit opportunity in blocked signals:
|
||||
- No changes needed
|
||||
- Continue monitoring
|
||||
- Revisit in 20 more trades
|
||||
|
||||
### Phase 3 Checklist
|
||||
- [ ] Decision made based on Phase 2 analysis
|
||||
- [ ] Code changes implemented
|
||||
- [ ] Updated signalQualityVersion to 'v5' in database
|
||||
- [ ] AI instructions updated
|
||||
- [ ] Tested with historical blocked signals
|
||||
- [ ] Deployed to production
|
||||
- [ ] Monitoring for 10 trades to validate improvement
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Price Analysis Automation 🤖 FUTURE
|
||||
|
||||
**Goal:** Automatically track if blocked signals would have been profitable
|
||||
**Complexity:** Medium - requires price monitoring job
|
||||
**Prerequisites:** Phase 3 complete OR 50+ blocked signals collected
|
||||
|
||||
### Architecture
|
||||
```
|
||||
Monitoring Job (runs every 30 min)
|
||||
↓
|
||||
Fetch BlockedSignal records where:
|
||||
- analysisComplete = false
|
||||
- createdAt > 30 minutes ago
|
||||
↓
|
||||
For each signal:
|
||||
- Get price history from Pyth/Drift
|
||||
- Calculate if TP1/TP2/SL would have been hit
|
||||
- Update priceAfter1Min/5Min/15Min/30Min
|
||||
- Set wouldHitTP1/TP2/SL flags
|
||||
- Mark analysisComplete = true
|
||||
↓
|
||||
Save results back to database
|
||||
```
|
||||
|
||||
### Implementation Tasks
|
||||
- [ ] Create price history fetching service
|
||||
- [ ] Implement TP/SL hit calculation logic
|
||||
- [ ] Create cron job or Next.js API route with scheduler
|
||||
- [ ] Add monitoring dashboard for blocked signal outcomes
|
||||
- [ ] Generate weekly reports on missed opportunities
|
||||
|
||||
### Success Metrics
|
||||
- X% of blocked signals would have hit SL (blocks were correct)
|
||||
- Y% would have hit TP1/TP2 (missed opportunities)
|
||||
- Overall P&L of hypothetical blocked trades
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: ML-Based Optimization 🧠 DISTANT FUTURE
|
||||
|
||||
**Goal:** Use machine learning to optimize scoring weights
|
||||
**Prerequisites:** 200+ trades with quality scores, 100+ blocked signals
|
||||
**Complexity:** High
|
||||
|
||||
### Approach
|
||||
1. Extract features: ATR, ADX, RSI, volume, price position, timeframe
|
||||
2. Train model on: executed trades (outcome = P&L)
|
||||
3. Validate on: blocked signals (if price analysis complete)
|
||||
4. Generate: Optimal scoring weights for each feature
|
||||
5. Implement: Dynamic threshold adjustment based on market conditions
|
||||
|
||||
### Not Implemented Yet
|
||||
This is a future consideration only. Current data-driven approach is sufficient.
|
||||
|
||||
---
|
||||
|
||||
## Key Principles
|
||||
|
||||
### 1. Data Before Action
|
||||
- Minimum 10 samples before any decision
|
||||
- Prefer 20+ for statistical confidence
|
||||
- No changes based on 1-2 outliers
|
||||
|
||||
### 2. Incremental Changes
|
||||
- Change one variable at a time
|
||||
- Test for 10-20 trades after each change
|
||||
- Revert if performance degrades
|
||||
|
||||
### 3. Version Tracking
|
||||
- Every scoring logic change gets new version (v4 → v5)
|
||||
- Store version with each trade/blocked signal
|
||||
- Enables A/B testing and rollback
|
||||
|
||||
### 4. Document Everything
|
||||
- Update this roadmap after each phase
|
||||
- Record decisions and rationale
|
||||
- Link to SQL queries and analysis
|
||||
|
||||
---
|
||||
|
||||
## Progress Tracking
|
||||
|
||||
### Milestones
|
||||
- [x] Nov 11, 2025: Phase 1 infrastructure complete
|
||||
- [ ] Target: ~Nov 20-25, 2025: Phase 1 complete (10-20 blocked signals)
|
||||
- [ ] Target: ~Nov 25-30, 2025: Phase 2 analysis complete
|
||||
- [ ] TBD: Phase 3 implementation (conditional)
|
||||
|
||||
### Metrics to Watch
|
||||
- **Blocked signals collected:** 0/10 minimum
|
||||
- **Close calls (60-64 score):** 0/2 minimum
|
||||
- **Days of data collection:** 0/7 minimum
|
||||
- **Market conditions covered:** 0/3 (trending, choppy, volatile)
|
||||
|
||||
### Review Schedule
|
||||
- **Weekly:** Check blocked signal count
|
||||
- **After 10 blocked:** Run Phase 2 analysis
|
||||
- **After Phase 2:** Decide on Phase 3 implementation
|
||||
- **Monthly:** Review overall system performance
|
||||
|
||||
---
|
||||
|
||||
## Questions to Answer
|
||||
|
||||
### Phase 1 Questions
|
||||
- [ ] How many signals get blocked per day?
|
||||
- [ ] What's the score distribution of blocked signals?
|
||||
- [ ] Are most blocks from quality score or other reasons?
|
||||
|
||||
### Phase 2 Questions
|
||||
- [ ] Do blocked signals at 60-64 have common characteristics?
|
||||
- [ ] Would lowering threshold to 60 improve performance?
|
||||
- [ ] Do extreme positions need different treatment?
|
||||
- [ ] Is ADX pattern valid in blocked signals?
|
||||
|
||||
### Phase 3 Questions
|
||||
- [ ] Did the change improve win rate?
|
||||
- [ ] Did it increase profitability?
|
||||
- [ ] Any unintended side effects?
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Historical Context
|
||||
|
||||
### Why This Roadmap Exists
|
||||
**Date:** November 11, 2025
|
||||
|
||||
**Situation:** Three TradingView signals fired:
|
||||
1. SHORT at 05:15 - Executed (score likely 65+) → Losing trade
|
||||
2. LONG at 05:20 - Executed (score likely 65+) → Losing trade
|
||||
3. SHORT at 05:30 - **BLOCKED** (score 45) → Would have been profitable
|
||||
|
||||
**User Question:** "What can we do about this?"
|
||||
|
||||
**Analysis Findings:**
|
||||
- Only 2 historical trades scored 60-64 (both winners +$45.78)
|
||||
- Sample size too small for confident decision
|
||||
- ADX 20-25 is a trap zone (-$23.41 in 23 trades)
|
||||
- Low volume (<0.8x) outperforms high volume (counterintuitive!)
|
||||
|
||||
**Decision:** Build data collection system instead of changing thresholds prematurely
|
||||
|
||||
**This Roadmap:** Systematic approach to optimization with proper data backing
|
||||
|
||||
---
|
||||
|
||||
**Remember:** The goal isn't to catch every winning trade. The goal is to optimize the **risk-adjusted return** by catching more winners than losers at each threshold level. Sometimes blocking a potential winner is correct if it also blocks 3 losers.
|
||||
@@ -8,7 +8,7 @@
|
||||
import { NextRequest, NextResponse } from 'next/server'
|
||||
import { getMergedConfig, TradingConfig } from '@/config/trading'
|
||||
import { getInitializedPositionManager, ActiveTrade } from '@/lib/trading/position-manager'
|
||||
import { getLastTradeTime, getLastTradeTimeForSymbol, getTradesInLastHour, getTodayPnL } from '@/lib/database/trades'
|
||||
import { getLastTradeTime, getLastTradeTimeForSymbol, getTradesInLastHour, getTodayPnL, createBlockedSignal } from '@/lib/database/trades'
|
||||
import { getPythPriceMonitor } from '@/lib/pyth/price-monitor'
|
||||
import { scoreSignalQuality, SignalQualityResult } from '@/lib/trading/signal-quality'
|
||||
|
||||
@@ -205,6 +205,9 @@ export async function POST(request: NextRequest): Promise<NextResponse<RiskCheck
|
||||
// Continue to quality checks below instead of returning early
|
||||
}
|
||||
|
||||
// Check if we have context metrics (used throughout the function)
|
||||
const hasContextMetrics = body.atr !== undefined && body.atr > 0
|
||||
|
||||
// 1. Check daily drawdown limit
|
||||
const todayPnL = await getTodayPnL()
|
||||
if (todayPnL < config.maxDailyDrawdown) {
|
||||
@@ -228,6 +231,28 @@ export async function POST(request: NextRequest): Promise<NextResponse<RiskCheck
|
||||
maxTradesPerHour: config.maxTradesPerHour,
|
||||
})
|
||||
|
||||
// Save blocked signal if we have metrics
|
||||
if (hasContextMetrics) {
|
||||
const priceMonitor = getPythPriceMonitor()
|
||||
const latestPrice = priceMonitor.getCachedPrice(body.symbol)
|
||||
|
||||
await createBlockedSignal({
|
||||
symbol: body.symbol,
|
||||
direction: body.direction,
|
||||
timeframe: body.timeframe,
|
||||
signalPrice: latestPrice?.price || 0,
|
||||
atr: body.atr,
|
||||
adx: body.adx,
|
||||
rsi: body.rsi,
|
||||
volumeRatio: body.volumeRatio,
|
||||
pricePosition: body.pricePosition,
|
||||
signalQualityScore: 0, // Not calculated yet
|
||||
minScoreRequired: config.minSignalQualityScore,
|
||||
blockReason: 'HOURLY_TRADE_LIMIT',
|
||||
blockDetails: `${tradesInLastHour} trades in last hour (max: ${config.maxTradesPerHour})`,
|
||||
})
|
||||
}
|
||||
|
||||
return NextResponse.json({
|
||||
allowed: false,
|
||||
reason: 'Hourly trade limit',
|
||||
@@ -252,6 +277,28 @@ export async function POST(request: NextRequest): Promise<NextResponse<RiskCheck
|
||||
remainingMinutes,
|
||||
})
|
||||
|
||||
// Save blocked signal if we have metrics
|
||||
if (hasContextMetrics) {
|
||||
const priceMonitor = getPythPriceMonitor()
|
||||
const latestPrice = priceMonitor.getCachedPrice(body.symbol)
|
||||
|
||||
await createBlockedSignal({
|
||||
symbol: body.symbol,
|
||||
direction: body.direction,
|
||||
timeframe: body.timeframe,
|
||||
signalPrice: latestPrice?.price || 0,
|
||||
atr: body.atr,
|
||||
adx: body.adx,
|
||||
rsi: body.rsi,
|
||||
volumeRatio: body.volumeRatio,
|
||||
pricePosition: body.pricePosition,
|
||||
signalQualityScore: 0, // Not calculated yet
|
||||
minScoreRequired: config.minSignalQualityScore,
|
||||
blockReason: 'COOLDOWN_PERIOD',
|
||||
blockDetails: `Wait ${remainingMinutes} more min (cooldown: ${config.minTimeBetweenTrades} min)`,
|
||||
})
|
||||
}
|
||||
|
||||
return NextResponse.json({
|
||||
allowed: false,
|
||||
reason: 'Cooldown period',
|
||||
@@ -261,8 +308,6 @@ export async function POST(request: NextRequest): Promise<NextResponse<RiskCheck
|
||||
}
|
||||
|
||||
// 4. Check signal quality (if context metrics provided)
|
||||
const hasContextMetrics = body.atr !== undefined && body.atr > 0
|
||||
|
||||
if (hasContextMetrics) {
|
||||
const qualityScore = scoreSignalQuality({
|
||||
atr: body.atr || 0,
|
||||
@@ -272,15 +317,39 @@ export async function POST(request: NextRequest): Promise<NextResponse<RiskCheck
|
||||
pricePosition: body.pricePosition || 0,
|
||||
direction: body.direction,
|
||||
timeframe: body.timeframe, // Pass timeframe for context-aware scoring
|
||||
minScore: 60 // Hardcoded threshold
|
||||
minScore: config.minSignalQualityScore // Use config value
|
||||
})
|
||||
|
||||
if (!qualityScore.passed) {
|
||||
console.log('🚫 Risk check BLOCKED: Signal quality too low', {
|
||||
score: qualityScore.score,
|
||||
threshold: config.minSignalQualityScore,
|
||||
reasons: qualityScore.reasons
|
||||
})
|
||||
|
||||
// Get current price for the blocked signal record
|
||||
const priceMonitor = getPythPriceMonitor()
|
||||
const latestPrice = priceMonitor.getCachedPrice(body.symbol)
|
||||
|
||||
// Save blocked signal to database for future analysis
|
||||
await createBlockedSignal({
|
||||
symbol: body.symbol,
|
||||
direction: body.direction,
|
||||
timeframe: body.timeframe,
|
||||
signalPrice: latestPrice?.price || 0,
|
||||
atr: body.atr,
|
||||
adx: body.adx,
|
||||
rsi: body.rsi,
|
||||
volumeRatio: body.volumeRatio,
|
||||
pricePosition: body.pricePosition,
|
||||
signalQualityScore: qualityScore.score,
|
||||
signalQualityVersion: 'v4', // Update this when scoring logic changes
|
||||
scoreBreakdown: { reasons: qualityScore.reasons },
|
||||
minScoreRequired: config.minSignalQualityScore,
|
||||
blockReason: 'QUALITY_SCORE_TOO_LOW',
|
||||
blockDetails: `Score: ${qualityScore.score}/${config.minSignalQualityScore} - ${qualityScore.reasons.join(', ')}`,
|
||||
})
|
||||
|
||||
return NextResponse.json({
|
||||
allowed: false,
|
||||
reason: 'Signal quality too low',
|
||||
|
||||
@@ -471,6 +471,88 @@ export async function getTradeStats(days: number = 30) {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Save blocked signal for analysis
|
||||
*/
|
||||
export interface CreateBlockedSignalParams {
|
||||
symbol: string
|
||||
direction: 'long' | 'short'
|
||||
timeframe?: string
|
||||
signalPrice: number
|
||||
atr?: number
|
||||
adx?: number
|
||||
rsi?: number
|
||||
volumeRatio?: number
|
||||
pricePosition?: number
|
||||
signalQualityScore: number
|
||||
signalQualityVersion?: string
|
||||
scoreBreakdown?: any
|
||||
minScoreRequired: number
|
||||
blockReason: string
|
||||
blockDetails?: string
|
||||
}
|
||||
|
||||
export async function createBlockedSignal(params: CreateBlockedSignalParams) {
|
||||
const client = getPrismaClient()
|
||||
|
||||
try {
|
||||
const blockedSignal = await client.blockedSignal.create({
|
||||
data: {
|
||||
symbol: params.symbol,
|
||||
direction: params.direction,
|
||||
timeframe: params.timeframe,
|
||||
signalPrice: params.signalPrice,
|
||||
atr: params.atr,
|
||||
adx: params.adx,
|
||||
rsi: params.rsi,
|
||||
volumeRatio: params.volumeRatio,
|
||||
pricePosition: params.pricePosition,
|
||||
signalQualityScore: params.signalQualityScore,
|
||||
signalQualityVersion: params.signalQualityVersion,
|
||||
scoreBreakdown: params.scoreBreakdown,
|
||||
minScoreRequired: params.minScoreRequired,
|
||||
blockReason: params.blockReason,
|
||||
blockDetails: params.blockDetails,
|
||||
},
|
||||
})
|
||||
|
||||
console.log(`📝 Blocked signal saved: ${params.symbol} ${params.direction} (score: ${params.signalQualityScore}/${params.minScoreRequired})`)
|
||||
return blockedSignal
|
||||
} catch (error) {
|
||||
console.error('❌ Failed to save blocked signal:', error)
|
||||
// Don't throw - blocking shouldn't fail the check-risk process
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get recent blocked signals for analysis
|
||||
*/
|
||||
export async function getRecentBlockedSignals(limit: number = 20) {
|
||||
const client = getPrismaClient()
|
||||
return client.blockedSignal.findMany({
|
||||
orderBy: { createdAt: 'desc' },
|
||||
take: limit,
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Get blocked signals that need price analysis
|
||||
*/
|
||||
export async function getBlockedSignalsForAnalysis(olderThanMinutes: number = 30) {
|
||||
const client = getPrismaClient()
|
||||
const cutoffTime = new Date(Date.now() - olderThanMinutes * 60 * 1000)
|
||||
|
||||
return client.blockedSignal.findMany({
|
||||
where: {
|
||||
analysisComplete: false,
|
||||
createdAt: { lt: cutoffTime },
|
||||
},
|
||||
orderBy: { createdAt: 'asc' },
|
||||
take: 50,
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Disconnect Prisma client (for graceful shutdown)
|
||||
*/
|
||||
|
||||
@@ -77,6 +77,7 @@ model Trade {
|
||||
volumeAtEntry Float? // Volume relative to MA
|
||||
pricePositionAtEntry Float? // Price position in range (0-100%)
|
||||
signalQualityScore Int? // Calculated quality score (0-100)
|
||||
signalQualityVersion String? @default("v1") // Tracks which scoring logic was used
|
||||
fundingRateAtEntry Float? // Perp funding rate at entry
|
||||
basisAtEntry Float? // Perp-spot basis at entry
|
||||
|
||||
@@ -151,6 +152,52 @@ model SystemEvent {
|
||||
@@index([createdAt])
|
||||
}
|
||||
|
||||
// Blocked signals for analysis (signals that didn't pass quality checks)
|
||||
model BlockedSignal {
|
||||
id String @id @default(cuid())
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
// Signal identification
|
||||
symbol String // e.g., "SOL-PERP"
|
||||
direction String // "long" or "short"
|
||||
timeframe String? // "5", "15", "60"
|
||||
|
||||
// Price at signal time
|
||||
signalPrice Float // Price when signal was generated
|
||||
|
||||
// Market metrics at signal time
|
||||
atr Float? // ATR% at signal
|
||||
adx Float? // ADX trend strength
|
||||
rsi Float? // RSI momentum
|
||||
volumeRatio Float? // Volume relative to average
|
||||
pricePosition Float? // Position in range (0-100%)
|
||||
|
||||
// Quality scoring
|
||||
signalQualityScore Int // 0-100 score
|
||||
signalQualityVersion String? // Which scoring version
|
||||
scoreBreakdown Json? // Detailed breakdown of score components
|
||||
minScoreRequired Int // What threshold was used (e.g., 65)
|
||||
|
||||
// Block reason
|
||||
blockReason String // "QUALITY_SCORE_TOO_LOW", "DUPLICATE", "COOLDOWN", etc.
|
||||
blockDetails String? // Human-readable details
|
||||
|
||||
// For later analysis: track if it would have been profitable
|
||||
priceAfter1Min Float? // Price 1 minute after (filled by monitoring job)
|
||||
priceAfter5Min Float? // Price 5 minutes after
|
||||
priceAfter15Min Float? // Price 15 minutes after
|
||||
priceAfter30Min Float? // Price 30 minutes after
|
||||
wouldHitTP1 Boolean? // Would TP1 have been hit?
|
||||
wouldHitTP2 Boolean? // Would TP2 have been hit?
|
||||
wouldHitSL Boolean? // Would SL have been hit?
|
||||
analysisComplete Boolean @default(false) // Has post-analysis been done?
|
||||
|
||||
@@index([symbol])
|
||||
@@index([createdAt])
|
||||
@@index([signalQualityScore])
|
||||
@@index([blockReason])
|
||||
}
|
||||
|
||||
// Performance analytics (daily aggregates)
|
||||
model DailyStats {
|
||||
id String @id @default(cuid())
|
||||
|
||||
Reference in New Issue
Block a user