docs: Add comprehensive documentation for direction-specific quality thresholds
- Complete implementation guide with data-driven rationale - Testing results and verification steps - Deployment checklist and common pitfalls - Monitoring queries and configuration management - Future enhancement roadmap
This commit is contained in:
mindesbunister
313
docs/DIRECTION_SPECIFIC_QUALITY_THRESHOLDS.md
Normal file
313
docs/DIRECTION_SPECIFIC_QUALITY_THRESHOLDS.md
Normal file
@@ -0,0 +1,313 @@
|
||||
# Direction-Specific Quality Thresholds
|
||||
|
||||
**Implementation Date:** November 23, 2025
|
||||
**Status:** ✅ DEPLOYED and TESTED
|
||||
**Commits:** 01aaa09, 357626b
|
||||
|
||||
## Overview
|
||||
|
||||
Trading bot now uses different signal quality thresholds based on trade direction (LONG vs SHORT) to capture profitable setups while blocking toxic ones.
|
||||
|
||||
## Data-Driven Decision
|
||||
|
||||
### Historical Analysis (227 completed trades)
|
||||
|
||||
**Quality 90-94 Performance:**
|
||||
- **LONGS:** 7 trades, **71.4% WR**, +$44.77 total (+$6.40 avg per trade)
|
||||
- **SHORTS:** 7 trades, **28.6% WR**, -$553.76 total (-$79.11 avg per trade)
|
||||
- **Difference:** $598.53 P&L gap between same quality level
|
||||
|
||||
**Quality 90+ Overall (All History):**
|
||||
- **LONGS:** 38 trades, 50.0% WR, **+$600.62** total
|
||||
- **SHORTS:** 38 trades, 47.4% WR, **-$177.90** total
|
||||
- **Total difference:** $778.52 (longs vastly outperform)
|
||||
|
||||
**v8 Indicator Directional Performance:**
|
||||
- **LONGS:** 3 trades, **100% WR**, +$565.03 (avg +$188.34)
|
||||
- **SHORTS:** 7 trades, 42.9% WR, -$311.68 (avg -$44.53)
|
||||
|
||||
### User Decision
|
||||
|
||||
**User Query:** "are longs more profitable as shorts? what does our data say? should we maybe enable normal entries at 90 quality score long signals?"
|
||||
|
||||
**Agent Analysis:** Data shows clear directional edge - longs at quality 90-94 profitable (71.4% WR), shorts toxic (28.6% WR).
|
||||
|
||||
**User Approval:** "yes. go"
|
||||
|
||||
## Implementation
|
||||
|
||||
### Configuration
|
||||
|
||||
**New ENV Variables (.env):**
|
||||
```bash
|
||||
MIN_SIGNAL_QUALITY_SCORE=91 # Global fallback (when no direction-specific set)
|
||||
MIN_SIGNAL_QUALITY_SCORE_LONG=90 # Longs: 71.4% WR at 90-94
|
||||
MIN_SIGNAL_QUALITY_SCORE_SHORT=95 # Shorts: Block toxic 90-94 range
|
||||
```
|
||||
|
||||
**Docker Compose (docker-compose.yml):**
|
||||
```yaml
|
||||
environment:
|
||||
MIN_SIGNAL_QUALITY_SCORE: ${MIN_SIGNAL_QUALITY_SCORE:-91}
|
||||
MIN_SIGNAL_QUALITY_SCORE_LONG: ${MIN_SIGNAL_QUALITY_SCORE_LONG:-90}
|
||||
MIN_SIGNAL_QUALITY_SCORE_SHORT: ${MIN_SIGNAL_QUALITY_SCORE_SHORT:-95}
|
||||
```
|
||||
|
||||
### Code Changes
|
||||
|
||||
**1. Trading Config Interface (config/trading.ts):**
|
||||
```typescript
|
||||
export interface TradingConfig {
|
||||
// ... existing fields
|
||||
minSignalQualityScoreLong?: number // Direction-specific threshold for longs
|
||||
minSignalQualityScoreShort?: number // Direction-specific threshold for shorts
|
||||
}
|
||||
|
||||
export const DEFAULT_TRADING_CONFIG: TradingConfig = {
|
||||
// ... existing defaults
|
||||
minSignalQualityScoreLong: 90, // Data-driven: 71.4% WR at 90-94
|
||||
minSignalQualityScoreShort: 95, // Data-driven: Block toxic 90-94 shorts
|
||||
}
|
||||
```
|
||||
|
||||
**2. Helper Function (config/trading.ts):**
|
||||
```typescript
|
||||
/**
|
||||
* Get minimum quality score based on trade direction
|
||||
* Nov 23, 2025: Data shows longs profitable at 90-94 (71.4% WR), shorts toxic (28.6% WR)
|
||||
*/
|
||||
export function getMinQualityScoreForDirection(
|
||||
direction: 'long' | 'short',
|
||||
config: TradingConfig
|
||||
): number {
|
||||
// Direction-specific threshold if set
|
||||
if (direction === 'long' && config.minSignalQualityScoreLong !== undefined) {
|
||||
return config.minSignalQualityScoreLong
|
||||
}
|
||||
if (direction === 'short' && config.minSignalQualityScoreShort !== undefined) {
|
||||
return config.minSignalQualityScoreShort
|
||||
}
|
||||
|
||||
// Fallback: global → 60 default
|
||||
return config.minSignalQualityScore ?? 60
|
||||
}
|
||||
```
|
||||
|
||||
**3. Check-Risk Endpoint (app/api/trading/check-risk/route.ts):**
|
||||
```typescript
|
||||
// Use direction-specific quality threshold (Nov 23, 2025)
|
||||
const minQualityScore = getMinQualityScoreForDirection(body.direction, config)
|
||||
|
||||
const qualityScore = await scoreSignalQuality({
|
||||
atr: body.atr || 0,
|
||||
adx: body.adx || 0,
|
||||
rsi: body.rsi || 0,
|
||||
volumeRatio: body.volumeRatio || 0,
|
||||
pricePosition: body.pricePosition || 0,
|
||||
direction: body.direction,
|
||||
symbol: body.symbol,
|
||||
currentPrice: currentPrice,
|
||||
timeframe: body.timeframe,
|
||||
minScore: minQualityScore // Direction-specific threshold
|
||||
})
|
||||
|
||||
if (!qualityScore.passed) {
|
||||
console.log('🚫 Risk check BLOCKED: Signal quality too low', {
|
||||
score: qualityScore.score,
|
||||
direction: body.direction,
|
||||
threshold: minQualityScore, // Logs show 90 for longs, 95 for shorts
|
||||
reasons: qualityScore.reasons
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
**4. Signal Quality Scoring (lib/trading/signal-quality.ts):**
|
||||
```typescript
|
||||
export async function scoreSignalQuality(params: {
|
||||
atr: number
|
||||
adx: number
|
||||
rsi: number
|
||||
volumeRatio: number
|
||||
pricePosition: number
|
||||
direction: 'long' | 'short'
|
||||
symbol: string
|
||||
currentPrice?: number
|
||||
timeframe?: string
|
||||
minScore?: number // Direction-specific threshold passed in
|
||||
}): Promise<SignalQualityResult> {
|
||||
// ... scoring logic
|
||||
|
||||
const minScore = params.minScore || 60 // Use provided threshold or fallback
|
||||
const passed = score >= minScore
|
||||
|
||||
return { score, passed, reasons }
|
||||
}
|
||||
```
|
||||
|
||||
## Deployment Steps
|
||||
|
||||
1. ✅ Modified 3 code files (config, signal-quality, check-risk)
|
||||
2. ✅ Added ENV variables to .env file
|
||||
3. ✅ Added ENV variables to docker-compose.yml (required for process.env access)
|
||||
4. ✅ Built Docker container (71.8s build time)
|
||||
5. ✅ Restarted container with `docker compose down && docker compose up -d`
|
||||
6. ✅ Verified ENV variables loaded: `docker exec trading-bot-v4 printenv | grep MIN_SIGNAL`
|
||||
7. ✅ Tested with curl: LONG quality 90 ✅ ALLOWED, SHORT quality 70 ❌ BLOCKED
|
||||
|
||||
## Testing Results
|
||||
|
||||
**Test 1: Quality 90 LONG (should PASS)**
|
||||
```json
|
||||
{
|
||||
"allowed": true,
|
||||
"details": "All risk checks passed",
|
||||
"qualityScore": 90,
|
||||
"qualityReasons": [
|
||||
"ATR healthy (0.43%)",
|
||||
"Strong trend for 5min (ADX 22.5)",
|
||||
"RSI supports long (58.0)",
|
||||
"Price position OK (45%)"
|
||||
]
|
||||
}
|
||||
```
|
||||
✅ **PASSED** - Threshold 90 correctly applied
|
||||
|
||||
**Test 2: Quality 70 SHORT (should BLOCK)**
|
||||
```json
|
||||
{
|
||||
"allowed": false,
|
||||
"reason": "Signal quality too low",
|
||||
"details": "Score: 70/100",
|
||||
"qualityScore": 70
|
||||
}
|
||||
```
|
||||
✅ **BLOCKED** - Threshold 95 correctly applied (logs showed `threshold: 95`)
|
||||
|
||||
## Expected Impact
|
||||
|
||||
### Immediate Benefits
|
||||
- Capture quality 90-94 LONG signals that were previously blocked
|
||||
- Expected: ~7 additional profitable longs per 227 trades (3.1% more trades)
|
||||
- Historical data suggests +$44.77 potential profit on these signals
|
||||
|
||||
### Risk Management
|
||||
- Quality 90-94 SHORT signals remain blocked (prevent -$553.76 losses)
|
||||
- Maintain strict quality requirements for toxic directions
|
||||
- No degradation in overall win rate expected
|
||||
|
||||
### Statistical Validation
|
||||
After 50-100 trades with new thresholds:
|
||||
- Compare quality 90-94 LONG performance to historical 71.4% WR
|
||||
- Verify SHORT blocking prevents losses (vs historical -$79.11 avg)
|
||||
- Adjust thresholds if data diverges from expectations
|
||||
|
||||
## Fallback Logic
|
||||
|
||||
**Threshold Selection Priority:**
|
||||
1. **Direction-specific ENV** (`MIN_SIGNAL_QUALITY_SCORE_LONG` or `_SHORT`)
|
||||
2. **Global ENV** (`MIN_SIGNAL_QUALITY_SCORE`)
|
||||
3. **Default** (60)
|
||||
|
||||
**Example:**
|
||||
- LONG signal → Uses 90 (direction-specific ENV)
|
||||
- SHORT signal → Uses 95 (direction-specific ENV)
|
||||
- If LONG ENV missing → Uses 91 (global ENV)
|
||||
- If all missing → Uses 60 (hardcoded default)
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
✅ **Fully backward compatible:**
|
||||
- Existing code without direction parameter continues working
|
||||
- Global threshold still available as fallback
|
||||
- Default value (60) remains unchanged
|
||||
- No breaking changes to API contracts
|
||||
|
||||
## Monitoring
|
||||
|
||||
**Key Metrics to Track:**
|
||||
- Quality 90-94 LONG win rate (expect 71.4% or better)
|
||||
- Quality 90-94 SHORT blocked count (prevent losses)
|
||||
- Overall P&L impact from additional long trades
|
||||
- False positive rate (quality 90-94 longs that lose)
|
||||
|
||||
**SQL Query for Monitoring:**
|
||||
```sql
|
||||
SELECT
|
||||
direction,
|
||||
COUNT(*) as trades,
|
||||
ROUND(AVG(CASE WHEN "realizedPnL" > 0 THEN 100.0 ELSE 0.0 END)::numeric, 1) as win_rate,
|
||||
ROUND(SUM("realizedPnL")::numeric, 2) as total_pnl,
|
||||
ROUND(AVG("realizedPnL")::numeric, 2) as avg_pnl
|
||||
FROM "Trade"
|
||||
WHERE "exitReason" IS NOT NULL
|
||||
AND "signalQualityScore" >= 90
|
||||
AND "signalQualityScore" < 95
|
||||
AND "createdAt" >= '2025-11-23' -- After implementation
|
||||
GROUP BY direction
|
||||
ORDER BY direction;
|
||||
```
|
||||
|
||||
## Configuration Management
|
||||
|
||||
**To adjust thresholds:**
|
||||
1. Edit `.env` file: `MIN_SIGNAL_QUALITY_SCORE_LONG=XX`
|
||||
2. Restart container: `docker compose down trading-bot && docker compose up -d trading-bot`
|
||||
3. Verify: `docker exec trading-bot-v4 printenv | grep MIN_SIGNAL`
|
||||
|
||||
**To revert to single threshold:**
|
||||
1. Remove `MIN_SIGNAL_QUALITY_SCORE_LONG` and `_SHORT` from .env
|
||||
2. Keep `MIN_SIGNAL_QUALITY_SCORE=91` (global)
|
||||
3. Restart container
|
||||
4. System falls back to global threshold for all directions
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### ❌ Pitfall #1: ENV not in docker-compose.yml
|
||||
**Symptom:** Container restarts but direction-specific thresholds not applied
|
||||
**Cause:** ENV variables must be declared in docker-compose.yml `environment` section
|
||||
**Fix:** Add to docker-compose.yml, then restart container
|
||||
|
||||
### ❌ Pitfall #2: Using `--force-recreate` instead of `down && up`
|
||||
**Symptom:** ENV changes not loaded after restart
|
||||
**Cause:** `--force-recreate` doesn't reload docker-compose.yml environment section
|
||||
**Fix:** Always use `docker compose down && docker compose up -d` for ENV changes
|
||||
|
||||
### ❌ Pitfall #3: Threshold exactly at boundary (score = threshold)
|
||||
**Symptom:** Score 90 signal blocked when threshold is 90
|
||||
**Cause:** Code uses `score >= minScore`, so 90 >= 90 should pass
|
||||
**Fix:** This was NOT the issue - verify ENV actually loaded with `printenv`
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
**Phase 1 (Current):** Static direction-specific thresholds based on historical data
|
||||
**Phase 2 (Future):** Dynamic thresholds based on rolling 50-trade performance
|
||||
**Phase 3 (Future):** ML-based direction prediction using quality score components
|
||||
**Phase 4 (Future):** Per-symbol direction preferences (SOL longs, ETH shorts, etc.)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Signal Quality Optimization: `SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md`
|
||||
- Quality Scoring Logic: `lib/trading/signal-quality.ts`
|
||||
- Check-Risk Endpoint: `app/api/trading/check-risk/route.ts`
|
||||
- Historical Analysis: Database query results (Nov 23, 2025)
|
||||
|
||||
## Git Commits
|
||||
|
||||
**Feature Implementation:**
|
||||
- **01aaa09** - "feat: Direction-specific quality thresholds (long=90, short=95)"
|
||||
- Modified config/trading.ts, lib/trading/signal-quality.ts, app/api/trading/check-risk/route.ts
|
||||
- Added ENV variables to .env file
|
||||
- Fallback logic and helper function
|
||||
|
||||
**Environment Fix:**
|
||||
- **357626b** - "fix: Add direction-specific quality thresholds to docker-compose.yml"
|
||||
- Added MIN_SIGNAL_QUALITY_SCORE_LONG/SHORT to environment section
|
||||
- Required for Node.js process.env access
|
||||
- Testing verified correct threshold application
|
||||
|
||||
## Container Deployment
|
||||
|
||||
**Build:** Nov 23, 2025 14:01 UTC (15:01 CET)
|
||||
**Restart:** Nov 23, 2025 14:13 UTC (15:13 CET) - with ENV fix
|
||||
**Status:** ✅ OPERATIONAL
|
||||
**Verification:** ENV variables present, thresholds applying correctly
|
||||
Reference in New Issue
Block a user