docs: Add comprehensive documentation for direction-specific quality thresholds

- Complete implementation guide with data-driven rationale
- Testing results and verification steps
- Deployment checklist and common pitfalls
- Monitoring queries and configuration management
- Future enhancement roadmap
This commit is contained in:
mindesbunister
2025-11-23 15:07:19 +01:00
parent 357626b392
commit 9580c109b2

View File

@@ -0,0 +1,313 @@
# Direction-Specific Quality Thresholds
**Implementation Date:** November 23, 2025
**Status:** ✅ DEPLOYED and TESTED
**Commits:** 01aaa09, 357626b
## Overview
Trading bot now uses different signal quality thresholds based on trade direction (LONG vs SHORT) to capture profitable setups while blocking toxic ones.
## Data-Driven Decision
### Historical Analysis (227 completed trades)
**Quality 90-94 Performance:**
- **LONGS:** 7 trades, **71.4% WR**, +$44.77 total (+$6.40 avg per trade)
- **SHORTS:** 7 trades, **28.6% WR**, -$553.76 total (-$79.11 avg per trade)
- **Difference:** $598.53 P&L gap between same quality level
**Quality 90+ Overall (All History):**
- **LONGS:** 38 trades, 50.0% WR, **+$600.62** total
- **SHORTS:** 38 trades, 47.4% WR, **-$177.90** total
- **Total difference:** $778.52 (longs vastly outperform)
**v8 Indicator Directional Performance:**
- **LONGS:** 3 trades, **100% WR**, +$565.03 (avg +$188.34)
- **SHORTS:** 7 trades, 42.9% WR, -$311.68 (avg -$44.53)
### User Decision
**User Query:** "are longs more profitable as shorts? what does our data say? should we maybe enable normal entries at 90 quality score long signals?"
**Agent Analysis:** Data shows clear directional edge - longs at quality 90-94 profitable (71.4% WR), shorts toxic (28.6% WR).
**User Approval:** "yes. go"
## Implementation
### Configuration
**New ENV Variables (.env):**
```bash
MIN_SIGNAL_QUALITY_SCORE=91 # Global fallback (when no direction-specific set)
MIN_SIGNAL_QUALITY_SCORE_LONG=90 # Longs: 71.4% WR at 90-94
MIN_SIGNAL_QUALITY_SCORE_SHORT=95 # Shorts: Block toxic 90-94 range
```
**Docker Compose (docker-compose.yml):**
```yaml
environment:
MIN_SIGNAL_QUALITY_SCORE: ${MIN_SIGNAL_QUALITY_SCORE:-91}
MIN_SIGNAL_QUALITY_SCORE_LONG: ${MIN_SIGNAL_QUALITY_SCORE_LONG:-90}
MIN_SIGNAL_QUALITY_SCORE_SHORT: ${MIN_SIGNAL_QUALITY_SCORE_SHORT:-95}
```
### Code Changes
**1. Trading Config Interface (config/trading.ts):**
```typescript
export interface TradingConfig {
// ... existing fields
minSignalQualityScoreLong?: number // Direction-specific threshold for longs
minSignalQualityScoreShort?: number // Direction-specific threshold for shorts
}
export const DEFAULT_TRADING_CONFIG: TradingConfig = {
// ... existing defaults
minSignalQualityScoreLong: 90, // Data-driven: 71.4% WR at 90-94
minSignalQualityScoreShort: 95, // Data-driven: Block toxic 90-94 shorts
}
```
**2. Helper Function (config/trading.ts):**
```typescript
/**
* Get minimum quality score based on trade direction
* Nov 23, 2025: Data shows longs profitable at 90-94 (71.4% WR), shorts toxic (28.6% WR)
*/
export function getMinQualityScoreForDirection(
direction: 'long' | 'short',
config: TradingConfig
): number {
// Direction-specific threshold if set
if (direction === 'long' && config.minSignalQualityScoreLong !== undefined) {
return config.minSignalQualityScoreLong
}
if (direction === 'short' && config.minSignalQualityScoreShort !== undefined) {
return config.minSignalQualityScoreShort
}
// Fallback: global → 60 default
return config.minSignalQualityScore ?? 60
}
```
**3. Check-Risk Endpoint (app/api/trading/check-risk/route.ts):**
```typescript
// Use direction-specific quality threshold (Nov 23, 2025)
const minQualityScore = getMinQualityScoreForDirection(body.direction, config)
const qualityScore = await scoreSignalQuality({
atr: body.atr || 0,
adx: body.adx || 0,
rsi: body.rsi || 0,
volumeRatio: body.volumeRatio || 0,
pricePosition: body.pricePosition || 0,
direction: body.direction,
symbol: body.symbol,
currentPrice: currentPrice,
timeframe: body.timeframe,
minScore: minQualityScore // Direction-specific threshold
})
if (!qualityScore.passed) {
console.log('🚫 Risk check BLOCKED: Signal quality too low', {
score: qualityScore.score,
direction: body.direction,
threshold: minQualityScore, // Logs show 90 for longs, 95 for shorts
reasons: qualityScore.reasons
})
}
```
**4. Signal Quality Scoring (lib/trading/signal-quality.ts):**
```typescript
export async function scoreSignalQuality(params: {
atr: number
adx: number
rsi: number
volumeRatio: number
pricePosition: number
direction: 'long' | 'short'
symbol: string
currentPrice?: number
timeframe?: string
minScore?: number // Direction-specific threshold passed in
}): Promise<SignalQualityResult> {
// ... scoring logic
const minScore = params.minScore || 60 // Use provided threshold or fallback
const passed = score >= minScore
return { score, passed, reasons }
}
```
## Deployment Steps
1. ✅ Modified 3 code files (config, signal-quality, check-risk)
2. ✅ Added ENV variables to .env file
3. ✅ Added ENV variables to docker-compose.yml (required for process.env access)
4. ✅ Built Docker container (71.8s build time)
5. ✅ Restarted container with `docker compose down && docker compose up -d`
6. ✅ Verified ENV variables loaded: `docker exec trading-bot-v4 printenv | grep MIN_SIGNAL`
7. ✅ Tested with curl: LONG quality 90 ✅ ALLOWED, SHORT quality 70 ❌ BLOCKED
## Testing Results
**Test 1: Quality 90 LONG (should PASS)**
```json
{
"allowed": true,
"details": "All risk checks passed",
"qualityScore": 90,
"qualityReasons": [
"ATR healthy (0.43%)",
"Strong trend for 5min (ADX 22.5)",
"RSI supports long (58.0)",
"Price position OK (45%)"
]
}
```
**PASSED** - Threshold 90 correctly applied
**Test 2: Quality 70 SHORT (should BLOCK)**
```json
{
"allowed": false,
"reason": "Signal quality too low",
"details": "Score: 70/100",
"qualityScore": 70
}
```
**BLOCKED** - Threshold 95 correctly applied (logs showed `threshold: 95`)
## Expected Impact
### Immediate Benefits
- Capture quality 90-94 LONG signals that were previously blocked
- Expected: ~7 additional profitable longs per 227 trades (3.1% more trades)
- Historical data suggests +$44.77 potential profit on these signals
### Risk Management
- Quality 90-94 SHORT signals remain blocked (prevent -$553.76 losses)
- Maintain strict quality requirements for toxic directions
- No degradation in overall win rate expected
### Statistical Validation
After 50-100 trades with new thresholds:
- Compare quality 90-94 LONG performance to historical 71.4% WR
- Verify SHORT blocking prevents losses (vs historical -$79.11 avg)
- Adjust thresholds if data diverges from expectations
## Fallback Logic
**Threshold Selection Priority:**
1. **Direction-specific ENV** (`MIN_SIGNAL_QUALITY_SCORE_LONG` or `_SHORT`)
2. **Global ENV** (`MIN_SIGNAL_QUALITY_SCORE`)
3. **Default** (60)
**Example:**
- LONG signal → Uses 90 (direction-specific ENV)
- SHORT signal → Uses 95 (direction-specific ENV)
- If LONG ENV missing → Uses 91 (global ENV)
- If all missing → Uses 60 (hardcoded default)
## Backward Compatibility
**Fully backward compatible:**
- Existing code without direction parameter continues working
- Global threshold still available as fallback
- Default value (60) remains unchanged
- No breaking changes to API contracts
## Monitoring
**Key Metrics to Track:**
- Quality 90-94 LONG win rate (expect 71.4% or better)
- Quality 90-94 SHORT blocked count (prevent losses)
- Overall P&L impact from additional long trades
- False positive rate (quality 90-94 longs that lose)
**SQL Query for Monitoring:**
```sql
SELECT
direction,
COUNT(*) as trades,
ROUND(AVG(CASE WHEN "realizedPnL" > 0 THEN 100.0 ELSE 0.0 END)::numeric, 1) as win_rate,
ROUND(SUM("realizedPnL")::numeric, 2) as total_pnl,
ROUND(AVG("realizedPnL")::numeric, 2) as avg_pnl
FROM "Trade"
WHERE "exitReason" IS NOT NULL
AND "signalQualityScore" >= 90
AND "signalQualityScore" < 95
AND "createdAt" >= '2025-11-23' -- After implementation
GROUP BY direction
ORDER BY direction;
```
## Configuration Management
**To adjust thresholds:**
1. Edit `.env` file: `MIN_SIGNAL_QUALITY_SCORE_LONG=XX`
2. Restart container: `docker compose down trading-bot && docker compose up -d trading-bot`
3. Verify: `docker exec trading-bot-v4 printenv | grep MIN_SIGNAL`
**To revert to single threshold:**
1. Remove `MIN_SIGNAL_QUALITY_SCORE_LONG` and `_SHORT` from .env
2. Keep `MIN_SIGNAL_QUALITY_SCORE=91` (global)
3. Restart container
4. System falls back to global threshold for all directions
## Common Pitfalls
### ❌ Pitfall #1: ENV not in docker-compose.yml
**Symptom:** Container restarts but direction-specific thresholds not applied
**Cause:** ENV variables must be declared in docker-compose.yml `environment` section
**Fix:** Add to docker-compose.yml, then restart container
### ❌ Pitfall #2: Using `--force-recreate` instead of `down && up`
**Symptom:** ENV changes not loaded after restart
**Cause:** `--force-recreate` doesn't reload docker-compose.yml environment section
**Fix:** Always use `docker compose down && docker compose up -d` for ENV changes
### ❌ Pitfall #3: Threshold exactly at boundary (score = threshold)
**Symptom:** Score 90 signal blocked when threshold is 90
**Cause:** Code uses `score >= minScore`, so 90 >= 90 should pass
**Fix:** This was NOT the issue - verify ENV actually loaded with `printenv`
## Future Enhancements
**Phase 1 (Current):** Static direction-specific thresholds based on historical data
**Phase 2 (Future):** Dynamic thresholds based on rolling 50-trade performance
**Phase 3 (Future):** ML-based direction prediction using quality score components
**Phase 4 (Future):** Per-symbol direction preferences (SOL longs, ETH shorts, etc.)
## Related Documentation
- Signal Quality Optimization: `SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md`
- Quality Scoring Logic: `lib/trading/signal-quality.ts`
- Check-Risk Endpoint: `app/api/trading/check-risk/route.ts`
- Historical Analysis: Database query results (Nov 23, 2025)
## Git Commits
**Feature Implementation:**
- **01aaa09** - "feat: Direction-specific quality thresholds (long=90, short=95)"
- Modified config/trading.ts, lib/trading/signal-quality.ts, app/api/trading/check-risk/route.ts
- Added ENV variables to .env file
- Fallback logic and helper function
**Environment Fix:**
- **357626b** - "fix: Add direction-specific quality thresholds to docker-compose.yml"
- Added MIN_SIGNAL_QUALITY_SCORE_LONG/SHORT to environment section
- Required for Node.js process.env access
- Testing verified correct threshold application
## Container Deployment
**Build:** Nov 23, 2025 14:01 UTC (15:01 CET)
**Restart:** Nov 23, 2025 14:13 UTC (15:13 CET) - with ENV fix
**Status:** ✅ OPERATIONAL
**Verification:** ENV variables present, thresholds applying correctly