docs: Major documentation reorganization + ENV variable reference
**Documentation Structure:** - Created docs/ subdirectory organization (analysis/, architecture/, bugs/, cluster/, deployments/, roadmaps/, setup/, archived/) - Moved 68 root markdown files to appropriate categories - Root directory now clean (only README.md remains) - Total: 83 markdown files now organized by purpose **New Content:** - Added comprehensive Environment Variable Reference to copilot-instructions.md - 100+ ENV variables documented with types, defaults, purpose, notes - Organized by category: Required (Drift/RPC/Pyth), Trading Config (quality/ leverage/sizing), ATR System, Runner System, Risk Limits, Notifications, etc. - Includes usage examples (correct vs wrong patterns) **File Distribution:** - docs/analysis/ - Performance analyses, blocked signals, profit projections - docs/architecture/ - Adaptive leverage, ATR trailing, indicator tracking - docs/bugs/ - CRITICAL_*.md, FIXES_*.md bug reports (7 files) - docs/cluster/ - EPYC setup, distributed computing docs (3 files) - docs/deployments/ - *_COMPLETE.md, DEPLOYMENT_*.md status (12 files) - docs/roadmaps/ - All *ROADMAP*.md strategic planning files (7 files) - docs/setup/ - TradingView guides, signal quality, n8n setup (8 files) - docs/archived/2025_pre_nov/ - Obsolete verification checklist (1 file) **Key Improvements:** - ENV variable reference: Single source of truth for all configuration - Common Pitfalls #68-71: Already complete, verified during audit - Better findability: Category-based navigation vs 68 files in root - Preserves history: All files git mv (rename), not copy/delete - Zero broken functionality: Only documentation moved, no code changes **Verification:** - 83 markdown files now in docs/ subdirectories - Root directory cleaned: 68 files → 0 files (except README.md) - Git history preserved for all moved files - Container running: trading-bot-v4 (no restart needed) **Next Steps:** - Create README.md files in each docs subdirectory - Add navigation index - Update main README.md with new structure - Consolidate duplicate deployment docs - Archive truly obsolete files (old SQL backups) See: docs/analysis/CLEANUP_PLAN.md for complete reorganization strategy
This commit is contained in:
mindesbunister
380
docs/roadmaps/INDICATOR_V9_MA_GAP_ROADMAP.md
Normal file
380
docs/roadmaps/INDICATOR_V9_MA_GAP_ROADMAP.md
Normal file
@@ -0,0 +1,380 @@
|
||||
# Indicator v9: MA Gap Quality Enhancement
|
||||
|
||||
**Status:** 📋 PLANNED
|
||||
**Priority:** HIGH (addresses $380 missed profit from Nov 25 blocked signal)
|
||||
**Motivation:** v8 indicator catches trend changes BEFORE classic MA crossovers, but quality filter blocks these early signals
|
||||
|
||||
---
|
||||
|
||||
## 🎯 CRITICAL FINDING: MA Cross ADX Pattern (Nov 27, 2025)
|
||||
|
||||
**DATA-DRIVEN DISCOVERY:** v9 indicator signals arrive 35 minutes BEFORE actual MA crossover, with ADX strengthening DURING the cross event!
|
||||
|
||||
**Real Incident (Nov 27, 2025 - Death Cross Example):**
|
||||
|
||||
**Timeline (Berlin Time = UTC+1):**
|
||||
- **10:30** - 5-min v9 signal fires (EARLY warning)
|
||||
- ADX: 22.5 (weak, below 23 SHORT threshold)
|
||||
- Quality Score: 5 (blocked ❌)
|
||||
- Status: Signal arrives 35 minutes before actual cross
|
||||
|
||||
- **11:05** - Actual death cross happens on chart ✅
|
||||
- ADX: 29.5 (STRONG trend, crossed 23 threshold!)
|
||||
- 1-minute data progression shows strengthening:
|
||||
- 11:00: ADX 28.9 ✅
|
||||
- 11:01: ADX 29.2 ✅
|
||||
- 11:05: ADX 29.5 ✅ (exact cross time)
|
||||
- 11:06: ADX 29.8 ✅ (peak strength)
|
||||
- 11:08: ADX 26.8 ✅ (still strong)
|
||||
|
||||
**JACKPOT INSIGHT:**
|
||||
- Signal came **35 minutes EARLY** (10:30 vs 11:05) ✅
|
||||
- ADX was **weak BEFORE** the cross (22.5 at 10:30) ✅
|
||||
- ADX **strengthened DURING** the cross (22.5 → 29.5 in 35 minutes) ✅
|
||||
- **Pattern:** Early signal blocked by weak ADX, but ADX explodes at actual crossover!
|
||||
|
||||
**Current v9 System Status:**
|
||||
- ✅ v9 already includes MA Gap Analysis (+5 to +15 quality points)
|
||||
- ✅ v9 deployed Nov 26, 2025 with momentum-based SHORT filter
|
||||
- ✅ Current SHORT threshold: ADX ≥23 (would pass at 11:05 with ADX 29.5)
|
||||
- ✅ This finding validates v9's early detection capability!
|
||||
|
||||
**Validation Strategy:**
|
||||
- ✅ 1-minute data collection captures ADX progression in real-time
|
||||
- ✅ Smart Entry Timer + Phase 7.2 validation can catch strengthening trend
|
||||
- ✅ **n8n workflow updated (Nov 27, 2025):** MA crossover detection implemented
|
||||
- TradingView alert configured: "MA50&200 Crossing" triggers on bar close
|
||||
- Parse Signal Enhanced node now extracts: `isMACrossover`, `isDeathCross`, `isGoldenCross` flags
|
||||
- Bot will receive crossover events separately for data collection
|
||||
- 🎯 **Action:** Collect 5-10 more MA cross examples to validate pattern consistency
|
||||
- 🎯 **Goal:** Determine if weak → strong ADX pattern is consistent during crossovers
|
||||
|
||||
**User Quote:** *"This is EXACTLY what you were looking for!"*
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
**Historical Incident (Nov 25, 2025 21:15 UTC - Pre-v9):**
|
||||
- v8 generated LONG signal at $136.91 (quality 75, ADX 17.9)
|
||||
- Signal BLOCKED by quality threshold (75 < 90 required for LONGs)
|
||||
- Chart showed 50 MA converging toward 200 MA (gap ≈ -1% to 0%)
|
||||
- Golden cross occurred a few bars AFTER entry signal
|
||||
- Price moved to $142+ = **$380 missed profit** (~4% move)
|
||||
|
||||
**Key Insight:**
|
||||
- v9 indicator catches trend changes BEFORE classic MA crossovers
|
||||
- MA Gap Analysis (+5 to +15 points) now rewards convergence/proximity
|
||||
- v9 deployed Nov 26, 2025 with these enhancements already included
|
||||
- This roadmap tracks further optimizations based on data collection
|
||||
|
||||
---
|
||||
|
||||
## v9 Enhancement: MA Gap Analysis
|
||||
|
||||
### Core Concept
|
||||
|
||||
Instead of detecting exact crossover moment (lagging), measure **MA gap percentage**:
|
||||
- **Tight gap** (0-2%) = Strong trend with momentum or imminent crossover
|
||||
- **Converging gap** (-2% to 0%) = Potential golden cross brewing
|
||||
- **Wide gap** (>2%) = Established trend, less explosive but stable
|
||||
|
||||
### TradingView Indicator Changes
|
||||
|
||||
**Add after context metrics calculation (~line 221):**
|
||||
|
||||
```pinescript
|
||||
// ═══════════════════════════════════════════════════════════
|
||||
// 🎯 MA GAP ANALYSIS (v9 - for quality scoring)
|
||||
// ═══════════════════════════════════════════════════════════
|
||||
|
||||
// Calculate 50 and 200 period moving averages
|
||||
ma50 = ta.sma(calcC, 50)
|
||||
ma200 = ta.sma(calcC, 200)
|
||||
|
||||
// Calculate MA gap as percentage (negative = 50 below 200)
|
||||
maGap = ((ma50 - ma200) / ma200) * 100
|
||||
|
||||
// Detect convergence (MAs getting closer = potential crossover)
|
||||
maConverging = math.abs(maGap) < 2.0 // Within 2% = tight squeeze
|
||||
|
||||
// Current alignment
|
||||
maAlignmentBullish = ma50 > ma200
|
||||
|
||||
// Optional: Plot MAs on chart for visual confirmation
|
||||
plot(ma50, title="MA 50", color=color.yellow, linewidth=1)
|
||||
plot(ma200, title="MA 200", color=color.red, linewidth=1)
|
||||
```
|
||||
|
||||
**Update alert messages (~lines 257-258):**
|
||||
|
||||
```pinescript
|
||||
longAlertMsg = baseCurrency + " buy " + timeframe.period + " | ATR:" + str.tostring(atrPercent, "#.##") + " | ADX:" + str.tostring(adxVal, "#.#") + " | RSI:" + str.tostring(rsi14, "#.#") + " | VOL:" + str.tostring(volumeRatio, "#.##") + " | POS:" + str.tostring(pricePosition, "#.#") + " | MAGAP:" + str.tostring(maGap, "#.##") + " | IND:v9"
|
||||
|
||||
shortAlertMsg = baseCurrency + " sell " + timeframe.period + " | ATR:" + str.tostring(atrPercent, "#.##") + " | ADX:" + str.tostring(adxVal, "#.#") + " | RSI:" + str.tostring(rsi14, "#.#") + " | VOL:" + str.tostring(volumeRatio, "#.##") + " | POS:" + str.tostring(pricePosition, "#.#") + " | MAGAP:" + str.tostring(maGap, "#.##") + " | IND:v9"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backend Quality Scoring Changes
|
||||
|
||||
### Update scoreSignalQuality() function
|
||||
|
||||
**File:** `lib/trading/signal-quality.ts`
|
||||
|
||||
**Add parameters:**
|
||||
```typescript
|
||||
export async function scoreSignalQuality(params: {
|
||||
// ... existing params ...
|
||||
maGap?: number // NEW: % gap between 50 and 200 MA
|
||||
maAlignmentBullish?: boolean // NEW: is 50 above 200?
|
||||
}): Promise<SignalQualityResult> {
|
||||
```
|
||||
|
||||
**Add scoring logic (after existing metrics):**
|
||||
|
||||
```typescript
|
||||
// ═══════════════════════════════════════════════════════════
|
||||
// MA Gap Analysis (v9 - Nov 26, 2025)
|
||||
// ═══════════════════════════════════════════════════════════
|
||||
|
||||
if (params.maGap !== undefined) {
|
||||
if (params.direction === 'long') {
|
||||
// LONG scenarios
|
||||
if (params.maGap >= 0 && params.maGap < 2.0) {
|
||||
// 50 MA above 200 MA, tight gap (0-2%)
|
||||
// = Bullish trend with momentum OR fresh golden cross
|
||||
score += 15
|
||||
reasons.push(`🎯 MA bullish + tight gap (${params.maGap.toFixed(2)}%) = strong momentum (+15 pts)`)
|
||||
|
||||
} else if (params.maGap < 0 && params.maGap > -2.0) {
|
||||
// 50 MA below 200 MA but converging (-2% to 0%)
|
||||
// = Potential golden cross brewing (early detection like Nov 25 signal!)
|
||||
score += 12
|
||||
reasons.push(`🌟 MA converging (${params.maGap.toFixed(2)}%) = golden cross potential (+12 pts)`)
|
||||
|
||||
} else if (params.maGap >= 2.0 && params.maGap < 5.0) {
|
||||
// 50 MA well above 200 MA (2-5%)
|
||||
// = Established bullish trend, stable but less explosive
|
||||
score += 8
|
||||
reasons.push(`📈 MA strong bullish trend (${params.maGap.toFixed(2)}%) (+8 pts)`)
|
||||
|
||||
} else if (params.maGap >= 5.0) {
|
||||
// 50 MA far above 200 MA (>5%)
|
||||
// = Very extended, potential exhaustion
|
||||
score += 5
|
||||
reasons.push(`⚠️ MA extended bullish (${params.maGap.toFixed(2)}%) = overbought risk (+5 pts)`)
|
||||
|
||||
} else if (params.maGap <= -2.0) {
|
||||
// 50 MA significantly below 200 MA
|
||||
// = Bearish trend, LONG signal is counter-trend (risky)
|
||||
score -= 10
|
||||
reasons.push(`❌ MA bearish divergence (${params.maGap.toFixed(2)}%) = counter-trend risk (-10 pts)`)
|
||||
}
|
||||
|
||||
} else if (params.direction === 'short') {
|
||||
// SHORT scenarios (inverse of LONG logic)
|
||||
if (params.maGap <= 0 && params.maGap > -2.0) {
|
||||
// 50 MA below 200 MA, tight gap (-2% to 0%)
|
||||
// = Bearish trend with momentum OR fresh death cross
|
||||
score += 15
|
||||
reasons.push(`🎯 MA bearish + tight gap (${params.maGap.toFixed(2)}%) = strong momentum (+15 pts)`)
|
||||
|
||||
} else if (params.maGap > 0 && params.maGap < 2.0) {
|
||||
// 50 MA above 200 MA but converging (0% to 2%)
|
||||
// = Potential death cross brewing
|
||||
score += 12
|
||||
reasons.push(`🌟 MA converging (${params.maGap.toFixed(2)}%) = death cross potential (+12 pts)`)
|
||||
|
||||
} else if (params.maGap <= -2.0 && params.maGap > -5.0) {
|
||||
// 50 MA well below 200 MA (-5% to -2%)
|
||||
// = Established bearish trend
|
||||
score += 8
|
||||
reasons.push(`📉 MA strong bearish trend (${params.maGap.toFixed(2)}%) (+8 pts)`)
|
||||
|
||||
} else if (params.maGap <= -5.0) {
|
||||
// 50 MA far below 200 MA (<-5%)
|
||||
// = Very extended, potential bounce risk
|
||||
score += 5
|
||||
reasons.push(`⚠️ MA extended bearish (${params.maGap.toFixed(2)}%) = oversold risk (+5 pts)`)
|
||||
|
||||
} else if (params.maGap >= 2.0) {
|
||||
// 50 MA significantly above 200 MA
|
||||
// = Bullish trend, SHORT signal is counter-trend (risky)
|
||||
score -= 10
|
||||
reasons.push(`❌ MA bullish divergence (${params.maGap.toFixed(2)}%) = counter-trend risk (-10 pts)`)
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected Impact
|
||||
|
||||
### Nov 25 21:15 Signal Reanalysis
|
||||
|
||||
**Original (v8):**
|
||||
- Quality: 75 (blocked)
|
||||
- ADX: 17.9 (weak)
|
||||
- MA Gap: ≈ -1.0% (50 below 200, converging)
|
||||
|
||||
**With v9 MA Gap Enhancement:**
|
||||
- Base quality: 75
|
||||
- MA converging bonus: +12
|
||||
- **New quality: 87** (closer but still blocked)
|
||||
|
||||
**Note:** Would still need ADX improvement OR slightly lower threshold (88-89?) to catch this specific signal. But the +12 points get us much closer and would catch signals with ADX 18-20.
|
||||
|
||||
### Alternative Scenarios
|
||||
|
||||
**Scenario A: MA Gap -0.5% (very tight convergence)**
|
||||
- Quality 75 + 12 = **87** (close)
|
||||
|
||||
**Scenario B: MA Gap +0.5% (just crossed, tight)**
|
||||
- Quality 75 + 15 = **90** ✅ **PASSES!**
|
||||
|
||||
**Scenario C: MA Gap +1.8% (recent golden cross, momentum strong)**
|
||||
- Quality 75 + 15 = **90** ✅ **PASSES!**
|
||||
|
||||
---
|
||||
|
||||
## Implementation Checklist
|
||||
|
||||
### Phase 1: TradingView Indicator
|
||||
- [ ] Add MA50 and MA200 calculations
|
||||
- [ ] Calculate maGap percentage
|
||||
- [ ] Add maGap to alert message payload
|
||||
- [ ] Optional: Plot MA lines on chart
|
||||
- [ ] Update indicatorVer from "v8" to "v9"
|
||||
- [ ] Test on SOL-PERP 5min chart
|
||||
|
||||
### Phase 2: Backend Integration
|
||||
- [ ] Update TypeScript interfaces for maGap parameter
|
||||
- [ ] Modify scoreSignalQuality() with MA gap logic
|
||||
- [ ] Update check-risk endpoint to accept maGap
|
||||
- [ ] Update execute endpoint to accept maGap
|
||||
- [ ] Add maGap to database fields (Trade table, BlockedSignal table)
|
||||
|
||||
### Phase 3: Testing & Validation
|
||||
- [ ] Deploy v9 indicator to TradingView
|
||||
- [ ] Trigger test signals manually
|
||||
- [ ] Verify maGap calculation matches chart visual
|
||||
- [ ] Check quality scores increase appropriately
|
||||
- [ ] Monitor first 20-30 signals for validation
|
||||
|
||||
### Phase 4: Data Collection
|
||||
- [ ] Collect 50+ v9 signals
|
||||
- [ ] Compare v8 vs v9 win rates
|
||||
- [ ] Analyze: Did MA gap bonus catch missed winners?
|
||||
- [ ] SQL queries to validate improvement
|
||||
- [ ] Adjust bonus points if needed (12/15 → 10/12 or 15/18)
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
**Target Improvements:**
|
||||
1. **Catch signals like Nov 25 21:15:** Quality 75 + MA converging → 87-90 range
|
||||
2. **Reduce false negatives:** Fewer blocked signals that would have been winners
|
||||
3. **Maintain safety:** Don't add too many low-quality signals
|
||||
4. **Win rate:** v9 should maintain or improve v8's 57.1% WR
|
||||
|
||||
**Validation Queries:**
|
||||
|
||||
```sql
|
||||
-- Compare v9 MA gap bonus impact
|
||||
SELECT
|
||||
CASE
|
||||
WHEN "signalQualityScore" >= 90 THEN 'Passed'
|
||||
WHEN "signalQualityScore" >= 80 THEN 'Close (80-89)'
|
||||
ELSE 'Blocked (<80)'
|
||||
END as category,
|
||||
COUNT(*) as signals,
|
||||
AVG("scoreBreakdown"->>'maGap')::numeric as avg_ma_gap
|
||||
FROM "BlockedSignal"
|
||||
WHERE "indicatorVersion" = 'v9'
|
||||
AND "scoreBreakdown"->>'maGap' IS NOT NULL
|
||||
GROUP BY category;
|
||||
|
||||
-- Find signals that would pass with v9 but were blocked in v8
|
||||
SELECT
|
||||
TO_CHAR("createdAt", 'MM-DD HH24:MI') as time,
|
||||
symbol, direction,
|
||||
"signalQualityScore" as original_score,
|
||||
-- Simulate v9 score (add 12 for converging, 15 for tight)
|
||||
CASE
|
||||
WHEN ("scoreBreakdown"->>'maGap')::numeric >= 0 AND ("scoreBreakdown"->>'maGap')::numeric < 2.0
|
||||
THEN "signalQualityScore" + 15
|
||||
WHEN ("scoreBreakdown"->>'maGap')::numeric < 0 AND ("scoreBreakdown"->>'maGap')::numeric > -2.0
|
||||
THEN "signalQualityScore" + 12
|
||||
ELSE "signalQualityScore"
|
||||
END as v9_score,
|
||||
"blockReason"
|
||||
FROM "BlockedSignal"
|
||||
WHERE "signalQualityScore" < 90 -- Was blocked
|
||||
AND "indicatorVersion" = 'v8'
|
||||
ORDER BY "createdAt" DESC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
### Potential Issues
|
||||
|
||||
1. **MA calculation lag:** 200-period MA requires significant history
|
||||
- **Mitigation:** TradingView has full history, no issue
|
||||
|
||||
2. **Whipsaw during sideways markets:** MAs converge often in chop
|
||||
- **Mitigation:** ADX filter still applies (weak ADX = less bonus effect)
|
||||
|
||||
3. **Over-optimization on single signal:** Nov 25 may be outlier
|
||||
- **Mitigation:** Collect 50+ v9 signals before final judgment
|
||||
|
||||
4. **Bonus points too generous:** Could inflate scores artificially
|
||||
- **Mitigation:** Start conservative (12/15), adjust based on data
|
||||
|
||||
### Rollback Plan
|
||||
|
||||
If v9 performs worse than v8:
|
||||
1. Revert TradingView indicator to v8
|
||||
2. Keep backend code but disable MA gap bonus
|
||||
3. Analyze what went wrong (false positives? whipsaw signals?)
|
||||
4. Redesign MA gap logic with tighter conditions
|
||||
|
||||
---
|
||||
|
||||
## Timeline
|
||||
|
||||
**Estimated Implementation Time:**
|
||||
- TradingView changes: 30 minutes
|
||||
- Backend integration: 1 hour
|
||||
- Testing & deployment: 30 minutes
|
||||
- **Total: ~2 hours**
|
||||
|
||||
**Data Collection:**
|
||||
- Minimum 50 signals: 2-3 weeks (at ~3-5 signals/day)
|
||||
- Comparative analysis: 1 week after 50 signals
|
||||
|
||||
**Decision Point:**
|
||||
- After 50 v9 signals: Keep, adjust, or revert based on performance data
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- This enhancement preserves v8's early detection advantage
|
||||
- Adds context awareness of MA positioning
|
||||
- Rewards both imminent crossovers (converging) AND fresh crossovers (tight gap)
|
||||
- Balances explosive potential (tight gaps) with trend stability (wider gaps)
|
||||
- Counter-trend penalties prevent chasing wrong direction
|
||||
|
||||
**Key Insight:** v8 catches momentum shifts BEFORE visible MA crossovers. v9 validates those shifts by checking if MA structure supports the move.
|
||||
|
||||
---
|
||||
|
||||
**Created:** Nov 26, 2025
|
||||
**Motivation:** $380 missed profit from Nov 25 21:15 blocked signal
|
||||
**Expected Impact:** Catch 15-25% more profitable signals while maintaining quality standards
|
||||
Reference in New Issue
Block a user