docs: Add 1-minute simplified price feed to reduce TradingView alert queue pressure
- Create moneyline_1min_price_feed.pinescript (70% smaller payload) - Remove ATR/ADX/RSI/VOL/POS from 1-minute alerts (not used for decisions) - Keep only price + symbol + timeframe for market data cache - Document rationale in docs/1MIN_SIMPLIFIED_FEED.md - Fix: 5-minute trading signals being dropped due to 1-minute flood (60/hour) - Impact: Preserve priority for actual trading signals
This commit is contained in:
mindesbunister
Binary file not shown.
157
docs/1MIN_SIMPLIFIED_FEED.md
Normal file
157
docs/1MIN_SIMPLIFIED_FEED.md
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
# 1-Minute Simplified Price Feed (Dec 4, 2025)
|
||||||
|
|
||||||
|
## Problem Statement
|
||||||
|
|
||||||
|
**Issue:** TradingView alert queue getting overwhelmed by 1-minute signal flood
|
||||||
|
- **Volume:** 60 alerts/hour (one per minute) from 1-minute data feed
|
||||||
|
- **Impact:** 5-minute trading signals occasionally skipped (happened 2× so far)
|
||||||
|
- **Root cause:** 1-minute alerts sending full trading signal format with all indicators
|
||||||
|
|
||||||
|
## Solution: Lightweight Price-Only Feed
|
||||||
|
|
||||||
|
**Change:** Reduce 1-minute alert payload by ~70%
|
||||||
|
|
||||||
|
### Before (Full Format)
|
||||||
|
```
|
||||||
|
SOLUSDT buy 1 @ 142.08 | ATR:0.65 | ADX:14.3 | RSI:51.3 | VOL:0.87 | POS:59.3 | IND:v9
|
||||||
|
```
|
||||||
|
**Size:** ~80 characters
|
||||||
|
**Processing:** n8n parses 7 fields, bot calculates quality score, saves to BlockedSignal + MarketData
|
||||||
|
|
||||||
|
### After (Minimal Format)
|
||||||
|
```
|
||||||
|
SOLUSDT buy 1 @ 142.08
|
||||||
|
```
|
||||||
|
**Size:** ~22 characters (~73% reduction)
|
||||||
|
**Processing:** n8n parses 3 fields (symbol/timeframe/price), bot routes to BlockedSignal only
|
||||||
|
|
||||||
|
## Why This Works
|
||||||
|
|
||||||
|
### What We Actually Need from 1-Minute Data
|
||||||
|
1. ✅ **Price updates** - For market data cache (5-minute expiry)
|
||||||
|
2. ✅ **Symbol identification** - Routing to correct market
|
||||||
|
3. ✅ **Timeframe marker** - `timeframe="1"` triggers data collection path
|
||||||
|
4. ❌ **ATR/ADX/RSI/VOL/POS** - NOT used in real-time (recalculated from DB if needed)
|
||||||
|
5. ❌ **Direction** - Meaningless for data-only signals (see copilot-instructions.md)
|
||||||
|
|
||||||
|
### What We Already Have
|
||||||
|
- **5-minute signals** include full indicator context for trading decisions
|
||||||
|
- **BlockedSignalTracker** recalculates TP1/TP2/SL from database data (not live indicators)
|
||||||
|
- **Market data cache** only needs `currentPrice` + `timestamp` for validation (5-minute expiry)
|
||||||
|
- **MarketData table** can store NULL for indicators (not critical for analysis)
|
||||||
|
|
||||||
|
### What Gets Removed
|
||||||
|
```typescript
|
||||||
|
// OLD: Execute endpoint saves full metrics to MarketData table
|
||||||
|
await prisma.marketData.create({
|
||||||
|
data: {
|
||||||
|
atr: Number(body.atr) || 0, // ❌ Remove
|
||||||
|
adx: Number(body.adx) || 0, // ❌ Remove
|
||||||
|
rsi: Number(body.rsi) || 50, // ❌ Remove
|
||||||
|
volumeRatio: Number(body.volumeRatio) || 1.0, // ❌ Remove
|
||||||
|
pricePosition: Number(body.pricePosition) || 50, // ❌ Remove
|
||||||
|
maGap: Number(body.maGap) || undefined, // ❌ Remove
|
||||||
|
|
||||||
|
// Keep essentials
|
||||||
|
symbol: driftSymbol, // ✅ Keep
|
||||||
|
timeframe: '1', // ✅ Keep
|
||||||
|
price: currentPrice, // ✅ Keep
|
||||||
|
timestamp: new Date() // ✅ Keep (auto-generated)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## Implementation Plan
|
||||||
|
|
||||||
|
### Phase 1: Update TradingView Alert (IMMEDIATE)
|
||||||
|
1. Replace `moneyline_1min_data_feed.pinescript` with `moneyline_1min_price_feed.pinescript`
|
||||||
|
2. Update TradingView alert to use new script
|
||||||
|
3. Test: Verify alerts still route to BlockedSignal correctly
|
||||||
|
|
||||||
|
### Phase 2: Update Execute Endpoint (OPTIONAL)
|
||||||
|
Since we're only sending price, make execute endpoint resilient to missing fields:
|
||||||
|
```typescript
|
||||||
|
// Lines 193-207: MarketData.create()
|
||||||
|
// Change required fields to optional with safe defaults
|
||||||
|
atr: Number(body.atr) || null, // NULL if missing
|
||||||
|
adx: Number(body.adx) || null,
|
||||||
|
rsi: Number(body.rsi) || null,
|
||||||
|
// ... etc
|
||||||
|
```
|
||||||
|
|
||||||
|
### Phase 3: Monitor Results
|
||||||
|
- Track 5-minute signal reliability (should go from 98% → 100%)
|
||||||
|
- Verify 1-minute data still saves to BlockedSignal
|
||||||
|
- Check TradingView alert queue no longer shows congestion
|
||||||
|
|
||||||
|
## Expected Impact
|
||||||
|
|
||||||
|
### Benefits
|
||||||
|
1. **70% reduction** in TradingView alert payload size
|
||||||
|
2. **Priority preservation** - 5-minute trading signals less likely to be dropped
|
||||||
|
3. **Faster processing** - Less n8n parsing, less bot processing
|
||||||
|
4. **Same functionality** - No loss of critical features
|
||||||
|
|
||||||
|
### Tradeoffs
|
||||||
|
1. MarketData table will have NULL indicators (acceptable - not used for decisions)
|
||||||
|
2. Historical 1-minute indicator data less complete (acceptable - we have 5-minute full data)
|
||||||
|
|
||||||
|
### No Impact On
|
||||||
|
- ✅ 5-minute trading signals (still full format with all indicators)
|
||||||
|
- ✅ BlockedSignalTracker (calculates from price movements, not live indicators)
|
||||||
|
- ✅ Market data cache (only needs price + timestamp)
|
||||||
|
- ✅ Smart Entry Validation (uses 5-minute cached data, not 1-minute)
|
||||||
|
|
||||||
|
## Files Changed
|
||||||
|
|
||||||
|
### TradingView Pine Script
|
||||||
|
- **NEW:** `workflows/trading/moneyline_1min_price_feed.pinescript` - Simplified version
|
||||||
|
- **OLD:** `workflows/trading/moneyline_1min_data_feed.pinescript` - Full format (archived)
|
||||||
|
|
||||||
|
### n8n Workflow
|
||||||
|
- **UNCHANGED:** `workflows/trading/parse_signal_enhanced.json` - Already handles minimal format
|
||||||
|
- Extracts: `symbol`, `timeframe`, `signalPrice` (from `@ price` pattern)
|
||||||
|
- Defaults: ATR/ADX/RSI/etc to 0 if missing
|
||||||
|
|
||||||
|
### Bot Code
|
||||||
|
- **UNCHANGED:** `app/api/trading/execute/route.ts` - Already handles missing fields with fallbacks
|
||||||
|
- Lines 193-207: Uses `Number(body.field) || default` pattern
|
||||||
|
|
||||||
|
## Migration Steps
|
||||||
|
|
||||||
|
1. ✅ **Read this document** - Understand rationale
|
||||||
|
2. ⏳ **Update TradingView alert** - Replace with `moneyline_1min_price_feed.pinescript`
|
||||||
|
3. ⏳ **Test with 1-2 alerts** - Verify BlockedSignal still populates
|
||||||
|
4. ⏳ **Monitor for 24 hours** - Check 5-minute signal reliability
|
||||||
|
5. ⏳ **Archive old script** - Move `moneyline_1min_data_feed.pinescript` to archive/
|
||||||
|
|
||||||
|
## Rollback Plan
|
||||||
|
|
||||||
|
If issues arise:
|
||||||
|
1. Switch TradingView alert back to `moneyline_1min_data_feed.pinescript`
|
||||||
|
2. No code changes needed - bot handles both formats
|
||||||
|
|
||||||
|
## Decision Criteria
|
||||||
|
|
||||||
|
**Use simplified feed IF:**
|
||||||
|
- 5-minute signal drops are occurring regularly (2+ per week)
|
||||||
|
- TradingView alert queue shows congestion
|
||||||
|
- 1-minute indicator data not being used for analysis
|
||||||
|
|
||||||
|
**Keep full feed IF:**
|
||||||
|
- 5-minute signals are 100% reliable
|
||||||
|
- Future analysis requires minute-by-minute indicator tracking
|
||||||
|
- No performance issues observed
|
||||||
|
|
||||||
|
## Current Status (Dec 4, 2025)
|
||||||
|
|
||||||
|
- ✅ Simplified script created
|
||||||
|
- ⏳ TradingView alert update pending
|
||||||
|
- ⏳ Monitoring results pending
|
||||||
|
- **User decision:** Test simplified feed for 24-48 hours
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Copilot Instructions: Section "1-Minute Data Collection System"
|
||||||
|
- Original Implementation: `docs/1MIN_DATA_COLLECTION_SIMPLE.md`
|
||||||
|
- BlockedSignalTracker: `lib/analysis/blocked-signal-tracker.ts`
|
||||||
161
docs/README.md
Normal file
161
docs/README.md
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
# Trading Bot v4 Documentation
|
||||||
|
|
||||||
|
**Comprehensive documentation for the Trading Bot v4 autonomous trading system.**
|
||||||
|
|
||||||
|
Last Updated: December 4, 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📂 Documentation Structure
|
||||||
|
|
||||||
|
### **[analysis/](analysis/)** - Performance Analysis & Optimization Studies
|
||||||
|
Performance analyses, blocked signals tracking, profit projections, and diagnostic results. Data-driven insights for system optimization.
|
||||||
|
|
||||||
|
### **[architecture/](architecture/)** - System Architecture & Design
|
||||||
|
Adaptive leverage system, ATR trailing stops, indicator version tracking, and position synchronization. Core technical design documentation.
|
||||||
|
|
||||||
|
### **[bugs/](bugs/)** - Bug Reports & Critical Fixes
|
||||||
|
CRITICAL_*.md and FIXES_*.md bug reports. Historical record of critical incidents and their resolutions.
|
||||||
|
|
||||||
|
### **[cluster/](cluster/)** - Distributed Computing & EPYC Cluster
|
||||||
|
EPYC cluster setup, distributed backtesting coordination, dual sweep configurations. Multi-server infrastructure documentation.
|
||||||
|
|
||||||
|
### **[deployments/](deployments/)** - Deployment Status & Verification
|
||||||
|
*_COMPLETE.md and DEPLOYMENT_*.md status files. Verification reports for major feature deployments.
|
||||||
|
|
||||||
|
### **[roadmaps/](roadmaps/)** - Strategic Planning & Roadmaps
|
||||||
|
All *ROADMAP*.md files. Long-term strategic planning for system enhancements across multiple dimensions.
|
||||||
|
|
||||||
|
### **[setup/](setup/)** - Setup Guides & Configuration
|
||||||
|
TradingView alert setup, n8n workflows, signal quality configuration, percentage sizing features.
|
||||||
|
|
||||||
|
### **[archived/](archived/)** - Historical Documentation
|
||||||
|
Obsolete verification checklists and documentation from previous development phases.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Quick Start
|
||||||
|
|
||||||
|
**New Developers:**
|
||||||
|
1. Read [TRADING_GOALS.md](TRADING_GOALS.md) - Understand financial objectives
|
||||||
|
2. Review [roadmaps/OPTIMIZATION_MASTER_ROADMAP.md](roadmaps/OPTIMIZATION_MASTER_ROADMAP.md) - Strategic vision
|
||||||
|
3. Check [setup/QUICK_SETUP_CARD.md](setup/QUICK_SETUP_CARD.md) - Quick configuration reference
|
||||||
|
4. Study [architecture/ADAPTIVE_LEVERAGE_SYSTEM.md](architecture/ADAPTIVE_LEVERAGE_SYSTEM.md) - Core risk management
|
||||||
|
|
||||||
|
**Debugging Issues:**
|
||||||
|
1. Search [bugs/](bugs/) directory for similar incidents
|
||||||
|
2. Review Common Pitfalls in `.github/copilot-instructions.md`
|
||||||
|
3. Check [deployments/](deployments/) for recent changes
|
||||||
|
4. Consult [analysis/DIAGNOSTIC_RESULTS_SUMMARY.md](analysis/DIAGNOSTIC_RESULTS_SUMMARY.md)
|
||||||
|
|
||||||
|
**Feature Development:**
|
||||||
|
1. Check [roadmaps/](roadmaps/) for planned enhancements
|
||||||
|
2. Review [architecture/](architecture/) for existing patterns
|
||||||
|
3. Read deployment verification docs in [deployments/](deployments/)
|
||||||
|
4. Follow verification procedures before marking features "complete"
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Current System Status
|
||||||
|
|
||||||
|
**Production Configuration (Dec 4, 2025):**
|
||||||
|
- **Capital:** $540 USDC (100% health, zero debt)
|
||||||
|
- **Quality Thresholds:** LONG ≥90, SHORT ≥95
|
||||||
|
- **Adaptive Leverage:** 10x for high-quality (≥95), 5x for borderline (90-94)
|
||||||
|
- **Indicator:** v9 Money Line with MA Gap + Momentum SHORT Filter
|
||||||
|
- **Position Sizing:** SOL-PERP at 100% allocation, 5x leverage
|
||||||
|
- **Runner System:** TP1 closes 60%, TP2 activates trailing stop on 40% runner
|
||||||
|
- **Smart Entry:** Enabled (waits for 0.3% pullback confirmation)
|
||||||
|
|
||||||
|
See [TRADING_GOALS.md](TRADING_GOALS.md) for complete financial roadmap ($106 → $100k+)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 Finding Documentation
|
||||||
|
|
||||||
|
**By Topic:**
|
||||||
|
- **Performance Data:** [analysis/](analysis/)
|
||||||
|
- **System Design:** [architecture/](architecture/)
|
||||||
|
- **Bug History:** [bugs/](bugs/)
|
||||||
|
- **Infrastructure:** [cluster/](cluster/)
|
||||||
|
- **Feature Status:** [deployments/](deployments/)
|
||||||
|
- **Future Plans:** [roadmaps/](roadmaps/)
|
||||||
|
- **Setup Guides:** [setup/](setup/)
|
||||||
|
|
||||||
|
**By Date:**
|
||||||
|
- Most recent deployments in [deployments/](deployments/)
|
||||||
|
- Historical bugs in [bugs/](bugs/)
|
||||||
|
- Archived docs in [archived/](archived/)
|
||||||
|
|
||||||
|
**By File Type:**
|
||||||
|
- `*ROADMAP.md` → [roadmaps/](roadmaps/)
|
||||||
|
- `CRITICAL_*.md` → [bugs/](bugs/)
|
||||||
|
- `*_COMPLETE.md` → [deployments/](deployments/)
|
||||||
|
- `*_ANALYSIS.md` → [analysis/](analysis/)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Documentation Standards
|
||||||
|
|
||||||
|
**File Naming Conventions:**
|
||||||
|
- `UPPERCASE_WITH_UNDERSCORES.md` - Important permanent docs
|
||||||
|
- `lowercase-with-dashes.md` - Temporary or draft docs
|
||||||
|
- `*_COMPLETE.md` - Verified deployment status
|
||||||
|
- `CRITICAL_*.md` - Bug reports requiring immediate attention
|
||||||
|
|
||||||
|
**Content Requirements:**
|
||||||
|
- **Date stamps:** All docs must include creation/update dates
|
||||||
|
- **Status indicators:** Use emojis (✅ ❌ 🔄 ⏳ 🎯) for quick status scanning
|
||||||
|
- **Code examples:** Always include working code snippets for complex features
|
||||||
|
- **Verification steps:** Document how to verify feature is working
|
||||||
|
- **Lessons learned:** Include "Why This Matters" and "Lessons Learned" sections
|
||||||
|
|
||||||
|
**Update Frequency:**
|
||||||
|
- **Real-time:** [deployments/](deployments/) - Update immediately after deployment
|
||||||
|
- **Weekly:** [roadmaps/](roadmaps/) - Review progress, update status
|
||||||
|
- **Monthly:** [analysis/](analysis/) - Performance review, optimization analysis
|
||||||
|
- **As-needed:** [bugs/](bugs/), [setup/](setup/), [architecture/](architecture/)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🤝 Contributing
|
||||||
|
|
||||||
|
**Before Committing:**
|
||||||
|
1. Update relevant documentation in appropriate subdirectory
|
||||||
|
2. Add entry to `.github/copilot-instructions.md` if system behavior changes
|
||||||
|
3. Create deployment verification doc in [deployments/](deployments/)
|
||||||
|
4. Update roadmap status in [roadmaps/](roadmaps/) if completing phase
|
||||||
|
5. Git commit with descriptive message (`docs:`, `feat:`, `fix:`, `critical:`)
|
||||||
|
|
||||||
|
**Documentation Mandate:**
|
||||||
|
- **EVERY git commit requires documentation update** - Code without docs = incomplete work
|
||||||
|
- See `.github/copilot-instructions.md` "DOCUMENTATION + GIT COMMIT: INSEPARABLE WORKFLOW"
|
||||||
|
- Common Pitfalls, new ENV variables, configuration changes → Document immediately
|
||||||
|
- Bug fixes → Add to Common Pitfalls with full incident details
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔗 External References
|
||||||
|
|
||||||
|
- **Main Repository:** `/home/icke/traderv4/`
|
||||||
|
- **Configuration:** `.github/copilot-instructions.md` (6,400+ lines)
|
||||||
|
- **Environment Variables:** `.env` (482 lines)
|
||||||
|
- **Database Schema:** `prisma/schema.prisma`
|
||||||
|
- **Primary Code:** `app/api/trading/`, `lib/trading/`, `lib/drift/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 Support
|
||||||
|
|
||||||
|
**For Issues:**
|
||||||
|
1. Check [bugs/](bugs/) for similar problems
|
||||||
|
2. Review Common Pitfalls in `.github/copilot-instructions.md`
|
||||||
|
3. Consult [deployments/](deployments/) for recent changes
|
||||||
|
4. Search git history: `git log --grep="keyword"`
|
||||||
|
|
||||||
|
**For Features:**
|
||||||
|
1. Check [roadmaps/](roadmaps/) for planned work
|
||||||
|
2. Review [architecture/](architecture/) for existing patterns
|
||||||
|
3. Study similar implementations in git history
|
||||||
|
|
||||||
|
**Remember:** This is a **real money trading system** - every change affects financial outcomes. Always verify thoroughly before declaring anything "working" or "fixed".
|
||||||
156
docs/analysis/README.md
Normal file
156
docs/analysis/README.md
Normal file
@@ -0,0 +1,156 @@
|
|||||||
|
# Performance Analysis & Optimization Studies
|
||||||
|
|
||||||
|
**Data-driven insights for trading system optimization.**
|
||||||
|
|
||||||
|
This directory contains performance analyses, blocked signals tracking, profit projections, and diagnostic results. All files focus on measuring system performance and identifying optimization opportunities.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Key Documents
|
||||||
|
|
||||||
|
### **Performance Reviews**
|
||||||
|
- `PROFIT_PROJECTION_NOV24_2025.md` - Monthly profit projections and compound growth analysis
|
||||||
|
- `QUALITY_90_TP1_ONLY_ANALYSIS.md` - Analysis of quality 90+ signals with TP1-only exits
|
||||||
|
- `SIGNAL_QUALITY_TEST_RESULTS.md` - Signal quality scoring validation results
|
||||||
|
|
||||||
|
### **Blocked Signals Analysis**
|
||||||
|
- `BLOCKED_SIGNALS_TRACKING.md` - System for tracking signals blocked by quality filters
|
||||||
|
- `BLOCKED_SIGNAL_ANALYSIS_DEC2.md` - December 2 analysis of blocked signals
|
||||||
|
- `BLOCKED_SIGNAL_CORRECTED_ANALYSIS_DEC2.md` - Corrected analysis after data quality fixes
|
||||||
|
|
||||||
|
### **Adaptive Leverage**
|
||||||
|
- `LONG_ADAPTIVE_LEVERAGE_VERIFICATION.md` - Verification of LONG direction adaptive leverage
|
||||||
|
- `SHORT_ADAPTIVE_LEVERAGE_IMPLEMENTED.md` - SHORT direction adaptive leverage implementation
|
||||||
|
|
||||||
|
### **System Health**
|
||||||
|
- `DIAGNOSTIC_RESULTS_SUMMARY.md` - Comprehensive diagnostic test results
|
||||||
|
- `SAFETY_ANALYSIS.md` - Risk management and safety parameter analysis
|
||||||
|
- `RECOVERY_PLAN.md` - System recovery procedures after incidents
|
||||||
|
- `ANALYTICS_STATUS_AND_NEXT_STEPS.md` - Analytics dashboard status and roadmap
|
||||||
|
|
||||||
|
### **Strategic Planning**
|
||||||
|
- `CLEANUP_PLAN.md` - Documentation reorganization strategy (precursor to this structure)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📈 Analysis Workflow
|
||||||
|
|
||||||
|
**1. Data Collection:**
|
||||||
|
- Trades executed with quality scores, ATR, ADX, RSI metrics
|
||||||
|
- Blocked signals saved to BlockedSignal table for comparison
|
||||||
|
- Position Manager tracks MAE (Max Adverse Excursion) and MFE (Max Favorable Excursion)
|
||||||
|
|
||||||
|
**2. Analysis Queries:**
|
||||||
|
```sql
|
||||||
|
-- Win rate by quality tier
|
||||||
|
SELECT
|
||||||
|
CASE
|
||||||
|
WHEN signalQualityScore >= 95 THEN '95-100'
|
||||||
|
WHEN signalQualityScore >= 90 THEN '90-94'
|
||||||
|
WHEN signalQualityScore >= 85 THEN '85-89'
|
||||||
|
ELSE '60-84'
|
||||||
|
END as tier,
|
||||||
|
COUNT(*) as trades,
|
||||||
|
ROUND(100.0 * SUM(CASE WHEN realizedPnL > 0 THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate,
|
||||||
|
ROUND(SUM(realizedPnL), 2) as total_pnl
|
||||||
|
FROM "Trade"
|
||||||
|
WHERE exitReason IS NOT NULL
|
||||||
|
GROUP BY tier
|
||||||
|
ORDER BY MIN(signalQualityScore) DESC;
|
||||||
|
```
|
||||||
|
|
||||||
|
**3. Optimization:**
|
||||||
|
- Compare blocked vs executed signal performance
|
||||||
|
- Identify quality threshold sweet spots
|
||||||
|
- Validate parameter changes with 50-100 trade samples
|
||||||
|
- Document findings in this directory
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Current Focus (Dec 2025)
|
||||||
|
|
||||||
|
**Active Analyses:**
|
||||||
|
- Quality threshold optimization (LONG 90, SHORT 95 under evaluation)
|
||||||
|
- Adaptive leverage performance tracking (10x vs 5x tiers)
|
||||||
|
- Blocked signal win rate comparison (would-be winners vs actual blocks)
|
||||||
|
- Runner system effectiveness (40% runner with ATR trailing stop)
|
||||||
|
|
||||||
|
**Upcoming:**
|
||||||
|
- Multi-timeframe comparison (5min vs 15min vs 1H)
|
||||||
|
- MA crossover pattern validation (ADX weak→strong hypothesis)
|
||||||
|
- Smart Entry Validation Queue effectiveness tracking
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Adding New Analyses
|
||||||
|
|
||||||
|
**Template Structure:**
|
||||||
|
```markdown
|
||||||
|
# [Analysis Title]
|
||||||
|
|
||||||
|
**Date:** [YYYY-MM-DD]
|
||||||
|
**Data Range:** [Date range or trade count]
|
||||||
|
**Objective:** [What you're analyzing]
|
||||||
|
|
||||||
|
## Executive Summary
|
||||||
|
[2-3 sentence key findings]
|
||||||
|
|
||||||
|
## Methodology
|
||||||
|
[How data was collected, SQL queries, filters applied]
|
||||||
|
|
||||||
|
## Results
|
||||||
|
[Tables, charts, statistics]
|
||||||
|
|
||||||
|
## Insights
|
||||||
|
[What the data means, patterns discovered]
|
||||||
|
|
||||||
|
## Recommendations
|
||||||
|
[Specific actions, parameter changes, threshold adjustments]
|
||||||
|
|
||||||
|
## Verification Required
|
||||||
|
[How to test recommendations, sample sizes needed]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Naming Convention:**
|
||||||
|
- `[TOPIC]_ANALYSIS_[DATE].md` for dated analyses
|
||||||
|
- `[FEATURE]_VERIFICATION.md` for feature validation
|
||||||
|
- `[METRIC]_TRACKING.md` for ongoing measurement
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 Finding Specific Analyses
|
||||||
|
|
||||||
|
**By Topic:**
|
||||||
|
- Quality scores → `SIGNAL_QUALITY_*`, `QUALITY_90_*`
|
||||||
|
- Leverage → `*_ADAPTIVE_LEVERAGE_*`
|
||||||
|
- Blocked signals → `BLOCKED_SIGNAL_*`
|
||||||
|
- System health → `DIAGNOSTIC_*`, `SAFETY_*`, `RECOVERY_*`
|
||||||
|
|
||||||
|
**By Date:**
|
||||||
|
- All files include creation dates in content
|
||||||
|
- Most recent: December 2025 blocked signal analyses
|
||||||
|
- Historical: CLEANUP_PLAN.md (documentation strategy)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ Important Notes
|
||||||
|
|
||||||
|
**Data Integrity:**
|
||||||
|
- Always filter out `signalSource='manual'` for indicator analysis
|
||||||
|
- Use `WHERE timeframe='5'` for production signal analysis
|
||||||
|
- Exclude `blockReason='DATA_COLLECTION_ONLY'` for execution analysis
|
||||||
|
|
||||||
|
**Sample Sizes:**
|
||||||
|
- Minimum 50 trades for statistical validity
|
||||||
|
- Minimum 20 blocked signals for threshold analysis
|
||||||
|
- 100+ trades preferred for major parameter changes
|
||||||
|
|
||||||
|
**Real Money System:**
|
||||||
|
- Every analysis affects financial outcomes
|
||||||
|
- Always include "Expected Impact" section (dollar amounts)
|
||||||
|
- Document both upside potential and downside risk
|
||||||
|
- Require user approval for threshold changes >10 points
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
See `../README.md` for overall documentation structure.
|
||||||
219
docs/architecture/README.md
Normal file
219
docs/architecture/README.md
Normal file
@@ -0,0 +1,219 @@
|
|||||||
|
# System Architecture & Design
|
||||||
|
|
||||||
|
**Core technical design documentation for Trading Bot v4 autonomous trading system.**
|
||||||
|
|
||||||
|
This directory contains architectural decisions, system design patterns, and technical specifications for major system components.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🏗️ Key Architecture Documents
|
||||||
|
|
||||||
|
### **Adaptive Leverage System**
|
||||||
|
- `ADAPTIVE_LEVERAGE_SYSTEM.md` - **Complete implementation guide**
|
||||||
|
- Quality-based leverage tiers (10x for ≥95, 5x for 90-94)
|
||||||
|
- Direction-specific thresholds (LONG ≥95, SHORT ≥90)
|
||||||
|
- Settings UI with real-time collateral display
|
||||||
|
- Risk-adjusted position sizing based on signal confidence
|
||||||
|
- **Status:** ✅ ACTIVE in production (Dec 1, 2025)
|
||||||
|
|
||||||
|
### **ATR Trailing Stop System**
|
||||||
|
- `ATR_TRAILING_STOP_FIX.md` - **Phase 7.3 adaptive trailing enhancement**
|
||||||
|
- Real-time 1-minute ADX monitoring (not static entry ADX)
|
||||||
|
- Dynamic trail multipliers: base 1.5× + ADX strength tier + acceleration bonus
|
||||||
|
- ADX acceleration (+7 points) → 1.3× multiplier
|
||||||
|
- Profit acceleration (>2%) → additional 1.3× multiplier
|
||||||
|
- Combined max: 3.16× multiplier for strong trending moves
|
||||||
|
- **Status:** ✅ DEPLOYED (Nov 27, 2025)
|
||||||
|
|
||||||
|
### **Indicator Version Tracking**
|
||||||
|
- `INDICATOR_VERSION_TRACKING.md` - **TradingView strategy versioning**
|
||||||
|
- v9: Money Line with MA Gap + Momentum SHORT Filter (PRODUCTION)
|
||||||
|
- v8: Money Line Sticky Trend (ARCHIVED - 57.1% WR baseline)
|
||||||
|
- Historical versions (v5-v7) for comparison
|
||||||
|
- Database field tracking for performance analysis
|
||||||
|
- **Purpose:** A/B testing and strategy evolution
|
||||||
|
|
||||||
|
### **Position Synchronization**
|
||||||
|
- `POSITION_SYNC_QUICK_REF.md` - **Position Manager sync procedures**
|
||||||
|
- Startup validation (orphan detection)
|
||||||
|
- External closure handling
|
||||||
|
- Ghost position cleanup
|
||||||
|
- Database-first state management
|
||||||
|
- **Critical for:** System restarts, container crashes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Design Principles
|
||||||
|
|
||||||
|
### **1. Database as Source of Truth**
|
||||||
|
- All state-changing operations write to database FIRST
|
||||||
|
- In-memory state (Position Manager) is derived from database
|
||||||
|
- Container restarts restore state from database
|
||||||
|
- External closures update database before removing from memory
|
||||||
|
|
||||||
|
### **2. Dual-Layer Redundancy**
|
||||||
|
- **Layer 1:** On-chain orders (Drift Protocol TP/SL limit orders)
|
||||||
|
- **Layer 2:** Software monitoring (Position Manager every 2 seconds)
|
||||||
|
- If on-chain orders fail, Position Manager executes via market orders
|
||||||
|
- Both layers track to same database for complete history
|
||||||
|
|
||||||
|
### **3. Singleton Pattern Everywhere**
|
||||||
|
```typescript
|
||||||
|
// CORRECT: Use getter functions
|
||||||
|
const driftService = await initializeDriftService()
|
||||||
|
const positionManager = await getInitializedPositionManager()
|
||||||
|
const prisma = getPrismaClient()
|
||||||
|
|
||||||
|
// WRONG: Never create multiple instances
|
||||||
|
const driftService = new DriftService() // ❌
|
||||||
|
```
|
||||||
|
|
||||||
|
### **4. Adaptive Systems Over Static Parameters**
|
||||||
|
- **ATR-based TP/SL:** Adapts to market volatility automatically
|
||||||
|
- **Adaptive Leverage:** Scales risk with signal quality
|
||||||
|
- **ADX-based Trailing:** Widens trail during trend acceleration
|
||||||
|
- **Smart Entry:** Validates price action before execution
|
||||||
|
|
||||||
|
### **5. Direction-Specific Configuration**
|
||||||
|
- LONG signals: Different quality thresholds than SHORT
|
||||||
|
- SHORT signals: Higher baseline difficulty, lower thresholds
|
||||||
|
- Momentum filters: Different for LONG (breakout) vs SHORT (reversal)
|
||||||
|
- Risk management: ADX-based runner SL positioning
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 System Components
|
||||||
|
|
||||||
|
### **Core Services (Singletons)**
|
||||||
|
- **DriftService:** Drift Protocol SDK wrapper, WebSocket subscriptions
|
||||||
|
- **Position Manager:** Software monitoring loop (2-second price checks)
|
||||||
|
- **Pyth Price Monitor:** Real-time oracle price feeds (WebSocket)
|
||||||
|
- **Stop Hunt Tracker:** 4-hour revenge window monitoring
|
||||||
|
- **Blocked Signal Tracker:** 30-minute price tracking for blocked signals
|
||||||
|
- **Smart Validation Queue:** Pullback confirmation system (30-second checks)
|
||||||
|
|
||||||
|
### **Key Workflows**
|
||||||
|
1. **Trade Execution:** TradingView → n8n → check-risk → execute → database → Position Manager
|
||||||
|
2. **Exit Management:** Price check → TP/SL trigger → closePosition → database → Telegram notification
|
||||||
|
3. **Startup Recovery:** Database query → Drift validation → restore Position Manager → resume monitoring
|
||||||
|
4. **Ghost Detection:** Drift API check → database cleanup → Position Manager removal
|
||||||
|
|
||||||
|
### **Data Flow**
|
||||||
|
```
|
||||||
|
TradingView Alert (5-min candle)
|
||||||
|
↓ [Webhook]
|
||||||
|
n8n Parse Signal Enhanced
|
||||||
|
↓ [HTTP POST]
|
||||||
|
/api/trading/check-risk
|
||||||
|
↓ [Quality score, cooldown, duplicates]
|
||||||
|
/api/trading/execute
|
||||||
|
↓ [Market order, TP/SL placement]
|
||||||
|
Database (Trade table)
|
||||||
|
↓ [State persistence]
|
||||||
|
Position Manager (monitoring)
|
||||||
|
↓ [2-second price checks]
|
||||||
|
Exit Detection (TP1/TP2/SL/Trailing)
|
||||||
|
↓ [Market close order]
|
||||||
|
Database Update (exit details)
|
||||||
|
↓ [P&L, exit reason, timestamps]
|
||||||
|
Telegram Notification
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Configuration Architecture
|
||||||
|
|
||||||
|
### **Three-Layer Merge Priority**
|
||||||
|
1. **Per-symbol ENV vars** (highest): `SOLANA_*`, `ETHEREUM_*`
|
||||||
|
2. **Market config** (code): `SUPPORTED_MARKETS` in config/trading.ts
|
||||||
|
3. **Global ENV vars** (fallback): `MAX_POSITION_SIZE_USD`, `LEVERAGE`
|
||||||
|
4. **Default config** (lowest): `DEFAULT_TRADING_CONFIG`
|
||||||
|
|
||||||
|
### **Key Configuration Files**
|
||||||
|
- `config/trading.ts` - Central config with merge logic
|
||||||
|
- `.env` - Environment variables (482 lines)
|
||||||
|
- `prisma/schema.prisma` - Database schema
|
||||||
|
- `.github/copilot-instructions.md` - System documentation (6,400+ lines)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Adding New Architecture
|
||||||
|
|
||||||
|
**When to Create Architecture Docs:**
|
||||||
|
- Major system component (Position Manager, Drift client, etc.)
|
||||||
|
- Risk management system (leverage, stops, entry validation)
|
||||||
|
- Data flow pattern (TradingView → execution → monitoring)
|
||||||
|
- Critical decision (database-first vs in-memory, singleton vs multiple instances)
|
||||||
|
|
||||||
|
**Template Structure:**
|
||||||
|
```markdown
|
||||||
|
# [Component Name]
|
||||||
|
|
||||||
|
**Purpose:** [One-line description]
|
||||||
|
|
||||||
|
## Architecture Decision
|
||||||
|
[Why this design was chosen, alternatives considered]
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
[Code structure, key classes, integration points]
|
||||||
|
|
||||||
|
## Data Flow
|
||||||
|
[How data moves through the component]
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
[ENV variables, defaults, merge priority]
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
[Failure scenarios, recovery procedures]
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
[How to test, expected behaviors, logs to watch]
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
[Restart requirements, migration steps]
|
||||||
|
|
||||||
|
## Lessons Learned
|
||||||
|
[What worked, what didn't, future improvements]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 Finding Architecture Info
|
||||||
|
|
||||||
|
**By Component:**
|
||||||
|
- Leverage system → `ADAPTIVE_LEVERAGE_SYSTEM.md`
|
||||||
|
- Trailing stops → `ATR_TRAILING_STOP_FIX.md`
|
||||||
|
- Indicator versions → `INDICATOR_VERSION_TRACKING.md`
|
||||||
|
- Position sync → `POSITION_SYNC_QUICK_REF.md`
|
||||||
|
|
||||||
|
**By Pattern:**
|
||||||
|
- Singleton → All architecture docs mention singleton pattern
|
||||||
|
- Adaptive systems → ATR, Leverage, Trailing Stop docs
|
||||||
|
- Database-first → Position Sync, all architecture patterns
|
||||||
|
- Direction-specific → Leverage, Indicator docs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ Critical Architecture Constraints
|
||||||
|
|
||||||
|
**Real Money System:**
|
||||||
|
- Every architectural decision affects financial outcomes
|
||||||
|
- Database integrity is non-negotiable (write FIRST, read to restore)
|
||||||
|
- Position Manager MUST have TP/SL backup (on-chain orders can fail)
|
||||||
|
- Container restarts MUST restore full state (database = source of truth)
|
||||||
|
|
||||||
|
**Performance Requirements:**
|
||||||
|
- Position Manager: 2-second monitoring loop (not slower)
|
||||||
|
- Price feeds: WebSocket real-time (not HTTP polling)
|
||||||
|
- Order placement: <1 second execution (market orders)
|
||||||
|
- Database writes: Non-blocking (async, don't wait for confirmation)
|
||||||
|
|
||||||
|
**Scalability Limits:**
|
||||||
|
- Maximum 20 simultaneous positions tracked
|
||||||
|
- Maximum 10 trades per hour per symbol
|
||||||
|
- Drift SDK: 50 RPC calls/second burst, 10/second sustained
|
||||||
|
- PostgreSQL: 10 concurrent connections max
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
See `../README.md` for overall documentation structure.
|
||||||
259
docs/archived/README.md
Normal file
259
docs/archived/README.md
Normal file
@@ -0,0 +1,259 @@
|
|||||||
|
# Archived Documentation
|
||||||
|
|
||||||
|
**Historical documentation for deprecated features and old system versions.**
|
||||||
|
|
||||||
|
This directory contains documentation for features that have been removed, deprecated, or replaced by newer systems. Kept for historical reference and learning from past decisions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📦 What's Archived Here
|
||||||
|
|
||||||
|
**Archived files are kept for:**
|
||||||
|
1. **Historical reference** - Understanding past design decisions
|
||||||
|
2. **Learning from failures** - Why certain approaches didn't work
|
||||||
|
3. **Baseline comparisons** - Comparing new systems to old
|
||||||
|
4. **Recovery information** - If rollback ever needed (rare)
|
||||||
|
|
||||||
|
**Archived files are NOT:**
|
||||||
|
- ❌ Active in production
|
||||||
|
- ❌ Maintained or updated
|
||||||
|
- ❌ Recommended for new implementations
|
||||||
|
- ❌ Complete or fully accurate anymore
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🗄️ Currently Archived (Examples)
|
||||||
|
|
||||||
|
### **Indicator Versions (Replaced by v9)**
|
||||||
|
|
||||||
|
**v5: Buy/Sell Signal Strategy**
|
||||||
|
- **Period:** Pre-Nov 12, 2024
|
||||||
|
- **Performance:** 36.4% WR, +$25.47 total
|
||||||
|
- **Why replaced:** Poor win rate, basic signal detection
|
||||||
|
- **Successor:** v6 HalfTrend + BarColor
|
||||||
|
|
||||||
|
**v6: HalfTrend + BarColor**
|
||||||
|
- **Period:** Nov 12-18, 2024
|
||||||
|
- **Performance:** 48% WR, -$47.70 total
|
||||||
|
- **Why replaced:** Marginal improvement, still underperforming
|
||||||
|
- **Successor:** v8 Money Line Sticky Trend
|
||||||
|
|
||||||
|
**v7: v6 with Toggles**
|
||||||
|
- **Period:** Nov 2024 (minimal data)
|
||||||
|
- **Performance:** Deprecated before significant data collected
|
||||||
|
- **Why replaced:** No fundamental improvements over v6
|
||||||
|
- **Successor:** v8 Money Line Sticky Trend
|
||||||
|
|
||||||
|
**v8: Money Line Sticky Trend**
|
||||||
|
- **Period:** Nov 18-26, 2024
|
||||||
|
- **Performance:** 57.1% WR, +$262.70 (8 trades)
|
||||||
|
- **Why replaced:** SHORT failures (5 oversold disasters), weak chop entry
|
||||||
|
- **Key issues:**
|
||||||
|
* RSI filter for SHORTs caught oversold (25-35 RSI) = wrong entries
|
||||||
|
* Quality 80 signal at ADX 20.7 (weak chop) = stopped out
|
||||||
|
- **Successor:** v9 Money Line with Momentum-Based SHORT Filter
|
||||||
|
- **Lesson learned:** Oversold doesn't mean reversal, need momentum at tops
|
||||||
|
|
||||||
|
**v10: Adaptive Position Scoring (REMOVED)**
|
||||||
|
- **Period:** Nov 28, 2024 (1 day)
|
||||||
|
- **Performance:** Parameter insensitivity (72 configs = identical $498.12 P&L)
|
||||||
|
- **Why removed:** Bug in penalty logic, no edge over v9, added complexity
|
||||||
|
- **Key issues:**
|
||||||
|
* Price position penalty applied to 18.9% (should only apply to 40-60% chop)
|
||||||
|
* No performance improvement despite added complexity
|
||||||
|
* Violated "simpler is better" principle
|
||||||
|
- **Lesson learned:** Parameter insensitivity indicates no real edge, just noise
|
||||||
|
- **Status:** ✅ FULLY REMOVED (Nov 28, 2025)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Current Production System
|
||||||
|
|
||||||
|
**v9: Money Line with MA Gap + Momentum SHORT Filter**
|
||||||
|
- **Period:** Nov 26, 2025+
|
||||||
|
- **Status:** ✅ PRODUCTION
|
||||||
|
- **Key Features:**
|
||||||
|
* MA Gap analysis (+5 to +15 quality points)
|
||||||
|
* Momentum-Based SHORT filter (ADX ≥23, Position ≥60% OR ≤40% + Vol ≥2.0x)
|
||||||
|
* Removed RSI filter for SHORTs (data showed RSI 50+ has 68.2% WR)
|
||||||
|
* Quality thresholds: LONG ≥90, SHORT ≥80
|
||||||
|
- **File:** `workflows/trading/moneyline_v9_ma_gap.pinescript`
|
||||||
|
- **Documentation:** `../roadmaps/INDICATOR_V9_MA_GAP_ROADMAP.md`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Performance Comparison
|
||||||
|
|
||||||
|
### **Win Rate Evolution**
|
||||||
|
```
|
||||||
|
v5: 36.4% WR (baseline - poor)
|
||||||
|
v6: 48.0% WR (+11.6% improvement)
|
||||||
|
v8: 57.1% WR (+9.1% improvement)
|
||||||
|
v9: TBD (expected >60% with momentum filter)
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Key Learnings from Archived Versions**
|
||||||
|
|
||||||
|
**From v5 → v6:**
|
||||||
|
- Basic signal detection insufficient
|
||||||
|
- Need trend confirmation (HalfTrend)
|
||||||
|
- Color-based bar confirmation helps
|
||||||
|
|
||||||
|
**From v6 → v8:**
|
||||||
|
- 0.6% flip threshold prevents whipsaw
|
||||||
|
- Momentum confirmation critical (not just price cross)
|
||||||
|
- Anti-whipsaw logic needed (MA spacing)
|
||||||
|
|
||||||
|
**From v8 → v9:**
|
||||||
|
- **CRITICAL:** Don't short oversold (RSI 25-35)
|
||||||
|
- Short momentum at tops (Position ≥60%)
|
||||||
|
- ADX ≥23 filters weak chop (avoid ADX 20.7 disasters)
|
||||||
|
- RSI 50+ SHORTs = 68.2% WR (best performance!)
|
||||||
|
- Quality thresholds should differ by direction (LONG 90, SHORT 80)
|
||||||
|
|
||||||
|
**From v10 attempt:**
|
||||||
|
- Parameter insensitivity = no edge
|
||||||
|
- Simpler systems often better than complex
|
||||||
|
- Bug-free simple logic > buggy complex logic
|
||||||
|
- Don't add complexity without proven edge
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 Finding Archived Information
|
||||||
|
|
||||||
|
**By Feature:**
|
||||||
|
- Old indicators → Check indicator version numbers (v5, v6, v7, v8, v10)
|
||||||
|
- Deprecated systems → Look for "REMOVED" or "DEPRECATED" in filenames
|
||||||
|
- Historical baselines → Compare performance metrics in docs
|
||||||
|
|
||||||
|
**By Date:**
|
||||||
|
- Pre-Nov 12, 2024 → v5 era
|
||||||
|
- Nov 12-18, 2024 → v6 era
|
||||||
|
- Nov 18-26, 2024 → v8 era
|
||||||
|
- Nov 26, 2024+ → v9 era (current)
|
||||||
|
|
||||||
|
**By Performance:**
|
||||||
|
- Worst: v5 (36.4% WR)
|
||||||
|
- Moderate: v6 (48% WR)
|
||||||
|
- Good: v8 (57.1% WR)
|
||||||
|
- Best: v9 (expected 60%+ WR)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ Using Archived Documentation
|
||||||
|
|
||||||
|
**DO:**
|
||||||
|
- ✅ Reference for historical context
|
||||||
|
- ✅ Learn from past mistakes
|
||||||
|
- ✅ Compare old vs new performance
|
||||||
|
- ✅ Understand why changes were made
|
||||||
|
|
||||||
|
**DON'T:**
|
||||||
|
- ❌ Implement archived features in production
|
||||||
|
- ❌ Assume archived docs are accurate for current system
|
||||||
|
- ❌ Use archived parameters without validation
|
||||||
|
- ❌ Copy code from archived implementations
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Archiving Process
|
||||||
|
|
||||||
|
**When to Archive:**
|
||||||
|
1. Feature completely replaced by newer system
|
||||||
|
2. Performance proven inferior to current
|
||||||
|
3. No longer maintained or updated
|
||||||
|
4. Kept only for historical reference
|
||||||
|
|
||||||
|
**How to Archive:**
|
||||||
|
1. Move file to `docs/archived/` directory
|
||||||
|
2. Add deprecation notice at top of file
|
||||||
|
3. Update references in active docs
|
||||||
|
4. Remove from main navigation
|
||||||
|
5. Keep in git history (never delete)
|
||||||
|
|
||||||
|
**Deprecation Notice Template:**
|
||||||
|
```markdown
|
||||||
|
# ⚠️ ARCHIVED - [Feature Name]
|
||||||
|
|
||||||
|
**Status:** DEPRECATED (Replaced by [Successor])
|
||||||
|
**Date Archived:** [Date]
|
||||||
|
**Reason:** [Why replaced/removed]
|
||||||
|
|
||||||
|
**For current system, see:** [Link to replacement docs]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
[Original content below...]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🗑️ What Gets Deleted vs Archived
|
||||||
|
|
||||||
|
**Archived (Kept):**
|
||||||
|
- ✅ Historical indicator versions
|
||||||
|
- ✅ Deprecated but once-production features
|
||||||
|
- ✅ Performance baselines and comparisons
|
||||||
|
- ✅ Design decisions that didn't work out
|
||||||
|
- ✅ Learning experiences worth documenting
|
||||||
|
|
||||||
|
**Deleted (Removed):**
|
||||||
|
- ❌ Temporary test files
|
||||||
|
- ❌ Duplicate documentation
|
||||||
|
- ❌ Incomplete drafts never used
|
||||||
|
- ❌ Outdated troubleshooting for removed systems
|
||||||
|
- ❌ Dead code with no historical value
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📚 Historical Analysis Resources
|
||||||
|
|
||||||
|
**Performance Tracking:**
|
||||||
|
- Database: Trade table with `indicatorVersion` field
|
||||||
|
- SQL queries: `WHERE indicatorVersion IN ('v5', 'v6', 'v8')`
|
||||||
|
- Analytics UI: Compare tab shows version performance
|
||||||
|
- CSV exports: Historical backtest results
|
||||||
|
|
||||||
|
**Commit History:**
|
||||||
|
```bash
|
||||||
|
# Find when feature was archived
|
||||||
|
git log --all --full-history -- "docs/archived/filename.md"
|
||||||
|
|
||||||
|
# See original implementation
|
||||||
|
git show [commit_hash]:path/to/original/file.ts
|
||||||
|
|
||||||
|
# Compare versions
|
||||||
|
git diff [old_commit] [new_commit] -- path/to/file
|
||||||
|
```
|
||||||
|
|
||||||
|
**Documentation Trail:**
|
||||||
|
- Copilot instructions: Common Pitfalls section documents issues
|
||||||
|
- Deployment docs: Show when features were removed
|
||||||
|
- Roadmaps: Track feature evolution over time
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎓 Learning from History
|
||||||
|
|
||||||
|
### **Successful Patterns**
|
||||||
|
- **Iterative improvement:** v5 → v6 → v8 → v9 steady progress
|
||||||
|
- **Data-driven decisions:** Each change validated with trade data
|
||||||
|
- **Simple foundations:** v9 builds on v8 core, removes complexity
|
||||||
|
- **Direction-specific logic:** LONG and SHORT need different filters
|
||||||
|
|
||||||
|
### **Failed Patterns**
|
||||||
|
- **Premature complexity:** v10 added complexity without proven edge
|
||||||
|
- **Parameter insensitivity:** Identical results across configs = no edge
|
||||||
|
- **Rigid filters:** RSI <35 for SHORT caught wrong phase (oversold)
|
||||||
|
- **Ignoring direction differences:** Same thresholds for LONG/SHORT suboptimal
|
||||||
|
|
||||||
|
### **Key Lessons**
|
||||||
|
1. **Simple > Complex:** v9 removed RSI filter, improved performance
|
||||||
|
2. **Test at scale:** 72 v10 configs with identical P&L = clear signal
|
||||||
|
3. **Direction matters:** LONGs and SHORTs need different strategies
|
||||||
|
4. **Data beats theory:** RSI 50+ SHORTs = 68.2% WR (counterintuitive)
|
||||||
|
5. **Fix bugs quickly:** v10 removed after 1 day when bug discovered
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
See `../README.md` for overall documentation structure.
|
||||||
242
docs/bugs/README.md
Normal file
242
docs/bugs/README.md
Normal file
@@ -0,0 +1,242 @@
|
|||||||
|
# Bug Reports & Critical Fixes
|
||||||
|
|
||||||
|
**Historical record of critical incidents, bugs, and their resolutions.**
|
||||||
|
|
||||||
|
This directory contains CRITICAL_*.md and FIXES_*.md bug reports. Every file documents a real incident that cost money, time, or could have caused financial loss if not caught.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚨 Critical Bug Reports
|
||||||
|
|
||||||
|
### **Position Management**
|
||||||
|
- `CRITICAL_FIX_POSITION_SIZE_BUG.md` - Smart Entry using webhook percentage as signal price
|
||||||
|
- **Impact:** $89 position sizes instead of $2,300, 97% pullback calculations
|
||||||
|
- **Root Cause:** TradingView webhook sent percentage (70.80) not price ($142.50)
|
||||||
|
- **Fix:** Use Pyth oracle price instead of webhook signal.price
|
||||||
|
- **Date:** Dec 3, 2025
|
||||||
|
|
||||||
|
- `CRITICAL_INCIDENT_UNPROTECTED_POSITION.md` - Database-first pattern violation
|
||||||
|
- **Impact:** Position opened with NO database record, NO Position Manager tracking
|
||||||
|
- **Root Cause:** Position Manager added before database save
|
||||||
|
- **Fix:** Database save FIRST, then Position Manager add
|
||||||
|
- **Date:** Nov 13, 2025
|
||||||
|
|
||||||
|
- `CRITICAL_TP1_FALSE_DETECTION_BUG.md` - TP1 detection fails when on-chain orders fill fast
|
||||||
|
- **Impact:** Winners marked as "SL" exits, analytics incorrect
|
||||||
|
- **Root Cause:** External closure detected after both TP1 + runner closed
|
||||||
|
- **Fix:** Percentage-based exit reason inference
|
||||||
|
- **Date:** Nov 19, 2025
|
||||||
|
|
||||||
|
### **Trade Execution**
|
||||||
|
- `CRITICAL_MISSING_TRADES_NOV19.md` - Trades executed but not in database
|
||||||
|
- **Impact:** 3 trades missing from database, P&L values inflated 5-14×
|
||||||
|
- **Root Cause:** P&L compounding in external closure detection
|
||||||
|
- **Fix:** Don't mutate trade.realizedPnL during calculation
|
||||||
|
- **Date:** Nov 19, 2025
|
||||||
|
|
||||||
|
- `CRITICAL_ISSUES_FOUND.md` - Multiple critical discovery incidents
|
||||||
|
- Container restart killing positions
|
||||||
|
- Phantom detection killing runners
|
||||||
|
- Field name mismatches in startup validation
|
||||||
|
- **Dates:** Various Nov 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Fix Documentation
|
||||||
|
|
||||||
|
### **Runner System**
|
||||||
|
- `FIXES_RUNNER_AND_CANCELLATION.md` - Runner stop loss gap fixes
|
||||||
|
- **Problem:** No SL protection between TP1 and TP2
|
||||||
|
- **Solution:** Explicit runner SL check in monitoring loop
|
||||||
|
- **Impact:** Prevented unlimited risk exposure on 25-40% runner
|
||||||
|
- **Date:** Nov 15, 2025
|
||||||
|
|
||||||
|
### **Applied Fixes**
|
||||||
|
- `FIXES_APPLIED.md` - Collection of multiple bug fixes
|
||||||
|
- TP1 detection
|
||||||
|
- P&L calculation
|
||||||
|
- External closure handling
|
||||||
|
- Order cancellation
|
||||||
|
- **Period:** Nov 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Common Bug Patterns
|
||||||
|
|
||||||
|
### **1. P&L Compounding (Multiple Incidents)**
|
||||||
|
**Pattern:** Monitoring loop detects closure multiple times, accumulates P&L
|
||||||
|
**Symptoms:** Database shows 5-20× actual P&L, duplicate Telegram notifications
|
||||||
|
**Root Cause:** Async operations + monitoring loop = race condition
|
||||||
|
**Solution:** Delete from Map IMMEDIATELY (atomic operation) before any async work
|
||||||
|
|
||||||
|
**Example Fixes:**
|
||||||
|
- Common Pitfall #49 (Nov 17, 2025)
|
||||||
|
- Common Pitfall #61 (Nov 22, 2025)
|
||||||
|
- Common Pitfall #67 (Dec 2, 2025)
|
||||||
|
|
||||||
|
### **2. Database-First Violations**
|
||||||
|
**Pattern:** In-memory state updated before database write
|
||||||
|
**Symptoms:** Container restart loses tracking, positions unprotected
|
||||||
|
**Root Cause:** Developer assumes database always succeeds
|
||||||
|
**Solution:** Always database write FIRST, then update in-memory state
|
||||||
|
|
||||||
|
**Example Fixes:**
|
||||||
|
- Common Pitfall #29 (Nov 13, 2025)
|
||||||
|
- CRITICAL_INCIDENT_UNPROTECTED_POSITION.md
|
||||||
|
|
||||||
|
### **3. External Closure Detection**
|
||||||
|
**Pattern:** Position closed on-chain, Position Manager detects late
|
||||||
|
**Symptoms:** Duplicate updates, wrong exit reasons, ghost positions
|
||||||
|
**Root Cause:** Drift state propagation delay (5-10 seconds)
|
||||||
|
**Solution:** Verification wait + closingInProgress flag + ghost detection
|
||||||
|
|
||||||
|
**Example Fixes:**
|
||||||
|
- Common Pitfall #47 (Nov 16, 2025) - Close verification gap
|
||||||
|
- Common Pitfall #56 (Nov 20, 2025) - Ghost orders
|
||||||
|
- Common Pitfall #57 (Nov 20, 2025) - P&L accuracy
|
||||||
|
|
||||||
|
### **4. Unit Conversion Errors**
|
||||||
|
**Pattern:** Tokens vs USD, percentage vs decimal, on-chain vs display units
|
||||||
|
**Symptoms:** Position sizes off by 100×+, TP/SL at wrong prices
|
||||||
|
**Root Cause:** SDK returns different units than expected
|
||||||
|
**Solution:** Always log raw values, verify units explicitly
|
||||||
|
|
||||||
|
**Example Fixes:**
|
||||||
|
- Common Pitfall #22 (Nov 12, 2025) - position.size is TOKENS not USD
|
||||||
|
- Common Pitfall #68 (Dec 3, 2025) - Signal price is percentage not USD
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 Debugging Checklist
|
||||||
|
|
||||||
|
**When investigating bugs:**
|
||||||
|
|
||||||
|
1. **Check logs first:**
|
||||||
|
```bash
|
||||||
|
docker logs -f trading-bot-v4 | grep -E "CRITICAL|ERROR|Failed"
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Verify deployment:**
|
||||||
|
```bash
|
||||||
|
# Container start time MUST be > commit timestamp
|
||||||
|
docker inspect trading-bot-v4 --format='{{.State.StartedAt}}'
|
||||||
|
git log -1 --format='%ai %s'
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Query database state:**
|
||||||
|
```sql
|
||||||
|
-- Check for inconsistencies
|
||||||
|
SELECT * FROM "Trade"
|
||||||
|
WHERE exitReason IS NULL
|
||||||
|
AND (createdAt < NOW() - INTERVAL '1 hour')
|
||||||
|
ORDER BY createdAt DESC;
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Compare to Drift:**
|
||||||
|
```bash
|
||||||
|
# Get actual position from Drift
|
||||||
|
curl -X GET http://localhost:3001/api/trading/positions \
|
||||||
|
-H "Authorization: Bearer $API_SECRET_KEY"
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Search Common Pitfalls:**
|
||||||
|
```bash
|
||||||
|
grep -n "symptom_keyword" .github/copilot-instructions.md | head -20
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Creating Bug Reports
|
||||||
|
|
||||||
|
**Required Sections:**
|
||||||
|
```markdown
|
||||||
|
# [Bug Title] (CRITICAL - Fixed [Date])
|
||||||
|
|
||||||
|
**Symptom:** [What user observed]
|
||||||
|
|
||||||
|
**Root Cause:** [Technical explanation]
|
||||||
|
|
||||||
|
**Real Incident ([Date]):**
|
||||||
|
* [Specific trade or event]
|
||||||
|
* [Expected behavior]
|
||||||
|
* [Actual behavior]
|
||||||
|
* [Financial impact]
|
||||||
|
|
||||||
|
**Impact:** [Scope of problem]
|
||||||
|
|
||||||
|
**Fix ([Date]):**
|
||||||
|
```typescript
|
||||||
|
// Code showing the fix
|
||||||
|
```
|
||||||
|
|
||||||
|
**Files changed:** [List of files]
|
||||||
|
|
||||||
|
**Git commit:** [Commit hash]
|
||||||
|
|
||||||
|
**Deployed:** [Deployment timestamp]
|
||||||
|
|
||||||
|
**Verification Required:**
|
||||||
|
* [How to test fix works]
|
||||||
|
* [Expected logs/behavior]
|
||||||
|
|
||||||
|
**Lessons Learned:**
|
||||||
|
1. [Key insight #1]
|
||||||
|
2. [Key insight #2]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Naming Convention:**
|
||||||
|
- `CRITICAL_[TOPIC]_BUG.md` - Single critical bug
|
||||||
|
- `CRITICAL_[TOPIC]_[DATE].md` - Dated incident report
|
||||||
|
- `FIXES_[SYSTEM].md` - Multiple related fixes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ Prevention Guidelines
|
||||||
|
|
||||||
|
**From Past Incidents:**
|
||||||
|
|
||||||
|
1. **Always verify deployment before declaring "fixed"**
|
||||||
|
- Container restart timestamp > commit timestamp
|
||||||
|
- Test with actual trade if possible
|
||||||
|
- Check logs for expected behavior change
|
||||||
|
|
||||||
|
2. **Never trust SDK data formats**
|
||||||
|
- Log raw values first
|
||||||
|
- Verify units explicitly (tokens vs USD, % vs decimal)
|
||||||
|
- Check SDK docs are often wrong
|
||||||
|
|
||||||
|
3. **Database writes before in-memory updates**
|
||||||
|
- Save to database FIRST
|
||||||
|
- Only then update Position Manager, caches, etc.
|
||||||
|
- If DB fails, don't proceed
|
||||||
|
|
||||||
|
4. **Async operations need serialization**
|
||||||
|
- Delete from Map IMMEDIATELY (atomic lock)
|
||||||
|
- Don't check Map.has() then delete later (race condition)
|
||||||
|
- Use Map.delete() return value as lock
|
||||||
|
|
||||||
|
5. **External closures need verification**
|
||||||
|
- Wait 5-10 seconds for Drift state propagation
|
||||||
|
- Query Drift to confirm position actually closed
|
||||||
|
- Keep monitoring if verification fails
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 Escalation
|
||||||
|
|
||||||
|
**When to declare "CRITICAL":**
|
||||||
|
- Financial loss occurred or was narrowly avoided
|
||||||
|
- System produced incorrect data (P&L, positions, exits)
|
||||||
|
- Position left unprotected (no monitoring, no TP/SL)
|
||||||
|
- Bug could recur and cause future losses
|
||||||
|
|
||||||
|
**Response Required:**
|
||||||
|
1. Stop trading immediately if data integrity affected
|
||||||
|
2. Document incident in this directory
|
||||||
|
3. Add to Common Pitfalls in copilot-instructions.md
|
||||||
|
4. Fix + deploy + verify within 24 hours
|
||||||
|
5. Update relevant architecture docs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
See `../README.md` for overall documentation structure.
|
||||||
291
docs/cluster/README.md
Normal file
291
docs/cluster/README.md
Normal file
@@ -0,0 +1,291 @@
|
|||||||
|
# Distributed Computing & EPYC Cluster
|
||||||
|
|
||||||
|
**Infrastructure for large-scale parameter optimization and backtesting.**
|
||||||
|
|
||||||
|
This directory contains documentation for the EPYC cluster setup, distributed backtesting coordination, and multi-server infrastructure.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🖥️ Cluster Documentation
|
||||||
|
|
||||||
|
### **EPYC Server Setup**
|
||||||
|
- `EPYC_SETUP_COMPREHENSIVE.md` - **Complete setup guide**
|
||||||
|
- Hardware: AMD EPYC 7282 16-Core Processor (Debian 12 Bookworm)
|
||||||
|
- Python environment: 3.11.2 with pandas 2.3.3, numpy 2.3.5
|
||||||
|
- SSH configuration: Nested hop (master → worker1 → worker2)
|
||||||
|
- Package deployment: tar.gz transfer with virtual environment
|
||||||
|
- **Status:** ✅ OPERATIONAL (24 workers processing 65,536 combos)
|
||||||
|
|
||||||
|
### **Distributed Architecture**
|
||||||
|
- `DUAL_SWEEP_README.md` - **Parallel sweep execution**
|
||||||
|
- Coordinator: Assigns chunks to workers
|
||||||
|
- Workers: Execute parameter combinations in parallel
|
||||||
|
- Database: SQLite exploration.db for state tracking
|
||||||
|
- Results: CSV files with top N configurations
|
||||||
|
- **Use case:** v9 exhaustive parameter optimization (Nov 28-29, 2025)
|
||||||
|
|
||||||
|
### **Cluster Control**
|
||||||
|
- `CLUSTER_START_BUTTON_FIX.md` - **Web UI integration**
|
||||||
|
- Dashboard: http://localhost:3001/cluster
|
||||||
|
- Start/Stop buttons with status detection
|
||||||
|
- Database-first status (SSH supplementary)
|
||||||
|
- Real-time progress tracking
|
||||||
|
- **Status:** ✅ DEPLOYED (Nov 30, 2025)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🏗️ Cluster Architecture
|
||||||
|
|
||||||
|
### **Physical Infrastructure**
|
||||||
|
```
|
||||||
|
Master Server (local development machine)
|
||||||
|
├── Coordinator Process (assigns chunks)
|
||||||
|
├── Database (exploration.db)
|
||||||
|
└── Web Dashboard (Next.js)
|
||||||
|
↓ [SSH]
|
||||||
|
Worker1 (EPYC 10.10.254.106)
|
||||||
|
├── 12 worker processes
|
||||||
|
├── 64GB RAM
|
||||||
|
└── Direct SSH connection
|
||||||
|
↓ [SSH ProxyJump]
|
||||||
|
Worker2 (EPYC 10.20.254.100)
|
||||||
|
├── 12 worker processes
|
||||||
|
├── 64GB RAM
|
||||||
|
└── Via worker1 hop
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Data Flow**
|
||||||
|
```
|
||||||
|
1. Coordinator creates chunks (2,000 combos each)
|
||||||
|
↓
|
||||||
|
2. Marks chunk status='pending' in database
|
||||||
|
↓
|
||||||
|
3. Worker queries database for pending chunks
|
||||||
|
↓
|
||||||
|
4. Coordinator assigns chunk to worker via SSH
|
||||||
|
↓
|
||||||
|
5. Worker updates status='running'
|
||||||
|
↓
|
||||||
|
6. Worker processes combinations in parallel
|
||||||
|
↓
|
||||||
|
7. Worker saves results to strategies table
|
||||||
|
↓
|
||||||
|
8. Worker updates status='completed'
|
||||||
|
↓
|
||||||
|
9. Coordinator assigns next pending chunk
|
||||||
|
↓
|
||||||
|
10. Dashboard shows real-time progress
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Database Schema**
|
||||||
|
```sql
|
||||||
|
-- chunks table: Work distribution
|
||||||
|
CREATE TABLE chunks (
|
||||||
|
id TEXT PRIMARY KEY, -- v9_chunk_000000
|
||||||
|
start_combo INTEGER, -- 0, 2000, 4000, etc.
|
||||||
|
end_combo INTEGER, -- 2000, 4000, 6000, etc.
|
||||||
|
status TEXT, -- 'pending', 'running', 'completed'
|
||||||
|
assigned_worker TEXT, -- 'worker1', 'worker2'
|
||||||
|
started_at INTEGER,
|
||||||
|
completed_at INTEGER
|
||||||
|
);
|
||||||
|
|
||||||
|
-- strategies table: Results storage
|
||||||
|
CREATE TABLE strategies (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
chunk_id TEXT,
|
||||||
|
params TEXT, -- JSON of parameter values
|
||||||
|
pnl REAL,
|
||||||
|
win_rate REAL,
|
||||||
|
profit_factor REAL,
|
||||||
|
max_drawdown REAL,
|
||||||
|
total_trades INTEGER
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Using the Cluster
|
||||||
|
|
||||||
|
### **Starting a Sweep**
|
||||||
|
```bash
|
||||||
|
# 1. Prepare package on master
|
||||||
|
cd /home/icke/traderv4/backtester
|
||||||
|
tar -czf backtest_v9_sweep.tar.gz data/ backtester_core.py v9_moneyline_ma_gap.py moneyline_core.py
|
||||||
|
|
||||||
|
# 2. Transfer to EPYC workers
|
||||||
|
scp backtest_v9_sweep.tar.gz root@10.10.254.106:/home/backtest/
|
||||||
|
ssh root@10.10.254.106 "scp backtest_v9_sweep.tar.gz root@10.20.254.100:/home/backtest/"
|
||||||
|
|
||||||
|
# 3. Extract on workers
|
||||||
|
ssh root@10.10.254.106 "cd /home/backtest && tar -xzf backtest_v9_sweep.tar.gz"
|
||||||
|
ssh root@10.10.254.106 "ssh root@10.20.254.100 'cd /home/backtest && tar -xzf backtest_v9_sweep.tar.gz'"
|
||||||
|
|
||||||
|
# 4. Start via web dashboard or CLI
|
||||||
|
# Web: http://localhost:3001/cluster → Click "Start Cluster"
|
||||||
|
# CLI: cd /home/icke/traderv4/cluster && python v9_advanced_coordinator.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Monitoring Progress**
|
||||||
|
```bash
|
||||||
|
# Dashboard
|
||||||
|
curl -s http://localhost:3001/api/cluster/status | jq
|
||||||
|
|
||||||
|
# Database query
|
||||||
|
sqlite3 cluster/exploration.db "SELECT status, COUNT(*) FROM chunks GROUP BY status;"
|
||||||
|
|
||||||
|
# Worker processes
|
||||||
|
ssh root@10.10.254.106 "ps aux | grep [p]ython | grep backtest | wc -l"
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Collecting Results**
|
||||||
|
```bash
|
||||||
|
# Results saved to cluster/results/
|
||||||
|
ls -lh cluster/results/sweep_v9_*.csv
|
||||||
|
|
||||||
|
# Top 100 configurations by P&L
|
||||||
|
sqlite3 cluster/exploration.db "SELECT params, pnl, win_rate FROM strategies ORDER BY pnl DESC LIMIT 100;"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Configuration
|
||||||
|
|
||||||
|
### **Coordinator Settings**
|
||||||
|
```python
|
||||||
|
# cluster/v9_advanced_coordinator.py
|
||||||
|
WORKERS = {
|
||||||
|
'worker1': {'host': '10.10.254.106', 'port': 22},
|
||||||
|
'worker2': {'host': '10.20.254.100', 'port': 22, 'proxy_jump': '10.10.254.106'}
|
||||||
|
}
|
||||||
|
|
||||||
|
CHUNK_SIZE = 2000 # Combinations per chunk
|
||||||
|
MAX_WORKERS = 24 # 12 per server
|
||||||
|
CHECK_INTERVAL = 60 # Status check frequency (seconds)
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Worker Settings**
|
||||||
|
```python
|
||||||
|
# cluster/distributed_worker.py
|
||||||
|
NUM_PROCESSES = 12 # Parallel backtests
|
||||||
|
BATCH_SIZE = 100 # Save results every N combos
|
||||||
|
TIMEOUT = 120 # Per-combo timeout (seconds)
|
||||||
|
```
|
||||||
|
|
||||||
|
### **SSH Configuration**
|
||||||
|
```bash
|
||||||
|
# ~/.ssh/config
|
||||||
|
Host worker1
|
||||||
|
HostName 10.10.254.106
|
||||||
|
User root
|
||||||
|
StrictHostKeyChecking no
|
||||||
|
ServerAliveInterval 30
|
||||||
|
|
||||||
|
Host worker2
|
||||||
|
HostName 10.20.254.100
|
||||||
|
User root
|
||||||
|
ProxyJump worker1
|
||||||
|
StrictHostKeyChecking no
|
||||||
|
ServerAliveInterval 30
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🐛 Common Issues
|
||||||
|
|
||||||
|
### **SSH Timeout Errors**
|
||||||
|
**Symptom:** "SSH command timed out for worker2"
|
||||||
|
**Root Cause:** Nested hop requires 60s timeout (not 30s)
|
||||||
|
**Fix:** Common Pitfall #64 - Increase subprocess timeout
|
||||||
|
```python
|
||||||
|
result = subprocess.run(ssh_cmd, timeout=60) # Not 30
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Database Lock Errors**
|
||||||
|
**Symptom:** "database is locked"
|
||||||
|
**Root Cause:** Multiple workers writing simultaneously
|
||||||
|
**Fix:** Use WAL mode + increase busy_timeout
|
||||||
|
```python
|
||||||
|
connection.execute('PRAGMA journal_mode=WAL')
|
||||||
|
connection.execute('PRAGMA busy_timeout=10000')
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Worker Not Processing**
|
||||||
|
**Symptom:** Chunk status='running' but no worker processes
|
||||||
|
**Root Cause:** Worker crashed or SSH session died
|
||||||
|
**Fix:** Cleanup database + restart
|
||||||
|
```bash
|
||||||
|
# Mark stuck chunks as pending
|
||||||
|
sqlite3 exploration.db "UPDATE chunks SET status='pending', assigned_worker=NULL WHERE status='running' AND started_at < (strftime('%s','now') - 600);"
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Status Shows "Idle" When Running**
|
||||||
|
**Symptom:** Dashboard shows idle despite workers running
|
||||||
|
**Root Cause:** SSH detection timing out, database not queried first
|
||||||
|
**Fix:** Database-first status detection (Common Pitfall #71)
|
||||||
|
```typescript
|
||||||
|
// Check database BEFORE SSH
|
||||||
|
const hasRunningChunks = explorationData.chunks.running > 0
|
||||||
|
if (hasRunningChunks) clusterStatus = 'active'
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Performance Metrics
|
||||||
|
|
||||||
|
**v9 Exhaustive Sweep (65,536 combos):**
|
||||||
|
- **Duration:** ~29 hours (24 workers)
|
||||||
|
- **Speed:** 1.60s per combo (4× faster than 6 local workers)
|
||||||
|
- **Throughput:** ~37 combos/minute across cluster
|
||||||
|
- **Data processed:** 139,678 OHLCV rows × 65,536 combos = 9.16B calculations
|
||||||
|
- **Results:** Top 100 saved to CSV (~10KB file)
|
||||||
|
|
||||||
|
**Cost Analysis:**
|
||||||
|
- Local (6 workers): 72 hours estimated
|
||||||
|
- EPYC (24 workers): 29 hours actual
|
||||||
|
- **Time savings:** 43 hours (60% faster)
|
||||||
|
- **Resource utilization:** 64 cores utilized vs 6 local
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Adding Cluster Features
|
||||||
|
|
||||||
|
**When to Use Cluster:**
|
||||||
|
- Parameter sweeps >10,000 combinations
|
||||||
|
- Backtests requiring >24 hours on local machine
|
||||||
|
- Multi-strategy comparison (need parallel execution)
|
||||||
|
- Production validation (test many configs simultaneously)
|
||||||
|
|
||||||
|
**Scaling Guidelines:**
|
||||||
|
- **<1,000 combos:** Local machine sufficient
|
||||||
|
- **1,000-10,000:** Single EPYC server (12 workers)
|
||||||
|
- **10,000-100,000:** Both EPYC servers (24 workers)
|
||||||
|
- **100,000+:** Consider cloud scaling (AWS Batch, etc.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ Important Notes
|
||||||
|
|
||||||
|
**Data Transfer:**
|
||||||
|
- Always compress packages: `tar -czf` (1.9MB → 1.1MB)
|
||||||
|
- Verify checksums after transfer
|
||||||
|
- Use rsync for incremental updates
|
||||||
|
|
||||||
|
**Process Management:**
|
||||||
|
- Always use `nohup` or `screen` for long-running coordinators
|
||||||
|
- Workers auto-terminate when chunks complete
|
||||||
|
- Coordinator sends Telegram notification on completion
|
||||||
|
|
||||||
|
**Database Safety:**
|
||||||
|
- SQLite WAL mode prevents most lock errors
|
||||||
|
- Backup exploration.db before major sweeps
|
||||||
|
- Never edit chunks table manually while coordinator running
|
||||||
|
|
||||||
|
**SSH Reliability:**
|
||||||
|
- ServerAliveInterval prevents silent disconnects
|
||||||
|
- StrictHostKeyChecking=no avoids interactive prompts
|
||||||
|
- ProxyJump handles nested hops automatically
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
See `../README.md` for overall documentation structure.
|
||||||
314
docs/deployments/README.md
Normal file
314
docs/deployments/README.md
Normal file
@@ -0,0 +1,314 @@
|
|||||||
|
# Deployment Status & Verification
|
||||||
|
|
||||||
|
**Verification reports for major feature deployments.**
|
||||||
|
|
||||||
|
This directory contains *_COMPLETE.md and DEPLOYMENT_*.md status files documenting deployment verification for significant system changes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ✅ Recent Deployments
|
||||||
|
|
||||||
|
### **December 2025**
|
||||||
|
- `DEPLOYMENT_SUCCESS_DEC3_2025.md` - **Bug fixes deployment**
|
||||||
|
- Bug #1: Revenge system external closure integration
|
||||||
|
- Bug #5: Smart Validation Queue bypass in execute endpoint
|
||||||
|
- 2 critical fixes deployed and verified
|
||||||
|
- **Status:** ✅ VERIFIED (Dec 3, 09:02 UTC)
|
||||||
|
|
||||||
|
### **November 2025**
|
||||||
|
- `V9_IMPLEMENTATION_COMPLETE.md` - **v9 Money Line indicator**
|
||||||
|
- MA Gap analysis (±5 to +15 quality points)
|
||||||
|
- Momentum-Based SHORT Filter (ADX ≥23, Position-based entry)
|
||||||
|
- Quality thresholds: LONG ≥90, SHORT ≥80
|
||||||
|
- **Status:** ✅ PRODUCTION (Nov 26, 2025)
|
||||||
|
|
||||||
|
- `PHASE_7.3_ADAPTIVE_TRAILING_DEPLOYED.md` - **Adaptive trailing stop**
|
||||||
|
- Real-time 1-minute ADX monitoring
|
||||||
|
- Dynamic multipliers: ADX acceleration/deceleration
|
||||||
|
- Combined max: 3.16× for strong trending moves
|
||||||
|
- **Status:** ✅ DEPLOYED (Nov 27, 2025)
|
||||||
|
|
||||||
|
- `ONE_YEAR_RETENTION_DEPLOYMENT.md` - **Data retention policy**
|
||||||
|
- Database: 1 year retention for Trade/PriceUpdate tables
|
||||||
|
- BlockedSignal: 90 days retention
|
||||||
|
- Automated cleanup job
|
||||||
|
- **Status:** ✅ ACTIVE
|
||||||
|
|
||||||
|
- `RUNNER_SYSTEM_FIX_COMPLETE.md` - **Runner stop loss protection**
|
||||||
|
- Fixed: No SL protection between TP1 and TP2
|
||||||
|
- Added: Explicit runner SL check in monitoring loop
|
||||||
|
- ADX-based runner SL positioning
|
||||||
|
- **Status:** ✅ VERIFIED (Nov 15, 2025)
|
||||||
|
|
||||||
|
- `REENTRY_SYSTEM_COMPLETE.md` - **Manual trade validation**
|
||||||
|
- Fresh TradingView data validation (<5min old)
|
||||||
|
- Recent performance modifiers (±20 points)
|
||||||
|
- Minimum score 55 (vs 60 for new signals)
|
||||||
|
- **Status:** ✅ OPERATIONAL
|
||||||
|
|
||||||
|
- `MA_CROSSOVER_DETECTION_COMPLETE.md` - **MA crossover tracking**
|
||||||
|
- n8n Parse Signal Enhanced detects "crossing" keyword
|
||||||
|
- Flags: isMACrossover, isDeathCross, isGoldenCross
|
||||||
|
- Purpose: Data collection for ADX weak→strong validation
|
||||||
|
- **Status:** ✅ IMPLEMENTED (Nov 27, 2025)
|
||||||
|
|
||||||
|
- `CLUSTER_STOP_BUTTON_FIX_COMPLETE.md` - **Cluster control UI**
|
||||||
|
- Database-first status detection (SSH supplementary)
|
||||||
|
- Stop button properly shown when cluster active
|
||||||
|
- Conditional rendering based on cluster status
|
||||||
|
- **Status:** ✅ DEPLOYED (Nov 30, 2025)
|
||||||
|
|
||||||
|
- `SMART_ENTRY_DEPLOYMENT_STATUS.md` - **Smart Entry Validation Queue**
|
||||||
|
- Block & Watch system for quality 50-89 signals
|
||||||
|
- ±0.3% confirmation thresholds (optimized from backtest)
|
||||||
|
- 30-second monitoring with 10-minute expiry
|
||||||
|
- **Status:** ✅ ACTIVE (Nov 30, 2025)
|
||||||
|
|
||||||
|
- `SMART_ENTRY_TIMING_STATUS.md` - **Smart Entry pullback detection**
|
||||||
|
- Waits for 0.5-2% pullback before entry
|
||||||
|
- Adaptive based on ATR and market conditions
|
||||||
|
- Improves entry prices on quality 85+ signals
|
||||||
|
- **Status:** ✅ ENABLED
|
||||||
|
|
||||||
|
- `TP1_FIX_DEPLOYMENT_SUMMARY.md` - **TP1 detection accuracy**
|
||||||
|
- Fixed: TP1 detection when on-chain orders fill fast
|
||||||
|
- Percentage-based exit reason inference (0.3-1.2% = TP1)
|
||||||
|
- Database records now accurate
|
||||||
|
- **Status:** ✅ VERIFIED (Nov 19, 2025)
|
||||||
|
|
||||||
|
- `PHASE_4_VERIFICATION_COMPLETE.md` - **Historical verification**
|
||||||
|
- Phase 4 position scaling optimizations
|
||||||
|
- TP2-as-runner system validation
|
||||||
|
- 40% runner with ATR trailing stop
|
||||||
|
- **Status:** ✅ COMPLETE
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Deployment Checklist Template
|
||||||
|
|
||||||
|
**For Every Major Deployment:**
|
||||||
|
|
||||||
|
### **1. Pre-Deployment**
|
||||||
|
- [ ] Code changes committed and pushed
|
||||||
|
- [ ] Unit tests pass (if applicable)
|
||||||
|
- [ ] Integration tests pass
|
||||||
|
- [ ] Documentation updated (copilot-instructions.md)
|
||||||
|
- [ ] Deployment doc created in this directory
|
||||||
|
|
||||||
|
### **2. Build & Deploy**
|
||||||
|
```bash
|
||||||
|
# Build Docker image
|
||||||
|
cd /home/icke/traderv4
|
||||||
|
docker compose build trading-bot
|
||||||
|
|
||||||
|
# Restart container
|
||||||
|
docker compose up -d --force-recreate trading-bot
|
||||||
|
|
||||||
|
# Verify container running
|
||||||
|
docker ps | grep trading-bot
|
||||||
|
```
|
||||||
|
|
||||||
|
### **3. Verification**
|
||||||
|
```bash
|
||||||
|
# 1. Check container start time > commit time
|
||||||
|
echo "=== Container Start Time ===" && \
|
||||||
|
docker inspect trading-bot-v4 --format='{{.State.StartedAt}}'
|
||||||
|
|
||||||
|
echo -e "\n=== Recent Commits ===" && \
|
||||||
|
git log --oneline --since="YYYY-MM-DD HH:MM" --format='%h %ai %s'
|
||||||
|
|
||||||
|
# 2. Verify logs show expected behavior
|
||||||
|
docker logs -f trading-bot-v4 | grep -E "expected_log_pattern"
|
||||||
|
|
||||||
|
# 3. Test API endpoints
|
||||||
|
curl http://localhost:3001/api/endpoint/path
|
||||||
|
|
||||||
|
# 4. Database verification (if schema changes)
|
||||||
|
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\\dt"
|
||||||
|
```
|
||||||
|
|
||||||
|
### **4. Post-Deployment**
|
||||||
|
- [ ] Container start time > all commit timestamps ✅
|
||||||
|
- [ ] Expected log messages appear ✅
|
||||||
|
- [ ] API endpoints respond correctly ✅
|
||||||
|
- [ ] Database schema correct (if applicable) ✅
|
||||||
|
- [ ] Test trade executed successfully (if trading logic changed) ✅
|
||||||
|
- [ ] User notified of deployment ✅
|
||||||
|
|
||||||
|
### **5. Documentation**
|
||||||
|
- [ ] Deployment doc updated with verification results
|
||||||
|
- [ ] Common Pitfalls added (if bug fix)
|
||||||
|
- [ ] Roadmap updated (if completing phase)
|
||||||
|
- [ ] Git commit for deployment verification
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚨 Critical Deployment Rules
|
||||||
|
|
||||||
|
**NEVER declare "deployed" or "working" without:**
|
||||||
|
1. ✅ Verifying container start time > commit timestamp
|
||||||
|
2. ✅ Observing expected logs in production
|
||||||
|
3. ✅ Testing API endpoint or feature behavior
|
||||||
|
4. ✅ SQL query confirming database state (if applicable)
|
||||||
|
5. ✅ Test trade validation (if Position Manager changes)
|
||||||
|
|
||||||
|
**Why This Matters:**
|
||||||
|
- This is a **real money trading system**
|
||||||
|
- Premature "working" declarations cause financial losses
|
||||||
|
- Container restart required for code changes to take effect
|
||||||
|
- Code committed ≠ Code deployed ≠ Code verified
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Creating Deployment Docs
|
||||||
|
|
||||||
|
**Naming Convention:**
|
||||||
|
- `[FEATURE]_COMPLETE.md` - Major feature fully implemented
|
||||||
|
- `DEPLOYMENT_SUCCESS_[DATE].md` - Multiple fixes deployed together
|
||||||
|
- `PHASE_[N]_VERIFICATION_COMPLETE.md` - Roadmap phase completion
|
||||||
|
|
||||||
|
**Required Sections:**
|
||||||
|
```markdown
|
||||||
|
# [Feature Name] - DEPLOYED ([Date])
|
||||||
|
|
||||||
|
**Purpose:** [What this deployment does]
|
||||||
|
|
||||||
|
**Status:** ✅ VERIFIED | ⏳ PENDING | ❌ FAILED
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Changes Deployed
|
||||||
|
|
||||||
|
### **Files Modified**
|
||||||
|
- `path/to/file1.ts` - [Description]
|
||||||
|
- `path/to/file2.ts` - [Description]
|
||||||
|
|
||||||
|
### **Key Improvements**
|
||||||
|
1. [Change 1]
|
||||||
|
2. [Change 2]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Deployment Process
|
||||||
|
|
||||||
|
**Build Time:** [Time]
|
||||||
|
**Container Restart:** [Time]
|
||||||
|
**Git Commit:** [Hash] - [Message]
|
||||||
|
|
||||||
|
### **Verification Steps**
|
||||||
|
```bash
|
||||||
|
# Commands used to verify deployment
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Verification Results**
|
||||||
|
- ✅ Container start time: [Time] > Commit time: [Time]
|
||||||
|
- ✅ Expected logs observed: "[Log message]"
|
||||||
|
- ✅ API test successful: [Response]
|
||||||
|
- ✅ Database state correct: [Query results]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Expected Behavior
|
||||||
|
|
||||||
|
**Before Deployment:**
|
||||||
|
- [Old behavior description]
|
||||||
|
|
||||||
|
**After Deployment:**
|
||||||
|
- [New behavior description]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Monitoring
|
||||||
|
|
||||||
|
**What to Watch:**
|
||||||
|
- [Log patterns to monitor]
|
||||||
|
- [Database queries to check]
|
||||||
|
- [API endpoints to test]
|
||||||
|
|
||||||
|
**Success Indicators:**
|
||||||
|
- [How to know it's working correctly]
|
||||||
|
|
||||||
|
**Failure Indicators:**
|
||||||
|
- [How to detect problems]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Rollback Procedure
|
||||||
|
|
||||||
|
**If Problems Occur:**
|
||||||
|
```bash
|
||||||
|
# Revert to previous container
|
||||||
|
docker tag traderv4-trading-bot:backup traderv4-trading-bot:latest
|
||||||
|
docker compose up -d --force-recreate trading-bot
|
||||||
|
|
||||||
|
# Or git revert
|
||||||
|
git revert [commit_hash]
|
||||||
|
git push
|
||||||
|
# Then rebuild/restart
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Impact
|
||||||
|
|
||||||
|
**Expected:**
|
||||||
|
- [Benefit 1]
|
||||||
|
- [Benefit 2]
|
||||||
|
|
||||||
|
**Actual:** (Update after 24-48 hours)
|
||||||
|
- [Measured impact]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Lessons Learned
|
||||||
|
|
||||||
|
1. [Key insight #1]
|
||||||
|
2. [Key insight #2]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 Finding Deployment Info
|
||||||
|
|
||||||
|
**By Date:**
|
||||||
|
- Most recent → Check filename dates (YYYY-MM-DD or Month YYYY)
|
||||||
|
- Historical → Older files in this directory
|
||||||
|
|
||||||
|
**By Feature:**
|
||||||
|
- v9 indicator → `V9_IMPLEMENTATION_COMPLETE.md`
|
||||||
|
- Trailing stops → `PHASE_7.3_ADAPTIVE_TRAILING_DEPLOYED.md`
|
||||||
|
- Runner system → `RUNNER_SYSTEM_FIX_COMPLETE.md`
|
||||||
|
- Smart Entry → `SMART_ENTRY_*_STATUS.md`
|
||||||
|
- Cluster UI → `CLUSTER_STOP_BUTTON_FIX_COMPLETE.md`
|
||||||
|
|
||||||
|
**By Status:**
|
||||||
|
- ✅ VERIFIED → Deployment confirmed working
|
||||||
|
- ⏳ PENDING → Awaiting verification
|
||||||
|
- 🔄 IN PROGRESS → Deployment ongoing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Deployment Metrics
|
||||||
|
|
||||||
|
**Success Rate:**
|
||||||
|
- Track deployments requiring rollback
|
||||||
|
- Track verification time (commit → confirmed working)
|
||||||
|
- Track time to detect issues
|
||||||
|
|
||||||
|
**Typical Timeline:**
|
||||||
|
- Code change → Commit: 5-30 minutes
|
||||||
|
- Build Docker image: 60-90 seconds
|
||||||
|
- Container restart: 5-10 seconds
|
||||||
|
- Verification: 1-5 minutes
|
||||||
|
- **Total:** 10-45 minutes from code to verified
|
||||||
|
|
||||||
|
**Best Practices:**
|
||||||
|
- Always use `--force-recreate` flag for container restart
|
||||||
|
- Wait 10 seconds after restart before testing
|
||||||
|
- Test in order: Logs → API → Database → Feature
|
||||||
|
- Document verification steps in deployment doc
|
||||||
|
- Update roadmaps after phase completions
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
See `../README.md` for overall documentation structure.
|
||||||
254
docs/roadmaps/README.md
Normal file
254
docs/roadmaps/README.md
Normal file
@@ -0,0 +1,254 @@
|
|||||||
|
# Strategic Planning & Roadmaps
|
||||||
|
|
||||||
|
**Long-term strategic planning for Trading Bot v4 system enhancements.**
|
||||||
|
|
||||||
|
This directory contains all *ROADMAP*.md files documenting planned improvements across multiple optimization dimensions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🗺️ Active Roadmaps
|
||||||
|
|
||||||
|
### **Master Roadmap**
|
||||||
|
- `OPTIMIZATION_MASTER_ROADMAP.md` - **Consolidated strategic view**
|
||||||
|
- 3 major initiatives: Signal Quality, Position Scaling, ATR-based TP
|
||||||
|
- Combined impact: 35-40% P&L improvement when complete
|
||||||
|
- Data collection phases with 50-100 trade validation requirements
|
||||||
|
- **Status:** Phase 1 (data collection) across all three initiatives
|
||||||
|
|
||||||
|
### **Signal Quality**
|
||||||
|
- `SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md` - **Data-driven quality threshold optimization**
|
||||||
|
- Phase 1: Collect 10-20 blocked signals (🔄 IN PROGRESS)
|
||||||
|
- Phase 2: Analyze patterns, adjust thresholds (🔜 NEXT)
|
||||||
|
- Phase 3: Dual-threshold system or quality-specific sizing
|
||||||
|
- Phase 4: Automated price analysis for blocked signals
|
||||||
|
- Phase 5: ML-based scoring weight optimization
|
||||||
|
- **Goal:** Recover profitable setups currently blocked, filter true losers
|
||||||
|
|
||||||
|
### **Position Scaling**
|
||||||
|
- `POSITION_SCALING_ROADMAP.md` - **TP/SL and runner optimization**
|
||||||
|
- Phase 1: ✅ COMPLETE - Data collection with quality scores
|
||||||
|
- Phase 2: ATR-based dynamic targets (adapt to volatility)
|
||||||
|
- Phase 3: Signal quality-based scaling (high quality = larger runners)
|
||||||
|
- Phase 4: Direction-based optimization (shorts vs longs)
|
||||||
|
- Phase 5: ✅ COMPLETE - TP2-as-runner system (40% runner with ATR trailing)
|
||||||
|
- Phase 6: ML-based exit prediction
|
||||||
|
- **Goal:** Capture more of winning moves, reduce stop-outs
|
||||||
|
|
||||||
|
### **ATR-Based Risk Management**
|
||||||
|
- `ATR_BASED_TP_ROADMAP.md` - **Volatility-adaptive TP/SL system**
|
||||||
|
- Phase 1: ATR data collection from TradingView
|
||||||
|
- Phase 2: ✅ COMPLETE - ATR-based TP/SL calculation (Nov 17, 2025)
|
||||||
|
- Phase 3: Validation with 50+ trades
|
||||||
|
- Phase 4: Per-symbol ATR calibration
|
||||||
|
- **Goal:** Regime-agnostic targets (bull/bear markets adapt automatically)
|
||||||
|
|
||||||
|
### **1-Minute Data Enhancements**
|
||||||
|
- `1MIN_DATA_ENHANCEMENTS_ROADMAP.md` - **High-frequency data utilization**
|
||||||
|
- Phase 7.1: Smart Entry Timing (pullback detection)
|
||||||
|
- Phase 7.2: Revenge Trading System (90s confirmation, ADX-based)
|
||||||
|
- Phase 7.3: ✅ DEPLOYED - Adaptive Trailing Stop with real-time ADX (Nov 27, 2025)
|
||||||
|
- Phase 7.4: Multi-timeframe signal validation
|
||||||
|
- **Goal:** Use 1-minute data for entry timing, trailing stop optimization
|
||||||
|
|
||||||
|
### **Indicator Evolution**
|
||||||
|
- `INDICATOR_V9_MA_GAP_ROADMAP.md` - **TradingView strategy development**
|
||||||
|
- v9 Money Line: MA Gap analysis + Momentum SHORT Filter
|
||||||
|
- Parameter optimization: 65,536 exhaustive sweep on EPYC cluster
|
||||||
|
- Quality threshold tuning: LONG ≥90, SHORT ≥95
|
||||||
|
- **Goal:** Maximum profitability while maintaining quality standards
|
||||||
|
|
||||||
|
### **High Availability**
|
||||||
|
- `HA_SETUP_ROADMAP.md` - **99.9%+ uptime infrastructure**
|
||||||
|
- Phase 1-8: ✅ ALL COMPLETE (Nov 25, 2025)
|
||||||
|
- Primary: srvdocker02 (95.216.52.28:3001)
|
||||||
|
- Secondary: Hostinger (72.62.39.24:3001)
|
||||||
|
- Database replication, DNS failover, auto-recovery
|
||||||
|
- **Status:** Zero-downtime failover validated in production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Roadmap Methodology
|
||||||
|
|
||||||
|
### **Phase Structure**
|
||||||
|
All roadmaps follow consistent phase naming:
|
||||||
|
- **Phase 1:** Data collection (50-100 trades typical)
|
||||||
|
- **Phase 2:** Analysis and pattern identification
|
||||||
|
- **Phase 3:** Implementation and testing
|
||||||
|
- **Phase 4:** Validation and optimization
|
||||||
|
- **Phase 5:** Production deployment
|
||||||
|
- **Phase 6+:** Advanced features (ML, automation)
|
||||||
|
|
||||||
|
### **Status Indicators**
|
||||||
|
- ✅ **COMPLETE:** Implemented, deployed, validated with production data
|
||||||
|
- 🔄 **IN PROGRESS:** Active development or data collection
|
||||||
|
- 🔜 **NEXT:** Planned for immediate future (within 1-2 weeks)
|
||||||
|
- 🎯 **FUTURE:** Planned but not yet started
|
||||||
|
- 🤖 **DISTANT:** Long-term goal (ML, advanced features)
|
||||||
|
|
||||||
|
### **Gating Criteria**
|
||||||
|
**Before moving to next phase:**
|
||||||
|
1. **Data requirement met:** Minimum sample size achieved (50-100 trades)
|
||||||
|
2. **SQL analysis complete:** Patterns identified, validated statistically
|
||||||
|
3. **User approval:** For parameter changes affecting risk/profitability
|
||||||
|
4. **Deployment verified:** Code deployed, container restarted, logs confirmed
|
||||||
|
5. **Documentation updated:** Roadmap status, copilot-instructions.md, deployment docs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Current Focus (Dec 2025)
|
||||||
|
|
||||||
|
### **Active Data Collection**
|
||||||
|
1. **Blocked Signals:** 0/20 target (need 10-20 for threshold analysis)
|
||||||
|
2. **v9 Indicator:** 10/100 trades (parameter optimization ongoing)
|
||||||
|
3. **Adaptive Leverage:** 5/50 trades (10x vs 5x tier validation)
|
||||||
|
4. **Smart Entry Queue:** 0/30 validations (pullback confirmation system)
|
||||||
|
|
||||||
|
### **Recently Completed**
|
||||||
|
- ✅ TP2-as-runner system (40% runner, ATR trailing)
|
||||||
|
- ✅ ATR-based TP/SL (regime-agnostic targets)
|
||||||
|
- ✅ Adaptive leverage (10x high quality, 5x borderline)
|
||||||
|
- ✅ Phase 7.3 adaptive trailing (real-time ADX monitoring)
|
||||||
|
- ✅ HA setup (zero-downtime failover)
|
||||||
|
|
||||||
|
### **Next Up**
|
||||||
|
1. **Quality threshold optimization** (LONG 90→?, SHORT 95→?)
|
||||||
|
2. **v9 parameter tuning** (65,536 combo EPYC sweep results)
|
||||||
|
3. **Multi-timeframe comparison** (5min vs 15min vs 1H)
|
||||||
|
4. **Smart Entry effectiveness** (±0.3% thresholds validation)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Adding New Roadmaps
|
||||||
|
|
||||||
|
**When to create roadmap:**
|
||||||
|
- Multi-phase feature (>3 phases, >1 month duration)
|
||||||
|
- Strategic initiative affecting core system behavior
|
||||||
|
- Data collection requirement (need validation before production)
|
||||||
|
- User financial goals alignment needed
|
||||||
|
|
||||||
|
**Template Structure:**
|
||||||
|
```markdown
|
||||||
|
# [Feature Name] Roadmap
|
||||||
|
|
||||||
|
**Objective:** [One-line goal]
|
||||||
|
**Expected Impact:** [Dollar amount or % improvement]
|
||||||
|
**Duration:** [Estimated timeline]
|
||||||
|
**Status:** [Current phase]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: [Name] (Status Emoji)
|
||||||
|
**Objective:** [What to achieve]
|
||||||
|
**Duration:** [Time estimate]
|
||||||
|
**Data Required:** [Sample size]
|
||||||
|
|
||||||
|
### Tasks
|
||||||
|
- [ ] Task 1
|
||||||
|
- [ ] Task 2
|
||||||
|
|
||||||
|
### Success Criteria
|
||||||
|
- [Measurable outcome]
|
||||||
|
|
||||||
|
### Gating Decision
|
||||||
|
[What determines if we proceed to Phase 2]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2-N: [Continue pattern]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Progress Tracking
|
||||||
|
| Phase | Status | Start Date | End Date | Key Milestone |
|
||||||
|
|-------|--------|-----------|----------|---------------|
|
||||||
|
| Phase 1 | 🔄 IN PROGRESS | YYYY-MM-DD | - | 50 trades collected |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
- [Other roadmaps or features]
|
||||||
|
|
||||||
|
## Risks
|
||||||
|
- [Technical challenges]
|
||||||
|
- [Financial risks]
|
||||||
|
|
||||||
|
## Decision Points
|
||||||
|
- [Key decisions requiring user approval]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 Finding Roadmap Info
|
||||||
|
|
||||||
|
**By Status:**
|
||||||
|
- Active → `OPTIMIZATION_MASTER_ROADMAP.md` (consolidated view)
|
||||||
|
- Complete → Check ✅ markers in individual roadmaps
|
||||||
|
- Next up → Look for 🔜 NEXT indicators
|
||||||
|
- Long-term → 🤖 DISTANT phases
|
||||||
|
|
||||||
|
**By Feature:**
|
||||||
|
- Quality → `SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md`
|
||||||
|
- Position sizing → `POSITION_SCALING_ROADMAP.md`
|
||||||
|
- ATR system → `ATR_BASED_TP_ROADMAP.md`
|
||||||
|
- 1-min data → `1MIN_DATA_ENHANCEMENTS_ROADMAP.md`
|
||||||
|
- Indicator → `INDICATOR_V9_MA_GAP_ROADMAP.md`
|
||||||
|
- Infrastructure → `HA_SETUP_ROADMAP.md`
|
||||||
|
|
||||||
|
**By Priority:**
|
||||||
|
1. **Critical path:** OPTIMIZATION_MASTER_ROADMAP.md
|
||||||
|
2. **Active development:** Any with 🔄 IN PROGRESS
|
||||||
|
3. **User-requested:** Check git commits for recent roadmap updates
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📈 Progress Tracking
|
||||||
|
|
||||||
|
**Weekly Review:**
|
||||||
|
1. Update trade counts in data collection phases
|
||||||
|
2. Check SQL for analysis readiness (sample size thresholds)
|
||||||
|
3. Move phases from 🔄 to ✅ when complete
|
||||||
|
4. Update OPTIMIZATION_MASTER_ROADMAP.md combined impact estimates
|
||||||
|
|
||||||
|
**Monthly Review:**
|
||||||
|
1. Validate expected vs actual impact (did optimization deliver promised %)
|
||||||
|
2. Adjust future phase estimates based on actual results
|
||||||
|
3. Reprioritize roadmaps based on performance data
|
||||||
|
4. Archive completed roadmaps to docs/archived/
|
||||||
|
|
||||||
|
**Milestone Commits:**
|
||||||
|
```bash
|
||||||
|
git commit -m "roadmap: Phase X complete - [Feature Name]
|
||||||
|
|
||||||
|
- Data collection: X/Y trades
|
||||||
|
- Analysis results: [Key findings]
|
||||||
|
- Deployment: [Date]
|
||||||
|
- Impact: [Actual vs expected]
|
||||||
|
|
||||||
|
Next: Phase X+1 starting [Date]
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ Important Notes
|
||||||
|
|
||||||
|
**Data-Driven Only:**
|
||||||
|
- Never implement Phase 3 without Phase 2 analysis complete
|
||||||
|
- Minimum 50 trades for statistical validity
|
||||||
|
- SQL queries required to prove assumptions
|
||||||
|
|
||||||
|
**User Approval Required:**
|
||||||
|
- Quality threshold changes
|
||||||
|
- Leverage adjustments
|
||||||
|
- Risk parameter modifications
|
||||||
|
- Position sizing changes
|
||||||
|
|
||||||
|
**Real Money System:**
|
||||||
|
- Every phase affects financial outcomes
|
||||||
|
- Conservative estimates (under-promise, over-deliver)
|
||||||
|
- Document both upside and downside scenarios
|
||||||
|
- Rollback procedures for every deployment
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
See `../README.md` for overall documentation structure.
|
||||||
396
docs/setup/README.md
Normal file
396
docs/setup/README.md
Normal file
@@ -0,0 +1,396 @@
|
|||||||
|
# Setup Guides & Configuration
|
||||||
|
|
||||||
|
**Step-by-step guides for setting up trading bot components.**
|
||||||
|
|
||||||
|
This directory contains setup instructions for TradingView alerts, n8n workflows, external services, and infrastructure configuration.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Quick Setup Guides
|
||||||
|
|
||||||
|
### **Essential Setup (Required for Trading)**
|
||||||
|
- `SIGNAL_QUALITY_SETUP_GUIDE.md` - **Signal quality system**
|
||||||
|
- TradingView alert configuration
|
||||||
|
- Webhook payload format (ATR, ADX, RSI, volume, price position)
|
||||||
|
- n8n Parse Signal Enhanced setup
|
||||||
|
- Quality scoring thresholds
|
||||||
|
- **Status:** ✅ PRODUCTION
|
||||||
|
|
||||||
|
- `N8N_MARKET_DATA_SETUP.md` - **1-minute data collection**
|
||||||
|
- TradingView 1-minute alerts for market data cache
|
||||||
|
- Webhook endpoint: `/api/trading/market-data`
|
||||||
|
- Purpose: Smart Entry Validation Queue price updates
|
||||||
|
- Alert frequency: Every 1-5 minutes
|
||||||
|
- **Status:** ✅ ACTIVE
|
||||||
|
|
||||||
|
### **Infrastructure Setup**
|
||||||
|
- `EPYC_SETUP_COMPREHENSIVE.md` - **EPYC cluster configuration**
|
||||||
|
- SSH configuration (nested hop through worker1)
|
||||||
|
- Python environment setup (3.11.2 + pandas + numpy)
|
||||||
|
- Package deployment (tar.gz transfer)
|
||||||
|
- Worker initialization scripts
|
||||||
|
- **Location:** Also in `../cluster/` directory
|
||||||
|
- **Status:** ✅ OPERATIONAL
|
||||||
|
|
||||||
|
### **Development Setup**
|
||||||
|
- `QUICK_SETUP_CARD.md` - **Development environment**
|
||||||
|
- Docker Compose configuration
|
||||||
|
- Environment variables (.env file)
|
||||||
|
- Database initialization (PostgreSQL + Prisma)
|
||||||
|
- Local testing procedures
|
||||||
|
- **Status:** ✅ CURRENT
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 TradingView Alert Setup
|
||||||
|
|
||||||
|
### **Production Trading Signals (5-minute chart)**
|
||||||
|
|
||||||
|
**Alert Message Format:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"action": "{{strategy.order.action}}",
|
||||||
|
"symbol": "{{ticker}}",
|
||||||
|
"timeframe": "{{interval}}",
|
||||||
|
"currentPrice": {{close}},
|
||||||
|
"atr": {{ta.atr(14)}},
|
||||||
|
"adx": {{ta.dmi(14, 14)}},
|
||||||
|
"rsi": {{ta.rsi(14)}},
|
||||||
|
"volumeRatio": {{volume / ta.sma(volume, 20)}},
|
||||||
|
"pricePosition": {{(close - ta.lowest(low, 100)) / (ta.highest(high, 100) - ta.lowest(low, 100)) * 100}},
|
||||||
|
"indicatorVersion": "v9"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Alert Settings:**
|
||||||
|
- **Condition:** Strategy entry/exit
|
||||||
|
- **Trigger:** Once Per Bar Close
|
||||||
|
- **Expiration:** None (always active)
|
||||||
|
- **Webhook URL:** `https://your-domain.com/api/trading/execute`
|
||||||
|
|
||||||
|
**Required Fields:**
|
||||||
|
- ✅ `atr` - ATR-based TP/SL system (CRITICAL)
|
||||||
|
- ✅ `adx` - Trend strength filtering
|
||||||
|
- ✅ `rsi` - Momentum confirmation
|
||||||
|
- ✅ `volumeRatio` - Volume validation
|
||||||
|
- ✅ `pricePosition` - Range position (0-100%)
|
||||||
|
- ✅ `timeframe` - Distinguishes 5min (trade) vs 15min/1H (data collection)
|
||||||
|
- ✅ `indicatorVersion` - Track strategy performance (v9 current)
|
||||||
|
|
||||||
|
### **Market Data Alerts (1-minute chart)**
|
||||||
|
|
||||||
|
**Alert Message Format:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"action": "market_data",
|
||||||
|
"symbol": "{{ticker}}",
|
||||||
|
"timeframe": "{{interval}}",
|
||||||
|
"currentPrice": {{close}},
|
||||||
|
"atr": {{ta.atr(14)}},
|
||||||
|
"adx": {{ta.dmi(14, 14)}},
|
||||||
|
"rsi": {{ta.rsi(14)}},
|
||||||
|
"volumeRatio": {{volume / ta.sma(volume, 20)}},
|
||||||
|
"pricePosition": {{(close - ta.lowest(low, 100)) / (ta.highest(high, 100) - ta.lowest(low, 100)) * 100}}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Alert Settings:**
|
||||||
|
- **Condition:** Time trigger (every bar close)
|
||||||
|
- **Trigger:** Once Per Bar Close
|
||||||
|
- **Expiration:** None
|
||||||
|
- **Webhook URL:** `https://your-domain.com/api/trading/market-data`
|
||||||
|
|
||||||
|
**Purpose:**
|
||||||
|
- Updates market data cache for Smart Entry Validation Queue
|
||||||
|
- Provides fresh prices for re-entry validation
|
||||||
|
- Enables sub-5-minute confirmation thresholds
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 n8n Workflow Setup
|
||||||
|
|
||||||
|
### **Parse Signal Enhanced**
|
||||||
|
|
||||||
|
**Location:** `workflows/trading/parse_signal_enhanced.json`
|
||||||
|
|
||||||
|
**Functions:**
|
||||||
|
1. **Extract Metrics:** ADX, ATR, RSI, volume, price position
|
||||||
|
2. **Normalize Timeframe:** "5", "15", "60", "240", "D"
|
||||||
|
3. **Extract Indicator Version:** "IND:v9" → "v9"
|
||||||
|
4. **Detect MA Crossovers:** "crossing" keyword → flags
|
||||||
|
5. **Route to Execute:** Calls `/api/trading/execute`
|
||||||
|
|
||||||
|
**Key Nodes:**
|
||||||
|
- HTTP Request node → Receives TradingView webhook
|
||||||
|
- Code node → Parses JSON payload
|
||||||
|
- Set node → Normalizes field names
|
||||||
|
- HTTP Request node → Sends to execute endpoint
|
||||||
|
|
||||||
|
### **Market Data Collector**
|
||||||
|
|
||||||
|
**Location:** `workflows/trading/market_data_collector.json`
|
||||||
|
|
||||||
|
**Functions:**
|
||||||
|
1. Receives 1-minute TradingView alerts
|
||||||
|
2. Validates symbol format (SOLUSDT → SOL-PERP)
|
||||||
|
3. Updates market data cache via `/api/trading/market-data`
|
||||||
|
4. Logs cache update timestamp
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🗄️ Database Setup
|
||||||
|
|
||||||
|
### **Initial Setup (First Time)**
|
||||||
|
```bash
|
||||||
|
# 1. Start PostgreSQL container
|
||||||
|
cd /home/icke/traderv4
|
||||||
|
docker compose up -d trading-bot-postgres
|
||||||
|
|
||||||
|
# 2. Generate Prisma client
|
||||||
|
npx prisma generate
|
||||||
|
|
||||||
|
# 3. Run migrations
|
||||||
|
DATABASE_URL="postgresql://postgres:your_password@localhost:5432/trading_bot_v4" \
|
||||||
|
npx prisma migrate dev
|
||||||
|
|
||||||
|
# 4. Verify tables created
|
||||||
|
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\\dt"
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Schema Updates (After Code Changes)**
|
||||||
|
```bash
|
||||||
|
# 1. Generate migration
|
||||||
|
DATABASE_URL="postgresql://...@localhost:5432/..." \
|
||||||
|
npx prisma migrate dev --name describe_changes_here
|
||||||
|
|
||||||
|
# 2. Regenerate client
|
||||||
|
npx prisma generate
|
||||||
|
|
||||||
|
# 3. Rebuild Docker image (includes new Prisma client)
|
||||||
|
docker compose build trading-bot
|
||||||
|
|
||||||
|
# 4. Restart container
|
||||||
|
docker compose up -d --force-recreate trading-bot
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Database Backup**
|
||||||
|
```bash
|
||||||
|
# Backup to SQL file
|
||||||
|
docker exec trading-bot-postgres pg_dump -U postgres trading_bot_v4 > backup_$(date +%Y%m%d_%H%M%S).sql
|
||||||
|
|
||||||
|
# Restore from backup
|
||||||
|
cat backup_file.sql | docker exec -i trading-bot-postgres psql -U postgres -d trading_bot_v4
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔐 Environment Variables Setup
|
||||||
|
|
||||||
|
### **Required Variables (.env file)**
|
||||||
|
|
||||||
|
**Trading Configuration:**
|
||||||
|
```bash
|
||||||
|
# Drift Protocol (REQUIRED)
|
||||||
|
DRIFT_WALLET_PRIVATE_KEY="[91,24,...]" # JSON array or base58
|
||||||
|
API_SECRET_KEY="your-secret-key-here"
|
||||||
|
|
||||||
|
# Solana RPC (REQUIRED - use Helius!)
|
||||||
|
SOLANA_RPC_URL="https://mainnet.helius-rpc.com/?api-key=YOUR_KEY"
|
||||||
|
BACKUP_RPC_URL="https://api.mainnet-beta.solana.com"
|
||||||
|
|
||||||
|
# Database (REQUIRED)
|
||||||
|
DATABASE_URL="postgresql://postgres:password@trading-bot-postgres:5432/trading_bot_v4"
|
||||||
|
POSTGRES_PASSWORD="your-secure-password"
|
||||||
|
|
||||||
|
# Telegram (REQUIRED for notifications)
|
||||||
|
TELEGRAM_BOT_TOKEN="123456789:ABCdefGHIjklMNOpqrsTUVwxyz"
|
||||||
|
TELEGRAM_CHAT_ID="123456789"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Position Sizing:**
|
||||||
|
```bash
|
||||||
|
# Per-Symbol Configuration
|
||||||
|
SOLANA_ENABLED=true
|
||||||
|
SOLANA_POSITION_SIZE=100 # Percentage (1-100)
|
||||||
|
SOLANA_LEVERAGE=15 # Leverage multiplier
|
||||||
|
|
||||||
|
ETHEREUM_ENABLED=false
|
||||||
|
ETHEREUM_POSITION_SIZE=100
|
||||||
|
ETHEREUM_LEVERAGE=1
|
||||||
|
|
||||||
|
# Global Fallback (BTC, etc.)
|
||||||
|
MAX_POSITION_SIZE_USD=100
|
||||||
|
LEVERAGE=1
|
||||||
|
```
|
||||||
|
|
||||||
|
**Quality Thresholds:**
|
||||||
|
```bash
|
||||||
|
MIN_SIGNAL_QUALITY_SCORE_LONG=90 # LONG signals
|
||||||
|
MIN_SIGNAL_QUALITY_SCORE_SHORT=80 # SHORT signals (more permissive)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Adaptive Leverage:**
|
||||||
|
```bash
|
||||||
|
USE_ADAPTIVE_LEVERAGE=true
|
||||||
|
HIGH_QUALITY_LEVERAGE=10 # Quality ≥ threshold
|
||||||
|
LOW_QUALITY_LEVERAGE=5 # Quality < threshold
|
||||||
|
QUALITY_LEVERAGE_THRESHOLD_LONG=95 # LONG threshold
|
||||||
|
QUALITY_LEVERAGE_THRESHOLD_SHORT=90 # SHORT threshold
|
||||||
|
```
|
||||||
|
|
||||||
|
**See:** `../.github/copilot-instructions.md` Environment Variable Reference for complete list (100+ variables)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🐳 Docker Setup
|
||||||
|
|
||||||
|
### **Development (docker-compose.dev.yml)**
|
||||||
|
```bash
|
||||||
|
# Start all services
|
||||||
|
docker compose -f docker-compose.dev.yml up -d
|
||||||
|
|
||||||
|
# View logs
|
||||||
|
docker compose -f docker-compose.dev.yml logs -f trading-bot
|
||||||
|
|
||||||
|
# Stop all services
|
||||||
|
docker compose -f docker-compose.dev.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Production (docker-compose.yml)**
|
||||||
|
```bash
|
||||||
|
# Build and start
|
||||||
|
docker compose up -d --build trading-bot
|
||||||
|
|
||||||
|
# Restart with force recreate
|
||||||
|
docker compose up -d --force-recreate trading-bot
|
||||||
|
|
||||||
|
# Check status
|
||||||
|
docker compose ps
|
||||||
|
docker logs -f trading-bot-v4
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Telegram Bot (docker-compose.telegram-bot.yml)**
|
||||||
|
```bash
|
||||||
|
# Start Telegram bot
|
||||||
|
docker compose -f docker-compose.telegram-bot.yml up -d
|
||||||
|
|
||||||
|
# View logs
|
||||||
|
docker logs -f trading-bot-telegram
|
||||||
|
|
||||||
|
# Test command
|
||||||
|
# Send "long sol" to Telegram bot
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧪 Testing Setup
|
||||||
|
|
||||||
|
### **Test Trade Execution**
|
||||||
|
```bash
|
||||||
|
# Via Settings UI
|
||||||
|
# 1. Open http://localhost:3001/settings
|
||||||
|
# 2. Scroll to "Test Trading" section
|
||||||
|
# 3. Click "Test SOL LONG" or "Test SOL SHORT"
|
||||||
|
# 4. Monitor docker logs for execution
|
||||||
|
|
||||||
|
# Via Telegram Bot
|
||||||
|
# Send message: "long sol --force"
|
||||||
|
# Watch for confirmation message
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Verify Position Manager**
|
||||||
|
```bash
|
||||||
|
# 1. Execute test trade
|
||||||
|
# 2. Watch logs for monitoring cycle
|
||||||
|
docker logs -f trading-bot-v4 | grep -E "Position Manager|TP1|TP2|SL"
|
||||||
|
|
||||||
|
# 3. Check database
|
||||||
|
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c \
|
||||||
|
"SELECT symbol, direction, entryPrice, currentSize, tp1Hit FROM \"Trade\" WHERE exitReason IS NULL;"
|
||||||
|
```
|
||||||
|
|
||||||
|
### **API Health Checks**
|
||||||
|
```bash
|
||||||
|
# Drift account health
|
||||||
|
curl http://localhost:3001/api/drift/account-health
|
||||||
|
|
||||||
|
# Market data cache
|
||||||
|
curl http://localhost:3001/api/trading/market-data
|
||||||
|
|
||||||
|
# Cluster status
|
||||||
|
curl http://localhost:3001/api/cluster/status
|
||||||
|
|
||||||
|
# Analytics
|
||||||
|
curl http://localhost:3001/api/analytics/last-trade
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🛠️ Troubleshooting
|
||||||
|
|
||||||
|
### **Container Won't Start**
|
||||||
|
```bash
|
||||||
|
# Check logs for errors
|
||||||
|
docker logs trading-bot-v4
|
||||||
|
|
||||||
|
# Common issues:
|
||||||
|
# - DATABASE_URL incorrect (check connection string)
|
||||||
|
# - DRIFT_WALLET_PRIVATE_KEY format (JSON array or base58)
|
||||||
|
# - Port 3000 already in use (check with: lsof -i :3000)
|
||||||
|
```
|
||||||
|
|
||||||
|
### **TradingView Alerts Not Received**
|
||||||
|
```bash
|
||||||
|
# 1. Check n8n workflow execution log
|
||||||
|
# 2. Test webhook manually:
|
||||||
|
curl -X POST http://localhost:5678/webhook/parse-signal \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"action":"buy","symbol":"SOLUSDT","timeframe":"5",...}'
|
||||||
|
|
||||||
|
# 3. Verify webhook URL in TradingView alert settings
|
||||||
|
# 4. Check firewall allows incoming webhooks
|
||||||
|
```
|
||||||
|
|
||||||
|
### **Database Connection Failed**
|
||||||
|
```bash
|
||||||
|
# 1. Verify PostgreSQL running
|
||||||
|
docker ps | grep postgres
|
||||||
|
|
||||||
|
# 2. Test connection
|
||||||
|
docker exec trading-bot-postgres psql -U postgres -c "SELECT 1"
|
||||||
|
|
||||||
|
# 3. Check DATABASE_URL matches container name
|
||||||
|
# Runtime: postgresql://postgres:pass@trading-bot-postgres:5432/...
|
||||||
|
# Prisma CLI: postgresql://postgres:pass@localhost:5432/...
|
||||||
|
```
|
||||||
|
|
||||||
|
### **RPC Rate Limiting (429 Errors)**
|
||||||
|
```bash
|
||||||
|
# Symptom: Logs show "429 Too Many Requests"
|
||||||
|
# Fix: Switch to Helius RPC (higher rate limits)
|
||||||
|
# Update .env:
|
||||||
|
SOLANA_RPC_URL="https://mainnet.helius-rpc.com/?api-key=YOUR_KEY"
|
||||||
|
|
||||||
|
# Restart container
|
||||||
|
docker compose restart trading-bot
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📚 Additional Resources
|
||||||
|
|
||||||
|
**External Documentation:**
|
||||||
|
- Drift Protocol: https://docs.drift.trade
|
||||||
|
- Pyth Network: https://docs.pyth.network
|
||||||
|
- Helius RPC: https://docs.helius.dev
|
||||||
|
- TradingView Alerts: https://www.tradingview.com/support/solutions/43000529348
|
||||||
|
|
||||||
|
**Internal Documentation:**
|
||||||
|
- **Main README:** `../README.md`
|
||||||
|
- **Architecture:** `../architecture/` directory
|
||||||
|
- **Roadmaps:** `../roadmaps/` directory
|
||||||
|
- **Common Pitfalls:** `../.github/copilot-instructions.md`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
See `../README.md` for overall documentation structure.
|
||||||
27
workflows/trading/moneyline_1min_price_feed.pinescript
Normal file
27
workflows/trading/moneyline_1min_price_feed.pinescript
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
//@version=6
|
||||||
|
indicator("Money Line - 1min Price Feed (Simplified)", overlay=false)
|
||||||
|
|
||||||
|
// ==========================================
|
||||||
|
// PURPOSE: Lightweight 1-minute price updates for market data cache
|
||||||
|
// CHANGE (Dec 4, 2025): Removed all indicators to reduce TradingView alert queue pressure
|
||||||
|
//
|
||||||
|
// WHY: 60 alerts/hour with full metrics can cause 5-minute trading signals to be skipped
|
||||||
|
// SOLUTION: Send ONLY price + symbol + timeframe (70% smaller payload)
|
||||||
|
//
|
||||||
|
// USAGE: Create alert on indicator with "alert() function calls"
|
||||||
|
// WEBHOOK: https://flow.egonetix.de/webhook/tradingview-bot-v4 (SAME as trading signals)
|
||||||
|
// FORMAT: Minimal format - bot recognizes timeframe="1" and routes to data collection
|
||||||
|
// ==========================================
|
||||||
|
|
||||||
|
// Display current price (visual confirmation alert is working)
|
||||||
|
plot(close, "Price", color=color.white, linewidth=2)
|
||||||
|
|
||||||
|
// Build MINIMAL JSON message - ONLY essential data
|
||||||
|
// Format: "SYMBOL buy 1 @ PRICE"
|
||||||
|
// Bot uses timeframe="1" to route to BlockedSignal (no trade execution)
|
||||||
|
// Indicators (ATR/ADX/RSI/etc) removed - calculated from DB when needed
|
||||||
|
jsonMessage = 'SOLUSDT buy 1 @ ' + str.tostring(close)
|
||||||
|
|
||||||
|
// Send alert every bar close (every 1 minute on 1min chart)
|
||||||
|
if barstate.isconfirmed
|
||||||
|
alert(jsonMessage, alert.freq_once_per_bar)
|
||||||
Reference in New Issue
Block a user