root/trading_bot_v4
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Add 1-minute simplified price feed to reduce TradingView alert queue pressure

Browse Source 
- Create moneyline_1min_price_feed.pinescript (70% smaller payload)
- Remove ATR/ADX/RSI/VOL/POS from 1-minute alerts (not used for decisions)
- Keep only price + symbol + timeframe for market data cache
- Document rationale in docs/1MIN_SIMPLIFIED_FEED.md
- Fix: 5-minute trading signals being dropped due to 1-minute flood (60/hour)
- Impact: Preserve priority for actual trading signals
This commit is contained in:
mindesbunister
2025-12-04 11:19:04 +01:00
parent 4c36fa2bc3
commit dc674ec6d5
12 changed files with 2476 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

157
docs/1MIN_SIMPLIFIED_FEED.md Normal file
View File

@@ -0,0 +1,157 @@
# 1-Minute Simplified Price Feed (Dec 4, 2025)
## Problem Statement
**Issue:** TradingView alert queue getting overwhelmed by 1-minute signal flood
- **Volume:** 60 alerts/hour (one per minute) from 1-minute data feed
- **Impact:** 5-minute trading signals occasionally skipped (happened 2× so far)
- **Root cause:** 1-minute alerts sending full trading signal format with all indicators
## Solution: Lightweight Price-Only Feed
**Change:** Reduce 1-minute alert payload by ~70%
### Before (Full Format)
```
SOLUSDT buy 1 @ 142.08 | ATR:0.65 | ADX:14.3 | RSI:51.3 | VOL:0.87 | POS:59.3 | IND:v9
```
**Size:** ~80 characters
**Processing:** n8n parses 7 fields, bot calculates quality score, saves to BlockedSignal + MarketData
### After (Minimal Format)
```
SOLUSDT buy 1 @ 142.08
```
**Size:** ~22 characters (~73% reduction)
**Processing:** n8n parses 3 fields (symbol/timeframe/price), bot routes to BlockedSignal only
## Why This Works
### What We Actually Need from 1-Minute Data
1. ✅ **Price updates** - For market data cache (5-minute expiry)
2. ✅ **Symbol identification** - Routing to correct market
3. ✅ **Timeframe marker** - `timeframe="1"` triggers data collection path
4. ❌ **ATR/ADX/RSI/VOL/POS** - NOT used in real-time (recalculated from DB if needed)
5. ❌ **Direction** - Meaningless for data-only signals (see copilot-instructions.md)
### What We Already Have
- **5-minute signals** include full indicator context for trading decisions
- **BlockedSignalTracker** recalculates TP1/TP2/SL from database data (not live indicators)
- **Market data cache** only needs `currentPrice` + `timestamp` for validation (5-minute expiry)
- **MarketData table** can store NULL for indicators (not critical for analysis)
### What Gets Removed
```typescript
// OLD: Execute endpoint saves full metrics to MarketData table
await prisma.marketData.create({
data: {
atr: Number(body.atr) || 0, // ❌ Remove
adx: Number(body.adx) || 0, // ❌ Remove
rsi: Number(body.rsi) || 50, // ❌ Remove
volumeRatio: Number(body.volumeRatio) || 1.0, // ❌ Remove
pricePosition: Number(body.pricePosition) || 50, // ❌ Remove
maGap: Number(body.maGap) || undefined, // ❌ Remove
// Keep essentials
symbol: driftSymbol, // ✅ Keep
timeframe: '1', // ✅ Keep
price: currentPrice, // ✅ Keep
timestamp: new Date() // ✅ Keep (auto-generated)
}
})
```
## Implementation Plan
### Phase 1: Update TradingView Alert (IMMEDIATE)
1. Replace `moneyline_1min_data_feed.pinescript` with `moneyline_1min_price_feed.pinescript`
2. Update TradingView alert to use new script
3. Test: Verify alerts still route to BlockedSignal correctly
### Phase 2: Update Execute Endpoint (OPTIONAL)
Since we're only sending price, make execute endpoint resilient to missing fields:
```typescript
// Lines 193-207: MarketData.create()
// Change required fields to optional with safe defaults
atr: Number(body.atr) || null, // NULL if missing
adx: Number(body.adx) || null,
rsi: Number(body.rsi) || null,
// ... etc
```
### Phase 3: Monitor Results
- Track 5-minute signal reliability (should go from 98% → 100%)
- Verify 1-minute data still saves to BlockedSignal
- Check TradingView alert queue no longer shows congestion
## Expected Impact
### Benefits
1. **70% reduction** in TradingView alert payload size
2. **Priority preservation** - 5-minute trading signals less likely to be dropped
3. **Faster processing** - Less n8n parsing, less bot processing
4. **Same functionality** - No loss of critical features
### Tradeoffs
1. MarketData table will have NULL indicators (acceptable - not used for decisions)
2. Historical 1-minute indicator data less complete (acceptable - we have 5-minute full data)
### No Impact On
- ✅ 5-minute trading signals (still full format with all indicators)
- ✅ BlockedSignalTracker (calculates from price movements, not live indicators)
- ✅ Market data cache (only needs price + timestamp)
- ✅ Smart Entry Validation (uses 5-minute cached data, not 1-minute)
## Files Changed
### TradingView Pine Script
- **NEW:** `workflows/trading/moneyline_1min_price_feed.pinescript` - Simplified version
- **OLD:** `workflows/trading/moneyline_1min_data_feed.pinescript` - Full format (archived)
### n8n Workflow
- **UNCHANGED:** `workflows/trading/parse_signal_enhanced.json` - Already handles minimal format
- Extracts: `symbol`, `timeframe`, `signalPrice` (from `@ price` pattern)
- Defaults: ATR/ADX/RSI/etc to 0 if missing
### Bot Code
- **UNCHANGED:** `app/api/trading/execute/route.ts` - Already handles missing fields with fallbacks
- Lines 193-207: Uses `Number(body.field) || default` pattern
## Migration Steps
1. ✅ **Read this document** - Understand rationale
2. ⏳ **Update TradingView alert** - Replace with `moneyline_1min_price_feed.pinescript`
3. ⏳ **Test with 1-2 alerts** - Verify BlockedSignal still populates
4. ⏳ **Monitor for 24 hours** - Check 5-minute signal reliability
5. ⏳ **Archive old script** - Move `moneyline_1min_data_feed.pinescript` to archive/
## Rollback Plan
If issues arise:
1. Switch TradingView alert back to `moneyline_1min_data_feed.pinescript`
2. No code changes needed - bot handles both formats
## Decision Criteria
**Use simplified feed IF:**
- 5-minute signal drops are occurring regularly (2+ per week)
- TradingView alert queue shows congestion
- 1-minute indicator data not being used for analysis
**Keep full feed IF:**
- 5-minute signals are 100% reliable
- Future analysis requires minute-by-minute indicator tracking
- No performance issues observed
## Current Status (Dec 4, 2025)
- ✅ Simplified script created
- ⏳ TradingView alert update pending
- ⏳ Monitoring results pending
- **User decision:** Test simplified feed for 24-48 hours
## References
- Copilot Instructions: Section "1-Minute Data Collection System"
- Original Implementation: `docs/1MIN_DATA_COLLECTION_SIMPLE.md`
- BlockedSignalTracker: `lib/analysis/blocked-signal-tracker.ts`

161
docs/README.md Normal file
View File

@@ -0,0 +1,161 @@
# Trading Bot v4 Documentation
**Comprehensive documentation for the Trading Bot v4 autonomous trading system.**
Last Updated: December 4, 2025
---
## 📂 Documentation Structure
### **[analysis/](analysis/)** - Performance Analysis & Optimization Studies
Performance analyses, blocked signals tracking, profit projections, and diagnostic results. Data-driven insights for system optimization.
### **[architecture/](architecture/)** - System Architecture & Design
Adaptive leverage system, ATR trailing stops, indicator version tracking, and position synchronization. Core technical design documentation.
### **[bugs/](bugs/)** - Bug Reports & Critical Fixes
CRITICAL_*.md and FIXES_*.md bug reports. Historical record of critical incidents and their resolutions.
### **[cluster/](cluster/)** - Distributed Computing & EPYC Cluster
EPYC cluster setup, distributed backtesting coordination, dual sweep configurations. Multi-server infrastructure documentation.
### **[deployments/](deployments/)** - Deployment Status & Verification
*_COMPLETE.md and DEPLOYMENT_*.md status files. Verification reports for major feature deployments.
### **[roadmaps/](roadmaps/)** - Strategic Planning & Roadmaps
All *ROADMAP*.md files. Long-term strategic planning for system enhancements across multiple dimensions.
### **[setup/](setup/)** - Setup Guides & Configuration
TradingView alert setup, n8n workflows, signal quality configuration, percentage sizing features.
### **[archived/](archived/)** - Historical Documentation
Obsolete verification checklists and documentation from previous development phases.
---
## 🚀 Quick Start
**New Developers:**
1. Read [TRADING_GOALS.md](TRADING_GOALS.md) - Understand financial objectives
2. Review [roadmaps/OPTIMIZATION_MASTER_ROADMAP.md](roadmaps/OPTIMIZATION_MASTER_ROADMAP.md) - Strategic vision
3. Check [setup/QUICK_SETUP_CARD.md](setup/QUICK_SETUP_CARD.md) - Quick configuration reference
4. Study [architecture/ADAPTIVE_LEVERAGE_SYSTEM.md](architecture/ADAPTIVE_LEVERAGE_SYSTEM.md) - Core risk management
**Debugging Issues:**
1. Search [bugs/](bugs/) directory for similar incidents
2. Review Common Pitfalls in `.github/copilot-instructions.md`
3. Check [deployments/](deployments/) for recent changes
4. Consult [analysis/DIAGNOSTIC_RESULTS_SUMMARY.md](analysis/DIAGNOSTIC_RESULTS_SUMMARY.md)
**Feature Development:**
1. Check [roadmaps/](roadmaps/) for planned enhancements
2. Review [architecture/](architecture/) for existing patterns
3. Read deployment verification docs in [deployments/](deployments/)
4. Follow verification procedures before marking features "complete"
---
## 📊 Current System Status
**Production Configuration (Dec 4, 2025):**
- **Capital:** $540 USDC (100% health, zero debt)
- **Quality Thresholds:** LONG ≥90, SHORT ≥95
- **Adaptive Leverage:** 10x for high-quality (≥95), 5x for borderline (90-94)
- **Indicator:** v9 Money Line with MA Gap + Momentum SHORT Filter
- **Position Sizing:** SOL-PERP at 100% allocation, 5x leverage
- **Runner System:** TP1 closes 60%, TP2 activates trailing stop on 40% runner
- **Smart Entry:** Enabled (waits for 0.3% pullback confirmation)
See [TRADING_GOALS.md](TRADING_GOALS.md) for complete financial roadmap ($106 → $100k+)
---
## 🔍 Finding Documentation
**By Topic:**
- **Performance Data:** [analysis/](analysis/)
- **System Design:** [architecture/](architecture/)
- **Bug History:** [bugs/](bugs/)
- **Infrastructure:** [cluster/](cluster/)
- **Feature Status:** [deployments/](deployments/)
- **Future Plans:** [roadmaps/](roadmaps/)
- **Setup Guides:** [setup/](setup/)
**By Date:**
- Most recent deployments in [deployments/](deployments/)
- Historical bugs in [bugs/](bugs/)
- Archived docs in [archived/](archived/)
**By File Type:**
- `*ROADMAP.md` → [roadmaps/](roadmaps/)
- `CRITICAL_*.md` → [bugs/](bugs/)
- `*_COMPLETE.md` → [deployments/](deployments/)
- `*_ANALYSIS.md` → [analysis/](analysis/)
---
## 📝 Documentation Standards
**File Naming Conventions:**
- `UPPERCASE_WITH_UNDERSCORES.md` - Important permanent docs
- `lowercase-with-dashes.md` - Temporary or draft docs
- `*_COMPLETE.md` - Verified deployment status
- `CRITICAL_*.md` - Bug reports requiring immediate attention
**Content Requirements:**
- **Date stamps:** All docs must include creation/update dates
- **Status indicators:** Use emojis (✅ ❌ 🔄 ⏳ 🎯) for quick status scanning
- **Code examples:** Always include working code snippets for complex features
- **Verification steps:** Document how to verify feature is working
- **Lessons learned:** Include "Why This Matters" and "Lessons Learned" sections
**Update Frequency:**
- **Real-time:** [deployments/](deployments/) - Update immediately after deployment
- **Weekly:** [roadmaps/](roadmaps/) - Review progress, update status
- **Monthly:** [analysis/](analysis/) - Performance review, optimization analysis
- **As-needed:** [bugs/](bugs/), [setup/](setup/), [architecture/](architecture/)
---
## 🤝 Contributing
**Before Committing:**
1. Update relevant documentation in appropriate subdirectory
2. Add entry to `.github/copilot-instructions.md` if system behavior changes
3. Create deployment verification doc in [deployments/](deployments/)
4. Update roadmap status in [roadmaps/](roadmaps/) if completing phase
5. Git commit with descriptive message (`docs:`, `feat:`, `fix:`, `critical:`)
**Documentation Mandate:**
- **EVERY git commit requires documentation update** - Code without docs = incomplete work
- See `.github/copilot-instructions.md` "DOCUMENTATION + GIT COMMIT: INSEPARABLE WORKFLOW"
- Common Pitfalls, new ENV variables, configuration changes → Document immediately
- Bug fixes → Add to Common Pitfalls with full incident details
---
## 🔗 External References
- **Main Repository:** `/home/icke/traderv4/`
- **Configuration:** `.github/copilot-instructions.md` (6,400+ lines)
- **Environment Variables:** `.env` (482 lines)
- **Database Schema:** `prisma/schema.prisma`
- **Primary Code:** `app/api/trading/`, `lib/trading/`, `lib/drift/`
---
## 📞 Support
**For Issues:**
1. Check [bugs/](bugs/) for similar problems
2. Review Common Pitfalls in `.github/copilot-instructions.md`
3. Consult [deployments/](deployments/) for recent changes
4. Search git history: `git log --grep="keyword"`
**For Features:**
1. Check [roadmaps/](roadmaps/) for planned work
2. Review [architecture/](architecture/) for existing patterns
3. Study similar implementations in git history
**Remember:** This is a **real money trading system** - every change affects financial outcomes. Always verify thoroughly before declaring anything "working" or "fixed".

156
docs/analysis/README.md Normal file
View File

@@ -0,0 +1,156 @@
# Performance Analysis & Optimization Studies
**Data-driven insights for trading system optimization.**
This directory contains performance analyses, blocked signals tracking, profit projections, and diagnostic results. All files focus on measuring system performance and identifying optimization opportunities.
---
## 📊 Key Documents
### **Performance Reviews**
- `PROFIT_PROJECTION_NOV24_2025.md` - Monthly profit projections and compound growth analysis
- `QUALITY_90_TP1_ONLY_ANALYSIS.md` - Analysis of quality 90+ signals with TP1-only exits
- `SIGNAL_QUALITY_TEST_RESULTS.md` - Signal quality scoring validation results
### **Blocked Signals Analysis**
- `BLOCKED_SIGNALS_TRACKING.md` - System for tracking signals blocked by quality filters
- `BLOCKED_SIGNAL_ANALYSIS_DEC2.md` - December 2 analysis of blocked signals
- `BLOCKED_SIGNAL_CORRECTED_ANALYSIS_DEC2.md` - Corrected analysis after data quality fixes
### **Adaptive Leverage**
- `LONG_ADAPTIVE_LEVERAGE_VERIFICATION.md` - Verification of LONG direction adaptive leverage
- `SHORT_ADAPTIVE_LEVERAGE_IMPLEMENTED.md` - SHORT direction adaptive leverage implementation
### **System Health**
- `DIAGNOSTIC_RESULTS_SUMMARY.md` - Comprehensive diagnostic test results
- `SAFETY_ANALYSIS.md` - Risk management and safety parameter analysis
- `RECOVERY_PLAN.md` - System recovery procedures after incidents
- `ANALYTICS_STATUS_AND_NEXT_STEPS.md` - Analytics dashboard status and roadmap
### **Strategic Planning**
- `CLEANUP_PLAN.md` - Documentation reorganization strategy (precursor to this structure)
---
## 📈 Analysis Workflow
**1. Data Collection:**
- Trades executed with quality scores, ATR, ADX, RSI metrics
- Blocked signals saved to BlockedSignal table for comparison
- Position Manager tracks MAE (Max Adverse Excursion) and MFE (Max Favorable Excursion)
**2. Analysis Queries:**
```sql
-- Win rate by quality tier
SELECT
CASE
WHEN signalQualityScore >= 95 THEN '95-100'
WHEN signalQualityScore >= 90 THEN '90-94'
WHEN signalQualityScore >= 85 THEN '85-89'
ELSE '60-84'
END as tier,
COUNT(*) as trades,
ROUND(100.0 * SUM(CASE WHEN realizedPnL > 0 THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate,
ROUND(SUM(realizedPnL), 2) as total_pnl
FROM "Trade"
WHERE exitReason IS NOT NULL
GROUP BY tier
ORDER BY MIN(signalQualityScore) DESC;
```
**3. Optimization:**
- Compare blocked vs executed signal performance
- Identify quality threshold sweet spots
- Validate parameter changes with 50-100 trade samples
- Document findings in this directory
---
## 🎯 Current Focus (Dec 2025)
**Active Analyses:**
- Quality threshold optimization (LONG 90, SHORT 95 under evaluation)
- Adaptive leverage performance tracking (10x vs 5x tiers)
- Blocked signal win rate comparison (would-be winners vs actual blocks)
- Runner system effectiveness (40% runner with ATR trailing stop)
**Upcoming:**
- Multi-timeframe comparison (5min vs 15min vs 1H)
- MA crossover pattern validation (ADX weak→strong hypothesis)
- Smart Entry Validation Queue effectiveness tracking
---
## 📝 Adding New Analyses
**Template Structure:**
```markdown
# [Analysis Title]
**Date:** [YYYY-MM-DD]
**Data Range:** [Date range or trade count]
**Objective:** [What you're analyzing]
## Executive Summary
[2-3 sentence key findings]
## Methodology
[How data was collected, SQL queries, filters applied]
## Results
[Tables, charts, statistics]
## Insights
[What the data means, patterns discovered]
## Recommendations
[Specific actions, parameter changes, threshold adjustments]
## Verification Required
[How to test recommendations, sample sizes needed]
```
**Naming Convention:**
- `[TOPIC]_ANALYSIS_[DATE].md` for dated analyses
- `[FEATURE]_VERIFICATION.md` for feature validation
- `[METRIC]_TRACKING.md` for ongoing measurement
---
## 🔍 Finding Specific Analyses
**By Topic:**
- Quality scores → `SIGNAL_QUALITY_*`, `QUALITY_90_*`
- Leverage → `*_ADAPTIVE_LEVERAGE_*`
- Blocked signals → `BLOCKED_SIGNAL_*`
- System health → `DIAGNOSTIC_*`, `SAFETY_*`, `RECOVERY_*`
**By Date:**
- All files include creation dates in content
- Most recent: December 2025 blocked signal analyses
- Historical: CLEANUP_PLAN.md (documentation strategy)
---
## ⚠️ Important Notes
**Data Integrity:**
- Always filter out `signalSource='manual'` for indicator analysis
- Use `WHERE timeframe='5'` for production signal analysis
- Exclude `blockReason='DATA_COLLECTION_ONLY'` for execution analysis
**Sample Sizes:**
- Minimum 50 trades for statistical validity
- Minimum 20 blocked signals for threshold analysis
- 100+ trades preferred for major parameter changes
**Real Money System:**
- Every analysis affects financial outcomes
- Always include "Expected Impact" section (dollar amounts)
- Document both upside potential and downside risk
- Require user approval for threshold changes >10 points
---
See `../README.md` for overall documentation structure.

219
docs/architecture/README.md Normal file
View File

@@ -0,0 +1,219 @@
# System Architecture & Design
**Core technical design documentation for Trading Bot v4 autonomous trading system.**
This directory contains architectural decisions, system design patterns, and technical specifications for major system components.
---
## 🏗️ Key Architecture Documents
### **Adaptive Leverage System**
- `ADAPTIVE_LEVERAGE_SYSTEM.md` - **Complete implementation guide**
- Quality-based leverage tiers (10x for ≥95, 5x for 90-94)
- Direction-specific thresholds (LONG ≥95, SHORT ≥90)
- Settings UI with real-time collateral display
- Risk-adjusted position sizing based on signal confidence
- **Status:** ✅ ACTIVE in production (Dec 1, 2025)
### **ATR Trailing Stop System**
- `ATR_TRAILING_STOP_FIX.md` - **Phase 7.3 adaptive trailing enhancement**
- Real-time 1-minute ADX monitoring (not static entry ADX)
- Dynamic trail multipliers: base 1.5× + ADX strength tier + acceleration bonus
- ADX acceleration (+7 points) → 1.3× multiplier
- Profit acceleration (>2%) → additional 1.3× multiplier
- Combined max: 3.16× multiplier for strong trending moves
- **Status:** ✅ DEPLOYED (Nov 27, 2025)
### **Indicator Version Tracking**
- `INDICATOR_VERSION_TRACKING.md` - **TradingView strategy versioning**
- v9: Money Line with MA Gap + Momentum SHORT Filter (PRODUCTION)
- v8: Money Line Sticky Trend (ARCHIVED - 57.1% WR baseline)
- Historical versions (v5-v7) for comparison
- Database field tracking for performance analysis
- **Purpose:** A/B testing and strategy evolution
### **Position Synchronization**
- `POSITION_SYNC_QUICK_REF.md` - **Position Manager sync procedures**
- Startup validation (orphan detection)
- External closure handling
- Ghost position cleanup
- Database-first state management
- **Critical for:** System restarts, container crashes
---
## 🎯 Design Principles
### **1. Database as Source of Truth**
- All state-changing operations write to database FIRST
- In-memory state (Position Manager) is derived from database
- Container restarts restore state from database
- External closures update database before removing from memory
### **2. Dual-Layer Redundancy**
- **Layer 1:** On-chain orders (Drift Protocol TP/SL limit orders)
- **Layer 2:** Software monitoring (Position Manager every 2 seconds)
- If on-chain orders fail, Position Manager executes via market orders
- Both layers track to same database for complete history
### **3. Singleton Pattern Everywhere**
```typescript
// CORRECT: Use getter functions
const driftService = await initializeDriftService()
const positionManager = await getInitializedPositionManager()
const prisma = getPrismaClient()
// WRONG: Never create multiple instances
const driftService = new DriftService() // ❌
```
### **4. Adaptive Systems Over Static Parameters**
- **ATR-based TP/SL:** Adapts to market volatility automatically
- **Adaptive Leverage:** Scales risk with signal quality
- **ADX-based Trailing:** Widens trail during trend acceleration
- **Smart Entry:** Validates price action before execution
### **5. Direction-Specific Configuration**
- LONG signals: Different quality thresholds than SHORT
- SHORT signals: Higher baseline difficulty, lower thresholds
- Momentum filters: Different for LONG (breakout) vs SHORT (reversal)
- Risk management: ADX-based runner SL positioning
---
## 📊 System Components
### **Core Services (Singletons)**
- **DriftService:** Drift Protocol SDK wrapper, WebSocket subscriptions
- **Position Manager:** Software monitoring loop (2-second price checks)
- **Pyth Price Monitor:** Real-time oracle price feeds (WebSocket)
- **Stop Hunt Tracker:** 4-hour revenge window monitoring
- **Blocked Signal Tracker:** 30-minute price tracking for blocked signals
- **Smart Validation Queue:** Pullback confirmation system (30-second checks)
### **Key Workflows**
1. **Trade Execution:** TradingView → n8n → check-risk → execute → database → Position Manager
2. **Exit Management:** Price check → TP/SL trigger → closePosition → database → Telegram notification
3. **Startup Recovery:** Database query → Drift validation → restore Position Manager → resume monitoring
4. **Ghost Detection:** Drift API check → database cleanup → Position Manager removal
### **Data Flow**
```
TradingView Alert (5-min candle)
↓ [Webhook]
n8n Parse Signal Enhanced
↓ [HTTP POST]
/api/trading/check-risk
↓ [Quality score, cooldown, duplicates]
/api/trading/execute
↓ [Market order, TP/SL placement]
Database (Trade table)
↓ [State persistence]
Position Manager (monitoring)
↓ [2-second price checks]
Exit Detection (TP1/TP2/SL/Trailing)
↓ [Market close order]
Database Update (exit details)
↓ [P&L, exit reason, timestamps]
Telegram Notification
```
---
## 🔧 Configuration Architecture
### **Three-Layer Merge Priority**
1. **Per-symbol ENV vars** (highest): `SOLANA_*`, `ETHEREUM_*`
2. **Market config** (code): `SUPPORTED_MARKETS` in config/trading.ts
3. **Global ENV vars** (fallback): `MAX_POSITION_SIZE_USD`, `LEVERAGE`
4. **Default config** (lowest): `DEFAULT_TRADING_CONFIG`
### **Key Configuration Files**
- `config/trading.ts` - Central config with merge logic
- `.env` - Environment variables (482 lines)
- `prisma/schema.prisma` - Database schema
- `.github/copilot-instructions.md` - System documentation (6,400+ lines)
---
## 🚀 Adding New Architecture
**When to Create Architecture Docs:**
- Major system component (Position Manager, Drift client, etc.)
- Risk management system (leverage, stops, entry validation)
- Data flow pattern (TradingView → execution → monitoring)
- Critical decision (database-first vs in-memory, singleton vs multiple instances)
**Template Structure:**
```markdown
# [Component Name]
**Purpose:** [One-line description]
## Architecture Decision
[Why this design was chosen, alternatives considered]
## Implementation
[Code structure, key classes, integration points]
## Data Flow
[How data moves through the component]
## Configuration
[ENV variables, defaults, merge priority]
## Error Handling
[Failure scenarios, recovery procedures]
## Verification
[How to test, expected behaviors, logs to watch]
## Deployment
[Restart requirements, migration steps]
## Lessons Learned
[What worked, what didn't, future improvements]
```
---
## 🔍 Finding Architecture Info
**By Component:**
- Leverage system → `ADAPTIVE_LEVERAGE_SYSTEM.md`
- Trailing stops → `ATR_TRAILING_STOP_FIX.md`
- Indicator versions → `INDICATOR_VERSION_TRACKING.md`
- Position sync → `POSITION_SYNC_QUICK_REF.md`
**By Pattern:**
- Singleton → All architecture docs mention singleton pattern
- Adaptive systems → ATR, Leverage, Trailing Stop docs
- Database-first → Position Sync, all architecture patterns
- Direction-specific → Leverage, Indicator docs
---
## ⚠️ Critical Architecture Constraints
**Real Money System:**
- Every architectural decision affects financial outcomes
- Database integrity is non-negotiable (write FIRST, read to restore)
- Position Manager MUST have TP/SL backup (on-chain orders can fail)
- Container restarts MUST restore full state (database = source of truth)
**Performance Requirements:**
- Position Manager: 2-second monitoring loop (not slower)
- Price feeds: WebSocket real-time (not HTTP polling)
- Order placement: <1 second execution (market orders)
- Database writes: Non-blocking (async, don't wait for confirmation)
**Scalability Limits:**
- Maximum 20 simultaneous positions tracked
- Maximum 10 trades per hour per symbol
- Drift SDK: 50 RPC calls/second burst, 10/second sustained
- PostgreSQL: 10 concurrent connections max
---
See `../README.md` for overall documentation structure.

259
docs/archived/README.md Normal file
View File

@@ -0,0 +1,259 @@
# Archived Documentation
**Historical documentation for deprecated features and old system versions.**
This directory contains documentation for features that have been removed, deprecated, or replaced by newer systems. Kept for historical reference and learning from past decisions.
---
## 📦 What's Archived Here
**Archived files are kept for:**
1. **Historical reference** - Understanding past design decisions
2. **Learning from failures** - Why certain approaches didn't work
3. **Baseline comparisons** - Comparing new systems to old
4. **Recovery information** - If rollback ever needed (rare)
**Archived files are NOT:**
- ❌ Active in production
- ❌ Maintained or updated
- ❌ Recommended for new implementations
- ❌ Complete or fully accurate anymore
---
## 🗄️ Currently Archived (Examples)
### **Indicator Versions (Replaced by v9)**
**v5: Buy/Sell Signal Strategy**
- **Period:** Pre-Nov 12, 2024
- **Performance:** 36.4% WR, +$25.47 total
- **Why replaced:** Poor win rate, basic signal detection
- **Successor:** v6 HalfTrend + BarColor
**v6: HalfTrend + BarColor**
- **Period:** Nov 12-18, 2024
- **Performance:** 48% WR, -$47.70 total
- **Why replaced:** Marginal improvement, still underperforming
- **Successor:** v8 Money Line Sticky Trend
**v7: v6 with Toggles**
- **Period:** Nov 2024 (minimal data)
- **Performance:** Deprecated before significant data collected
- **Why replaced:** No fundamental improvements over v6
- **Successor:** v8 Money Line Sticky Trend
**v8: Money Line Sticky Trend**
- **Period:** Nov 18-26, 2024
- **Performance:** 57.1% WR, +$262.70 (8 trades)
- **Why replaced:** SHORT failures (5 oversold disasters), weak chop entry
- **Key issues:**
* RSI filter for SHORTs caught oversold (25-35 RSI) = wrong entries
* Quality 80 signal at ADX 20.7 (weak chop) = stopped out
- **Successor:** v9 Money Line with Momentum-Based SHORT Filter
- **Lesson learned:** Oversold doesn't mean reversal, need momentum at tops
**v10: Adaptive Position Scoring (REMOVED)**
- **Period:** Nov 28, 2024 (1 day)
- **Performance:** Parameter insensitivity (72 configs = identical $498.12 P&L)
- **Why removed:** Bug in penalty logic, no edge over v9, added complexity
- **Key issues:**
* Price position penalty applied to 18.9% (should only apply to 40-60% chop)
* No performance improvement despite added complexity
* Violated "simpler is better" principle
- **Lesson learned:** Parameter insensitivity indicates no real edge, just noise
- **Status:** ✅ FULLY REMOVED (Nov 28, 2025)
---
## 🎯 Current Production System
**v9: Money Line with MA Gap + Momentum SHORT Filter**
- **Period:** Nov 26, 2025+
- **Status:** ✅ PRODUCTION
- **Key Features:**
* MA Gap analysis (+5 to +15 quality points)
* Momentum-Based SHORT filter (ADX ≥23, Position ≥60% OR ≤40% + Vol ≥2.0x)
* Removed RSI filter for SHORTs (data showed RSI 50+ has 68.2% WR)
* Quality thresholds: LONG ≥90, SHORT ≥80
- **File:** `workflows/trading/moneyline_v9_ma_gap.pinescript`
- **Documentation:** `../roadmaps/INDICATOR_V9_MA_GAP_ROADMAP.md`
---
## 📊 Performance Comparison
### **Win Rate Evolution**
```
v5: 36.4% WR (baseline - poor)
v6: 48.0% WR (+11.6% improvement)
v8: 57.1% WR (+9.1% improvement)
v9: TBD (expected >60% with momentum filter)
```
### **Key Learnings from Archived Versions**
**From v5 → v6:**
- Basic signal detection insufficient
- Need trend confirmation (HalfTrend)
- Color-based bar confirmation helps
**From v6 → v8:**
- 0.6% flip threshold prevents whipsaw
- Momentum confirmation critical (not just price cross)
- Anti-whipsaw logic needed (MA spacing)
**From v8 → v9:**
- **CRITICAL:** Don't short oversold (RSI 25-35)
- Short momentum at tops (Position ≥60%)
- ADX ≥23 filters weak chop (avoid ADX 20.7 disasters)
- RSI 50+ SHORTs = 68.2% WR (best performance!)
- Quality thresholds should differ by direction (LONG 90, SHORT 80)
**From v10 attempt:**
- Parameter insensitivity = no edge
- Simpler systems often better than complex
- Bug-free simple logic > buggy complex logic
- Don't add complexity without proven edge
---
## 🔍 Finding Archived Information
**By Feature:**
- Old indicators → Check indicator version numbers (v5, v6, v7, v8, v10)
- Deprecated systems → Look for "REMOVED" or "DEPRECATED" in filenames
- Historical baselines → Compare performance metrics in docs
**By Date:**
- Pre-Nov 12, 2024 → v5 era
- Nov 12-18, 2024 → v6 era
- Nov 18-26, 2024 → v8 era
- Nov 26, 2024+ → v9 era (current)
**By Performance:**
- Worst: v5 (36.4% WR)
- Moderate: v6 (48% WR)
- Good: v8 (57.1% WR)
- Best: v9 (expected 60%+ WR)
---
## ⚠️ Using Archived Documentation
**DO:**
- ✅ Reference for historical context
- ✅ Learn from past mistakes
- ✅ Compare old vs new performance
- ✅ Understand why changes were made
**DON'T:**
- ❌ Implement archived features in production
- ❌ Assume archived docs are accurate for current system
- ❌ Use archived parameters without validation
- ❌ Copy code from archived implementations
---
## 📝 Archiving Process
**When to Archive:**
1. Feature completely replaced by newer system
2. Performance proven inferior to current
3. No longer maintained or updated
4. Kept only for historical reference
**How to Archive:**
1. Move file to `docs/archived/` directory
2. Add deprecation notice at top of file
3. Update references in active docs
4. Remove from main navigation
5. Keep in git history (never delete)
**Deprecation Notice Template:**
```markdown
# ⚠️ ARCHIVED - [Feature Name]
**Status:** DEPRECATED (Replaced by [Successor])
**Date Archived:** [Date]
**Reason:** [Why replaced/removed]
**For current system, see:** [Link to replacement docs]
---
[Original content below...]
```
---
## 🗑️ What Gets Deleted vs Archived
**Archived (Kept):**
- ✅ Historical indicator versions
- ✅ Deprecated but once-production features
- ✅ Performance baselines and comparisons
- ✅ Design decisions that didn't work out
- ✅ Learning experiences worth documenting
**Deleted (Removed):**
- ❌ Temporary test files
- ❌ Duplicate documentation
- ❌ Incomplete drafts never used
- ❌ Outdated troubleshooting for removed systems
- ❌ Dead code with no historical value
---
## 📚 Historical Analysis Resources
**Performance Tracking:**
- Database: Trade table with `indicatorVersion` field
- SQL queries: `WHERE indicatorVersion IN ('v5', 'v6', 'v8')`
- Analytics UI: Compare tab shows version performance
- CSV exports: Historical backtest results
**Commit History:**
```bash
# Find when feature was archived
git log --all --full-history -- "docs/archived/filename.md"
# See original implementation
git show [commit_hash]:path/to/original/file.ts
# Compare versions
git diff [old_commit] [new_commit] -- path/to/file
```
**Documentation Trail:**
- Copilot instructions: Common Pitfalls section documents issues
- Deployment docs: Show when features were removed
- Roadmaps: Track feature evolution over time
---
## 🎓 Learning from History
### **Successful Patterns**
- **Iterative improvement:** v5 → v6 → v8 → v9 steady progress
- **Data-driven decisions:** Each change validated with trade data
- **Simple foundations:** v9 builds on v8 core, removes complexity
- **Direction-specific logic:** LONG and SHORT need different filters
### **Failed Patterns**
- **Premature complexity:** v10 added complexity without proven edge
- **Parameter insensitivity:** Identical results across configs = no edge
- **Rigid filters:** RSI <35 for SHORT caught wrong phase (oversold)
- **Ignoring direction differences:** Same thresholds for LONG/SHORT suboptimal
### **Key Lessons**
1. **Simple > Complex:** v9 removed RSI filter, improved performance
2. **Test at scale:** 72 v10 configs with identical P&L = clear signal
3. **Direction matters:** LONGs and SHORTs need different strategies
4. **Data beats theory:** RSI 50+ SHORTs = 68.2% WR (counterintuitive)
5. **Fix bugs quickly:** v10 removed after 1 day when bug discovered
---
See `../README.md` for overall documentation structure.

242
docs/bugs/README.md Normal file
View File

@@ -0,0 +1,242 @@
# Bug Reports & Critical Fixes
**Historical record of critical incidents, bugs, and their resolutions.**
This directory contains CRITICAL_*.md and FIXES_*.md bug reports. Every file documents a real incident that cost money, time, or could have caused financial loss if not caught.
---
## 🚨 Critical Bug Reports
### **Position Management**
- `CRITICAL_FIX_POSITION_SIZE_BUG.md` - Smart Entry using webhook percentage as signal price
- **Impact:** $89 position sizes instead of $2,300, 97% pullback calculations
- **Root Cause:** TradingView webhook sent percentage (70.80) not price ($142.50)
- **Fix:** Use Pyth oracle price instead of webhook signal.price
- **Date:** Dec 3, 2025
- `CRITICAL_INCIDENT_UNPROTECTED_POSITION.md` - Database-first pattern violation
- **Impact:** Position opened with NO database record, NO Position Manager tracking
- **Root Cause:** Position Manager added before database save
- **Fix:** Database save FIRST, then Position Manager add
- **Date:** Nov 13, 2025
- `CRITICAL_TP1_FALSE_DETECTION_BUG.md` - TP1 detection fails when on-chain orders fill fast
- **Impact:** Winners marked as "SL" exits, analytics incorrect
- **Root Cause:** External closure detected after both TP1 + runner closed
- **Fix:** Percentage-based exit reason inference
- **Date:** Nov 19, 2025
### **Trade Execution**
- `CRITICAL_MISSING_TRADES_NOV19.md` - Trades executed but not in database
- **Impact:** 3 trades missing from database, P&L values inflated 5-14×
- **Root Cause:** P&L compounding in external closure detection
- **Fix:** Don't mutate trade.realizedPnL during calculation
- **Date:** Nov 19, 2025
- `CRITICAL_ISSUES_FOUND.md` - Multiple critical discovery incidents
- Container restart killing positions
- Phantom detection killing runners
- Field name mismatches in startup validation
- **Dates:** Various Nov 2025
---
## 🔧 Fix Documentation
### **Runner System**
- `FIXES_RUNNER_AND_CANCELLATION.md` - Runner stop loss gap fixes
- **Problem:** No SL protection between TP1 and TP2
- **Solution:** Explicit runner SL check in monitoring loop
- **Impact:** Prevented unlimited risk exposure on 25-40% runner
- **Date:** Nov 15, 2025
### **Applied Fixes**
- `FIXES_APPLIED.md` - Collection of multiple bug fixes
- TP1 detection
- P&L calculation
- External closure handling
- Order cancellation
- **Period:** Nov 2025
---
## 📋 Common Bug Patterns
### **1. P&L Compounding (Multiple Incidents)**
**Pattern:** Monitoring loop detects closure multiple times, accumulates P&L
**Symptoms:** Database shows 5-20× actual P&L, duplicate Telegram notifications
**Root Cause:** Async operations + monitoring loop = race condition
**Solution:** Delete from Map IMMEDIATELY (atomic operation) before any async work
**Example Fixes:**
- Common Pitfall #49 (Nov 17, 2025)
- Common Pitfall #61 (Nov 22, 2025)
- Common Pitfall #67 (Dec 2, 2025)
### **2. Database-First Violations**
**Pattern:** In-memory state updated before database write
**Symptoms:** Container restart loses tracking, positions unprotected
**Root Cause:** Developer assumes database always succeeds
**Solution:** Always database write FIRST, then update in-memory state
**Example Fixes:**
- Common Pitfall #29 (Nov 13, 2025)
- CRITICAL_INCIDENT_UNPROTECTED_POSITION.md
### **3. External Closure Detection**
**Pattern:** Position closed on-chain, Position Manager detects late
**Symptoms:** Duplicate updates, wrong exit reasons, ghost positions
**Root Cause:** Drift state propagation delay (5-10 seconds)
**Solution:** Verification wait + closingInProgress flag + ghost detection
**Example Fixes:**
- Common Pitfall #47 (Nov 16, 2025) - Close verification gap
- Common Pitfall #56 (Nov 20, 2025) - Ghost orders
- Common Pitfall #57 (Nov 20, 2025) - P&L accuracy
### **4. Unit Conversion Errors**
**Pattern:** Tokens vs USD, percentage vs decimal, on-chain vs display units
**Symptoms:** Position sizes off by 100×+, TP/SL at wrong prices
**Root Cause:** SDK returns different units than expected
**Solution:** Always log raw values, verify units explicitly
**Example Fixes:**
- Common Pitfall #22 (Nov 12, 2025) - position.size is TOKENS not USD
- Common Pitfall #68 (Dec 3, 2025) - Signal price is percentage not USD
---
## 🔍 Debugging Checklist
**When investigating bugs:**
1. **Check logs first:**
```bash
docker logs -f trading-bot-v4 | grep -E "CRITICAL|ERROR|Failed"
```
2. **Verify deployment:**
```bash
# Container start time MUST be > commit timestamp
docker inspect trading-bot-v4 --format='{{.State.StartedAt}}'
git log -1 --format='%ai %s'
```
3. **Query database state:**
```sql
-- Check for inconsistencies
SELECT * FROM "Trade"
WHERE exitReason IS NULL
AND (createdAt < NOW() - INTERVAL '1 hour')
ORDER BY createdAt DESC;
```
4. **Compare to Drift:**
```bash
# Get actual position from Drift
curl -X GET http://localhost:3001/api/trading/positions \
-H "Authorization: Bearer $API_SECRET_KEY"
```
5. **Search Common Pitfalls:**
```bash
grep -n "symptom_keyword" .github/copilot-instructions.md | head -20
```
---
## 📝 Creating Bug Reports
**Required Sections:**
```markdown
# [Bug Title] (CRITICAL - Fixed [Date])
**Symptom:** [What user observed]
**Root Cause:** [Technical explanation]
**Real Incident ([Date]):**
* [Specific trade or event]
* [Expected behavior]
* [Actual behavior]
* [Financial impact]
**Impact:** [Scope of problem]
**Fix ([Date]):**
```typescript
// Code showing the fix
```
**Files changed:** [List of files]
**Git commit:** [Commit hash]
**Deployed:** [Deployment timestamp]
**Verification Required:**
* [How to test fix works]
* [Expected logs/behavior]
**Lessons Learned:**
1. [Key insight #1]
2. [Key insight #2]
```
**Naming Convention:**
- `CRITICAL_[TOPIC]_BUG.md` - Single critical bug
- `CRITICAL_[TOPIC]_[DATE].md` - Dated incident report
- `FIXES_[SYSTEM].md` - Multiple related fixes
---
## ⚠️ Prevention Guidelines
**From Past Incidents:**
1. **Always verify deployment before declaring "fixed"**
- Container restart timestamp > commit timestamp
- Test with actual trade if possible
- Check logs for expected behavior change
2. **Never trust SDK data formats**
- Log raw values first
- Verify units explicitly (tokens vs USD, % vs decimal)
- Check SDK docs are often wrong
3. **Database writes before in-memory updates**
- Save to database FIRST
- Only then update Position Manager, caches, etc.
- If DB fails, don't proceed
4. **Async operations need serialization**
- Delete from Map IMMEDIATELY (atomic lock)
- Don't check Map.has() then delete later (race condition)
- Use Map.delete() return value as lock
5. **External closures need verification**
- Wait 5-10 seconds for Drift state propagation
- Query Drift to confirm position actually closed
- Keep monitoring if verification fails
---
## 📞 Escalation
**When to declare "CRITICAL":**
- Financial loss occurred or was narrowly avoided
- System produced incorrect data (P&L, positions, exits)
- Position left unprotected (no monitoring, no TP/SL)
- Bug could recur and cause future losses
**Response Required:**
1. Stop trading immediately if data integrity affected
2. Document incident in this directory
3. Add to Common Pitfalls in copilot-instructions.md
4. Fix + deploy + verify within 24 hours
5. Update relevant architecture docs
---
See `../README.md` for overall documentation structure.

291
docs/cluster/README.md Normal file
View File

@@ -0,0 +1,291 @@
# Distributed Computing & EPYC Cluster
**Infrastructure for large-scale parameter optimization and backtesting.**
This directory contains documentation for the EPYC cluster setup, distributed backtesting coordination, and multi-server infrastructure.
---
## 🖥️ Cluster Documentation
### **EPYC Server Setup**
- `EPYC_SETUP_COMPREHENSIVE.md` - **Complete setup guide**
- Hardware: AMD EPYC 7282 16-Core Processor (Debian 12 Bookworm)
- Python environment: 3.11.2 with pandas 2.3.3, numpy 2.3.5
- SSH configuration: Nested hop (master → worker1 → worker2)
- Package deployment: tar.gz transfer with virtual environment
- **Status:** ✅ OPERATIONAL (24 workers processing 65,536 combos)
### **Distributed Architecture**
- `DUAL_SWEEP_README.md` - **Parallel sweep execution**
- Coordinator: Assigns chunks to workers
- Workers: Execute parameter combinations in parallel
- Database: SQLite exploration.db for state tracking
- Results: CSV files with top N configurations
- **Use case:** v9 exhaustive parameter optimization (Nov 28-29, 2025)
### **Cluster Control**
- `CLUSTER_START_BUTTON_FIX.md` - **Web UI integration**
- Dashboard: http://localhost:3001/cluster
- Start/Stop buttons with status detection
- Database-first status (SSH supplementary)
- Real-time progress tracking
- **Status:** ✅ DEPLOYED (Nov 30, 2025)
---
## 🏗️ Cluster Architecture
### **Physical Infrastructure**
```
Master Server (local development machine)
├── Coordinator Process (assigns chunks)
├── Database (exploration.db)
└── Web Dashboard (Next.js)
↓ [SSH]
Worker1 (EPYC 10.10.254.106)
├── 12 worker processes
├── 64GB RAM
└── Direct SSH connection
↓ [SSH ProxyJump]
Worker2 (EPYC 10.20.254.100)
├── 12 worker processes
├── 64GB RAM
└── Via worker1 hop
```
### **Data Flow**
```
1. Coordinator creates chunks (2,000 combos each)
↓
2. Marks chunk status='pending' in database
↓
3. Worker queries database for pending chunks
↓
4. Coordinator assigns chunk to worker via SSH
↓
5. Worker updates status='running'
↓
6. Worker processes combinations in parallel
↓
7. Worker saves results to strategies table
↓
8. Worker updates status='completed'
↓
9. Coordinator assigns next pending chunk
↓
10. Dashboard shows real-time progress
```
### **Database Schema**
```sql
-- chunks table: Work distribution
CREATE TABLE chunks (
id TEXT PRIMARY KEY, -- v9_chunk_000000
start_combo INTEGER, -- 0, 2000, 4000, etc.
end_combo INTEGER, -- 2000, 4000, 6000, etc.
status TEXT, -- 'pending', 'running', 'completed'
assigned_worker TEXT, -- 'worker1', 'worker2'
started_at INTEGER,
completed_at INTEGER
);
-- strategies table: Results storage
CREATE TABLE strategies (
id INTEGER PRIMARY KEY,
chunk_id TEXT,
params TEXT, -- JSON of parameter values
pnl REAL,
win_rate REAL,
profit_factor REAL,
max_drawdown REAL,
total_trades INTEGER
);
```
---
## 🚀 Using the Cluster
### **Starting a Sweep**
```bash
# 1. Prepare package on master
cd /home/icke/traderv4/backtester
tar -czf backtest_v9_sweep.tar.gz data/ backtester_core.py v9_moneyline_ma_gap.py moneyline_core.py
# 2. Transfer to EPYC workers
scp backtest_v9_sweep.tar.gz root@10.10.254.106:/home/backtest/
ssh root@10.10.254.106 "scp backtest_v9_sweep.tar.gz root@10.20.254.100:/home/backtest/"
# 3. Extract on workers
ssh root@10.10.254.106 "cd /home/backtest && tar -xzf backtest_v9_sweep.tar.gz"
ssh root@10.10.254.106 "ssh root@10.20.254.100 'cd /home/backtest && tar -xzf backtest_v9_sweep.tar.gz'"
# 4. Start via web dashboard or CLI
# Web: http://localhost:3001/cluster → Click "Start Cluster"
# CLI: cd /home/icke/traderv4/cluster && python v9_advanced_coordinator.py
```
### **Monitoring Progress**
```bash
# Dashboard
curl -s http://localhost:3001/api/cluster/status | jq
# Database query
sqlite3 cluster/exploration.db "SELECT status, COUNT(*) FROM chunks GROUP BY status;"
# Worker processes
ssh root@10.10.254.106 "ps aux | grep [p]ython | grep backtest | wc -l"
```
### **Collecting Results**
```bash
# Results saved to cluster/results/
ls -lh cluster/results/sweep_v9_*.csv
# Top 100 configurations by P&L
sqlite3 cluster/exploration.db "SELECT params, pnl, win_rate FROM strategies ORDER BY pnl DESC LIMIT 100;"
```
---
## 🔧 Configuration
### **Coordinator Settings**
```python
# cluster/v9_advanced_coordinator.py
WORKERS = {
'worker1': {'host': '10.10.254.106', 'port': 22},
'worker2': {'host': '10.20.254.100', 'port': 22, 'proxy_jump': '10.10.254.106'}
}
CHUNK_SIZE = 2000 # Combinations per chunk
MAX_WORKERS = 24 # 12 per server
CHECK_INTERVAL = 60 # Status check frequency (seconds)
```
### **Worker Settings**
```python
# cluster/distributed_worker.py
NUM_PROCESSES = 12 # Parallel backtests
BATCH_SIZE = 100 # Save results every N combos
TIMEOUT = 120 # Per-combo timeout (seconds)
```
### **SSH Configuration**
```bash
# ~/.ssh/config
Host worker1
HostName 10.10.254.106
User root
StrictHostKeyChecking no
ServerAliveInterval 30
Host worker2
HostName 10.20.254.100
User root
ProxyJump worker1
StrictHostKeyChecking no
ServerAliveInterval 30
```
---
## 🐛 Common Issues
### **SSH Timeout Errors**
**Symptom:** "SSH command timed out for worker2"
**Root Cause:** Nested hop requires 60s timeout (not 30s)
**Fix:** Common Pitfall #64 - Increase subprocess timeout
```python
result = subprocess.run(ssh_cmd, timeout=60) # Not 30
```
### **Database Lock Errors**
**Symptom:** "database is locked"
**Root Cause:** Multiple workers writing simultaneously
**Fix:** Use WAL mode + increase busy_timeout
```python
connection.execute('PRAGMA journal_mode=WAL')
connection.execute('PRAGMA busy_timeout=10000')
```
### **Worker Not Processing**
**Symptom:** Chunk status='running' but no worker processes
**Root Cause:** Worker crashed or SSH session died
**Fix:** Cleanup database + restart
```bash
# Mark stuck chunks as pending
sqlite3 exploration.db "UPDATE chunks SET status='pending', assigned_worker=NULL WHERE status='running' AND started_at < (strftime('%s','now') - 600);"
```
### **Status Shows "Idle" When Running**
**Symptom:** Dashboard shows idle despite workers running
**Root Cause:** SSH detection timing out, database not queried first
**Fix:** Database-first status detection (Common Pitfall #71)
```typescript
// Check database BEFORE SSH
const hasRunningChunks = explorationData.chunks.running > 0
if (hasRunningChunks) clusterStatus = 'active'
```
---
## 📊 Performance Metrics
**v9 Exhaustive Sweep (65,536 combos):**
- **Duration:** ~29 hours (24 workers)
- **Speed:** 1.60s per combo (4× faster than 6 local workers)
- **Throughput:** ~37 combos/minute across cluster
- **Data processed:** 139,678 OHLCV rows × 65,536 combos = 9.16B calculations
- **Results:** Top 100 saved to CSV (~10KB file)
**Cost Analysis:**
- Local (6 workers): 72 hours estimated
- EPYC (24 workers): 29 hours actual
- **Time savings:** 43 hours (60% faster)
- **Resource utilization:** 64 cores utilized vs 6 local
---
## 📝 Adding Cluster Features
**When to Use Cluster:**
- Parameter sweeps >10,000 combinations
- Backtests requiring >24 hours on local machine
- Multi-strategy comparison (need parallel execution)
- Production validation (test many configs simultaneously)
**Scaling Guidelines:**
- **<1,000 combos:** Local machine sufficient
- **1,000-10,000:** Single EPYC server (12 workers)
- **10,000-100,000:** Both EPYC servers (24 workers)
- **100,000+:** Consider cloud scaling (AWS Batch, etc.)
---
## ⚠️ Important Notes
**Data Transfer:**
- Always compress packages: `tar -czf` (1.9MB → 1.1MB)
- Verify checksums after transfer
- Use rsync for incremental updates
**Process Management:**
- Always use `nohup` or `screen` for long-running coordinators
- Workers auto-terminate when chunks complete
- Coordinator sends Telegram notification on completion
**Database Safety:**
- SQLite WAL mode prevents most lock errors
- Backup exploration.db before major sweeps
- Never edit chunks table manually while coordinator running
**SSH Reliability:**
- ServerAliveInterval prevents silent disconnects
- StrictHostKeyChecking=no avoids interactive prompts
- ProxyJump handles nested hops automatically
---
See `../README.md` for overall documentation structure.

314
docs/deployments/README.md Normal file
View File

@@ -0,0 +1,314 @@
# Deployment Status & Verification
**Verification reports for major feature deployments.**
This directory contains *_COMPLETE.md and DEPLOYMENT_*.md status files documenting deployment verification for significant system changes.
---
## ✅ Recent Deployments
### **December 2025**
- `DEPLOYMENT_SUCCESS_DEC3_2025.md` - **Bug fixes deployment**
- Bug #1: Revenge system external closure integration
- Bug #5: Smart Validation Queue bypass in execute endpoint
- 2 critical fixes deployed and verified
- **Status:** ✅ VERIFIED (Dec 3, 09:02 UTC)
### **November 2025**
- `V9_IMPLEMENTATION_COMPLETE.md` - **v9 Money Line indicator**
- MA Gap analysis (±5 to +15 quality points)
- Momentum-Based SHORT Filter (ADX ≥23, Position-based entry)
- Quality thresholds: LONG ≥90, SHORT ≥80
- **Status:** ✅ PRODUCTION (Nov 26, 2025)
- `PHASE_7.3_ADAPTIVE_TRAILING_DEPLOYED.md` - **Adaptive trailing stop**
- Real-time 1-minute ADX monitoring
- Dynamic multipliers: ADX acceleration/deceleration
- Combined max: 3.16× for strong trending moves
- **Status:** ✅ DEPLOYED (Nov 27, 2025)
- `ONE_YEAR_RETENTION_DEPLOYMENT.md` - **Data retention policy**
- Database: 1 year retention for Trade/PriceUpdate tables
- BlockedSignal: 90 days retention
- Automated cleanup job
- **Status:** ✅ ACTIVE
- `RUNNER_SYSTEM_FIX_COMPLETE.md` - **Runner stop loss protection**
- Fixed: No SL protection between TP1 and TP2
- Added: Explicit runner SL check in monitoring loop
- ADX-based runner SL positioning
- **Status:** ✅ VERIFIED (Nov 15, 2025)
- `REENTRY_SYSTEM_COMPLETE.md` - **Manual trade validation**
- Fresh TradingView data validation (<5min old)
- Recent performance modifiers (±20 points)
- Minimum score 55 (vs 60 for new signals)
- **Status:** ✅ OPERATIONAL
- `MA_CROSSOVER_DETECTION_COMPLETE.md` - **MA crossover tracking**
- n8n Parse Signal Enhanced detects "crossing" keyword
- Flags: isMACrossover, isDeathCross, isGoldenCross
- Purpose: Data collection for ADX weak→strong validation
- **Status:** ✅ IMPLEMENTED (Nov 27, 2025)
- `CLUSTER_STOP_BUTTON_FIX_COMPLETE.md` - **Cluster control UI**
- Database-first status detection (SSH supplementary)
- Stop button properly shown when cluster active
- Conditional rendering based on cluster status
- **Status:** ✅ DEPLOYED (Nov 30, 2025)
- `SMART_ENTRY_DEPLOYMENT_STATUS.md` - **Smart Entry Validation Queue**
- Block & Watch system for quality 50-89 signals
- ±0.3% confirmation thresholds (optimized from backtest)
- 30-second monitoring with 10-minute expiry
- **Status:** ✅ ACTIVE (Nov 30, 2025)
- `SMART_ENTRY_TIMING_STATUS.md` - **Smart Entry pullback detection**
- Waits for 0.5-2% pullback before entry
- Adaptive based on ATR and market conditions
- Improves entry prices on quality 85+ signals
- **Status:** ✅ ENABLED
- `TP1_FIX_DEPLOYMENT_SUMMARY.md` - **TP1 detection accuracy**
- Fixed: TP1 detection when on-chain orders fill fast
- Percentage-based exit reason inference (0.3-1.2% = TP1)
- Database records now accurate
- **Status:** ✅ VERIFIED (Nov 19, 2025)
- `PHASE_4_VERIFICATION_COMPLETE.md` - **Historical verification**
- Phase 4 position scaling optimizations
- TP2-as-runner system validation
- 40% runner with ATR trailing stop
- **Status:** ✅ COMPLETE
---
## 📋 Deployment Checklist Template
**For Every Major Deployment:**
### **1. Pre-Deployment**
- [ ] Code changes committed and pushed
- [ ] Unit tests pass (if applicable)
- [ ] Integration tests pass
- [ ] Documentation updated (copilot-instructions.md)
- [ ] Deployment doc created in this directory
### **2. Build & Deploy**
```bash
# Build Docker image
cd /home/icke/traderv4
docker compose build trading-bot
# Restart container
docker compose up -d --force-recreate trading-bot
# Verify container running
docker ps | grep trading-bot
```
### **3. Verification**
```bash
# 1. Check container start time > commit time
echo "=== Container Start Time ===" && \
docker inspect trading-bot-v4 --format='{{.State.StartedAt}}'
echo -e "\n=== Recent Commits ===" && \
git log --oneline --since="YYYY-MM-DD HH:MM" --format='%h %ai %s'
# 2. Verify logs show expected behavior
docker logs -f trading-bot-v4 | grep -E "expected_log_pattern"
# 3. Test API endpoints
curl http://localhost:3001/api/endpoint/path
# 4. Database verification (if schema changes)
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\\dt"
```
### **4. Post-Deployment**
- [ ] Container start time > all commit timestamps ✅
- [ ] Expected log messages appear ✅
- [ ] API endpoints respond correctly ✅
- [ ] Database schema correct (if applicable) ✅
- [ ] Test trade executed successfully (if trading logic changed) ✅
- [ ] User notified of deployment ✅
### **5. Documentation**
- [ ] Deployment doc updated with verification results
- [ ] Common Pitfalls added (if bug fix)
- [ ] Roadmap updated (if completing phase)
- [ ] Git commit for deployment verification
---
## 🚨 Critical Deployment Rules
**NEVER declare "deployed" or "working" without:**
1. ✅ Verifying container start time > commit timestamp
2. ✅ Observing expected logs in production
3. ✅ Testing API endpoint or feature behavior
4. ✅ SQL query confirming database state (if applicable)
5. ✅ Test trade validation (if Position Manager changes)
**Why This Matters:**
- This is a **real money trading system**
- Premature "working" declarations cause financial losses
- Container restart required for code changes to take effect
- Code committed ≠ Code deployed ≠ Code verified
---
## 📝 Creating Deployment Docs
**Naming Convention:**
- `[FEATURE]_COMPLETE.md` - Major feature fully implemented
- `DEPLOYMENT_SUCCESS_[DATE].md` - Multiple fixes deployed together
- `PHASE_[N]_VERIFICATION_COMPLETE.md` - Roadmap phase completion
**Required Sections:**
```markdown
# [Feature Name] - DEPLOYED ([Date])
**Purpose:** [What this deployment does]
**Status:** ✅ VERIFIED | ⏳ PENDING | ❌ FAILED
---
## Changes Deployed
### **Files Modified**
- `path/to/file1.ts` - [Description]
- `path/to/file2.ts` - [Description]
### **Key Improvements**
1. [Change 1]
2. [Change 2]
---
## Deployment Process
**Build Time:** [Time]
**Container Restart:** [Time]
**Git Commit:** [Hash] - [Message]
### **Verification Steps**
```bash
# Commands used to verify deployment
```
### **Verification Results**
- ✅ Container start time: [Time] > Commit time: [Time]
- ✅ Expected logs observed: "[Log message]"
- ✅ API test successful: [Response]
- ✅ Database state correct: [Query results]
---
## Expected Behavior
**Before Deployment:**
- [Old behavior description]
**After Deployment:**
- [New behavior description]
---
## Monitoring
**What to Watch:**
- [Log patterns to monitor]
- [Database queries to check]
- [API endpoints to test]
**Success Indicators:**
- [How to know it's working correctly]
**Failure Indicators:**
- [How to detect problems]
---
## Rollback Procedure
**If Problems Occur:**
```bash
# Revert to previous container
docker tag traderv4-trading-bot:backup traderv4-trading-bot:latest
docker compose up -d --force-recreate trading-bot
# Or git revert
git revert [commit_hash]
git push
# Then rebuild/restart
```
---
## Impact
**Expected:**
- [Benefit 1]
- [Benefit 2]
**Actual:** (Update after 24-48 hours)
- [Measured impact]
---
## Lessons Learned
1. [Key insight #1]
2. [Key insight #2]
```
---
## 🔍 Finding Deployment Info
**By Date:**
- Most recent → Check filename dates (YYYY-MM-DD or Month YYYY)
- Historical → Older files in this directory
**By Feature:**
- v9 indicator → `V9_IMPLEMENTATION_COMPLETE.md`
- Trailing stops → `PHASE_7.3_ADAPTIVE_TRAILING_DEPLOYED.md`
- Runner system → `RUNNER_SYSTEM_FIX_COMPLETE.md`
- Smart Entry → `SMART_ENTRY_*_STATUS.md`
- Cluster UI → `CLUSTER_STOP_BUTTON_FIX_COMPLETE.md`
**By Status:**
- ✅ VERIFIED → Deployment confirmed working
- ⏳ PENDING → Awaiting verification
- 🔄 IN PROGRESS → Deployment ongoing
---
## 📊 Deployment Metrics
**Success Rate:**
- Track deployments requiring rollback
- Track verification time (commit → confirmed working)
- Track time to detect issues
**Typical Timeline:**
- Code change → Commit: 5-30 minutes
- Build Docker image: 60-90 seconds
- Container restart: 5-10 seconds
- Verification: 1-5 minutes
- **Total:** 10-45 minutes from code to verified
**Best Practices:**
- Always use `--force-recreate` flag for container restart
- Wait 10 seconds after restart before testing
- Test in order: Logs → API → Database → Feature
- Document verification steps in deployment doc
- Update roadmaps after phase completions
---
See `../README.md` for overall documentation structure.

254
docs/roadmaps/README.md Normal file
View File

@@ -0,0 +1,254 @@
# Strategic Planning & Roadmaps
**Long-term strategic planning for Trading Bot v4 system enhancements.**
This directory contains all *ROADMAP*.md files documenting planned improvements across multiple optimization dimensions.
---
## 🗺️ Active Roadmaps
### **Master Roadmap**
- `OPTIMIZATION_MASTER_ROADMAP.md` - **Consolidated strategic view**
- 3 major initiatives: Signal Quality, Position Scaling, ATR-based TP
- Combined impact: 35-40% P&L improvement when complete
- Data collection phases with 50-100 trade validation requirements
- **Status:** Phase 1 (data collection) across all three initiatives
### **Signal Quality**
- `SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md` - **Data-driven quality threshold optimization**
- Phase 1: Collect 10-20 blocked signals (🔄 IN PROGRESS)
- Phase 2: Analyze patterns, adjust thresholds (🔜 NEXT)
- Phase 3: Dual-threshold system or quality-specific sizing
- Phase 4: Automated price analysis for blocked signals
- Phase 5: ML-based scoring weight optimization
- **Goal:** Recover profitable setups currently blocked, filter true losers
### **Position Scaling**
- `POSITION_SCALING_ROADMAP.md` - **TP/SL and runner optimization**
- Phase 1: ✅ COMPLETE - Data collection with quality scores
- Phase 2: ATR-based dynamic targets (adapt to volatility)
- Phase 3: Signal quality-based scaling (high quality = larger runners)
- Phase 4: Direction-based optimization (shorts vs longs)
- Phase 5: ✅ COMPLETE - TP2-as-runner system (40% runner with ATR trailing)
- Phase 6: ML-based exit prediction
- **Goal:** Capture more of winning moves, reduce stop-outs
### **ATR-Based Risk Management**
- `ATR_BASED_TP_ROADMAP.md` - **Volatility-adaptive TP/SL system**
- Phase 1: ATR data collection from TradingView
- Phase 2: ✅ COMPLETE - ATR-based TP/SL calculation (Nov 17, 2025)
- Phase 3: Validation with 50+ trades
- Phase 4: Per-symbol ATR calibration
- **Goal:** Regime-agnostic targets (bull/bear markets adapt automatically)
### **1-Minute Data Enhancements**
- `1MIN_DATA_ENHANCEMENTS_ROADMAP.md` - **High-frequency data utilization**
- Phase 7.1: Smart Entry Timing (pullback detection)
- Phase 7.2: Revenge Trading System (90s confirmation, ADX-based)
- Phase 7.3: ✅ DEPLOYED - Adaptive Trailing Stop with real-time ADX (Nov 27, 2025)
- Phase 7.4: Multi-timeframe signal validation
- **Goal:** Use 1-minute data for entry timing, trailing stop optimization
### **Indicator Evolution**
- `INDICATOR_V9_MA_GAP_ROADMAP.md` - **TradingView strategy development**
- v9 Money Line: MA Gap analysis + Momentum SHORT Filter
- Parameter optimization: 65,536 exhaustive sweep on EPYC cluster
- Quality threshold tuning: LONG ≥90, SHORT ≥95
- **Goal:** Maximum profitability while maintaining quality standards
### **High Availability**
- `HA_SETUP_ROADMAP.md` - **99.9%+ uptime infrastructure**
- Phase 1-8: ✅ ALL COMPLETE (Nov 25, 2025)
- Primary: srvdocker02 (95.216.52.28:3001)
- Secondary: Hostinger (72.62.39.24:3001)
- Database replication, DNS failover, auto-recovery
- **Status:** Zero-downtime failover validated in production
---
## 📊 Roadmap Methodology
### **Phase Structure**
All roadmaps follow consistent phase naming:
- **Phase 1:** Data collection (50-100 trades typical)
- **Phase 2:** Analysis and pattern identification
- **Phase 3:** Implementation and testing
- **Phase 4:** Validation and optimization
- **Phase 5:** Production deployment
- **Phase 6+:** Advanced features (ML, automation)
### **Status Indicators**
- ✅ **COMPLETE:** Implemented, deployed, validated with production data
- 🔄 **IN PROGRESS:** Active development or data collection
- 🔜 **NEXT:** Planned for immediate future (within 1-2 weeks)
- 🎯 **FUTURE:** Planned but not yet started
- 🤖 **DISTANT:** Long-term goal (ML, advanced features)
### **Gating Criteria**
**Before moving to next phase:**
1. **Data requirement met:** Minimum sample size achieved (50-100 trades)
2. **SQL analysis complete:** Patterns identified, validated statistically
3. **User approval:** For parameter changes affecting risk/profitability
4. **Deployment verified:** Code deployed, container restarted, logs confirmed
5. **Documentation updated:** Roadmap status, copilot-instructions.md, deployment docs
---
## 🎯 Current Focus (Dec 2025)
### **Active Data Collection**
1. **Blocked Signals:** 0/20 target (need 10-20 for threshold analysis)
2. **v9 Indicator:** 10/100 trades (parameter optimization ongoing)
3. **Adaptive Leverage:** 5/50 trades (10x vs 5x tier validation)
4. **Smart Entry Queue:** 0/30 validations (pullback confirmation system)
### **Recently Completed**
- ✅ TP2-as-runner system (40% runner, ATR trailing)
- ✅ ATR-based TP/SL (regime-agnostic targets)
- ✅ Adaptive leverage (10x high quality, 5x borderline)
- ✅ Phase 7.3 adaptive trailing (real-time ADX monitoring)
- ✅ HA setup (zero-downtime failover)
### **Next Up**
1. **Quality threshold optimization** (LONG 90→?, SHORT 95→?)
2. **v9 parameter tuning** (65,536 combo EPYC sweep results)
3. **Multi-timeframe comparison** (5min vs 15min vs 1H)
4. **Smart Entry effectiveness** (±0.3% thresholds validation)
---
## 📝 Adding New Roadmaps
**When to create roadmap:**
- Multi-phase feature (>3 phases, >1 month duration)
- Strategic initiative affecting core system behavior
- Data collection requirement (need validation before production)
- User financial goals alignment needed
**Template Structure:**
```markdown
# [Feature Name] Roadmap
**Objective:** [One-line goal]
**Expected Impact:** [Dollar amount or % improvement]
**Duration:** [Estimated timeline]
**Status:** [Current phase]
---
## Phase 1: [Name] (Status Emoji)
**Objective:** [What to achieve]
**Duration:** [Time estimate]
**Data Required:** [Sample size]
### Tasks
- [ ] Task 1
- [ ] Task 2
### Success Criteria
- [Measurable outcome]
### Gating Decision
[What determines if we proceed to Phase 2]
---
## Phase 2-N: [Continue pattern]
---
## Progress Tracking
| Phase | Status | Start Date | End Date | Key Milestone |
|-------|--------|-----------|----------|---------------|
| Phase 1 | 🔄 IN PROGRESS | YYYY-MM-DD | - | 50 trades collected |
---
## Dependencies
- [Other roadmaps or features]
## Risks
- [Technical challenges]
- [Financial risks]
## Decision Points
- [Key decisions requiring user approval]
```
---
## 🔍 Finding Roadmap Info
**By Status:**
- Active → `OPTIMIZATION_MASTER_ROADMAP.md` (consolidated view)
- Complete → Check ✅ markers in individual roadmaps
- Next up → Look for 🔜 NEXT indicators
- Long-term → 🤖 DISTANT phases
**By Feature:**
- Quality → `SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md`
- Position sizing → `POSITION_SCALING_ROADMAP.md`
- ATR system → `ATR_BASED_TP_ROADMAP.md`
- 1-min data → `1MIN_DATA_ENHANCEMENTS_ROADMAP.md`
- Indicator → `INDICATOR_V9_MA_GAP_ROADMAP.md`
- Infrastructure → `HA_SETUP_ROADMAP.md`
**By Priority:**
1. **Critical path:** OPTIMIZATION_MASTER_ROADMAP.md
2. **Active development:** Any with 🔄 IN PROGRESS
3. **User-requested:** Check git commits for recent roadmap updates
---
## 📈 Progress Tracking
**Weekly Review:**
1. Update trade counts in data collection phases
2. Check SQL for analysis readiness (sample size thresholds)
3. Move phases from 🔄 to ✅ when complete
4. Update OPTIMIZATION_MASTER_ROADMAP.md combined impact estimates
**Monthly Review:**
1. Validate expected vs actual impact (did optimization deliver promised %)
2. Adjust future phase estimates based on actual results
3. Reprioritize roadmaps based on performance data
4. Archive completed roadmaps to docs/archived/
**Milestone Commits:**
```bash
git commit -m "roadmap: Phase X complete - [Feature Name]
- Data collection: X/Y trades
- Analysis results: [Key findings]
- Deployment: [Date]
- Impact: [Actual vs expected]
Next: Phase X+1 starting [Date]
"
```
---
## ⚠️ Important Notes
**Data-Driven Only:**
- Never implement Phase 3 without Phase 2 analysis complete
- Minimum 50 trades for statistical validity
- SQL queries required to prove assumptions
**User Approval Required:**
- Quality threshold changes
- Leverage adjustments
- Risk parameter modifications
- Position sizing changes
**Real Money System:**
- Every phase affects financial outcomes
- Conservative estimates (under-promise, over-deliver)
- Document both upside and downside scenarios
- Rollback procedures for every deployment
---
See `../README.md` for overall documentation structure.

396
docs/setup/README.md Normal file
View File

@@ -0,0 +1,396 @@
# Setup Guides & Configuration
**Step-by-step guides for setting up trading bot components.**
This directory contains setup instructions for TradingView alerts, n8n workflows, external services, and infrastructure configuration.
---
## 🚀 Quick Setup Guides
### **Essential Setup (Required for Trading)**
- `SIGNAL_QUALITY_SETUP_GUIDE.md` - **Signal quality system**
- TradingView alert configuration
- Webhook payload format (ATR, ADX, RSI, volume, price position)
- n8n Parse Signal Enhanced setup
- Quality scoring thresholds
- **Status:** ✅ PRODUCTION
- `N8N_MARKET_DATA_SETUP.md` - **1-minute data collection**
- TradingView 1-minute alerts for market data cache
- Webhook endpoint: `/api/trading/market-data`
- Purpose: Smart Entry Validation Queue price updates
- Alert frequency: Every 1-5 minutes
- **Status:** ✅ ACTIVE
### **Infrastructure Setup**
- `EPYC_SETUP_COMPREHENSIVE.md` - **EPYC cluster configuration**
- SSH configuration (nested hop through worker1)
- Python environment setup (3.11.2 + pandas + numpy)
- Package deployment (tar.gz transfer)
- Worker initialization scripts
- **Location:** Also in `../cluster/` directory
- **Status:** ✅ OPERATIONAL
### **Development Setup**
- `QUICK_SETUP_CARD.md` - **Development environment**
- Docker Compose configuration
- Environment variables (.env file)
- Database initialization (PostgreSQL + Prisma)
- Local testing procedures
- **Status:** ✅ CURRENT
---
## 📋 TradingView Alert Setup
### **Production Trading Signals (5-minute chart)**
**Alert Message Format:**
```json
{
"action": "{{strategy.order.action}}",
"symbol": "{{ticker}}",
"timeframe": "{{interval}}",
"currentPrice": {{close}},
"atr": {{ta.atr(14)}},
"adx": {{ta.dmi(14, 14)}},
"rsi": {{ta.rsi(14)}},
"volumeRatio": {{volume / ta.sma(volume, 20)}},
"pricePosition": {{(close - ta.lowest(low, 100)) / (ta.highest(high, 100) - ta.lowest(low, 100)) * 100}},
"indicatorVersion": "v9"
}
```
**Alert Settings:**
- **Condition:** Strategy entry/exit
- **Trigger:** Once Per Bar Close
- **Expiration:** None (always active)
- **Webhook URL:** `https://your-domain.com/api/trading/execute`
**Required Fields:**
- ✅ `atr` - ATR-based TP/SL system (CRITICAL)
- ✅ `adx` - Trend strength filtering
- ✅ `rsi` - Momentum confirmation
- ✅ `volumeRatio` - Volume validation
- ✅ `pricePosition` - Range position (0-100%)
- ✅ `timeframe` - Distinguishes 5min (trade) vs 15min/1H (data collection)
- ✅ `indicatorVersion` - Track strategy performance (v9 current)
### **Market Data Alerts (1-minute chart)**
**Alert Message Format:**
```json
{
"action": "market_data",
"symbol": "{{ticker}}",
"timeframe": "{{interval}}",
"currentPrice": {{close}},
"atr": {{ta.atr(14)}},
"adx": {{ta.dmi(14, 14)}},
"rsi": {{ta.rsi(14)}},
"volumeRatio": {{volume / ta.sma(volume, 20)}},
"pricePosition": {{(close - ta.lowest(low, 100)) / (ta.highest(high, 100) - ta.lowest(low, 100)) * 100}}
}
```
**Alert Settings:**
- **Condition:** Time trigger (every bar close)
- **Trigger:** Once Per Bar Close
- **Expiration:** None
- **Webhook URL:** `https://your-domain.com/api/trading/market-data`
**Purpose:**
- Updates market data cache for Smart Entry Validation Queue
- Provides fresh prices for re-entry validation
- Enables sub-5-minute confirmation thresholds
---
## 🔧 n8n Workflow Setup
### **Parse Signal Enhanced**
**Location:** `workflows/trading/parse_signal_enhanced.json`
**Functions:**
1. **Extract Metrics:** ADX, ATR, RSI, volume, price position
2. **Normalize Timeframe:** "5", "15", "60", "240", "D"
3. **Extract Indicator Version:** "IND:v9" → "v9"
4. **Detect MA Crossovers:** "crossing" keyword → flags
5. **Route to Execute:** Calls `/api/trading/execute`
**Key Nodes:**
- HTTP Request node → Receives TradingView webhook
- Code node → Parses JSON payload
- Set node → Normalizes field names
- HTTP Request node → Sends to execute endpoint
### **Market Data Collector**
**Location:** `workflows/trading/market_data_collector.json`
**Functions:**
1. Receives 1-minute TradingView alerts
2. Validates symbol format (SOLUSDT → SOL-PERP)
3. Updates market data cache via `/api/trading/market-data`
4. Logs cache update timestamp
---
## 🗄️ Database Setup
### **Initial Setup (First Time)**
```bash
# 1. Start PostgreSQL container
cd /home/icke/traderv4
docker compose up -d trading-bot-postgres
# 2. Generate Prisma client
npx prisma generate
# 3. Run migrations
DATABASE_URL="postgresql://postgres:your_password@localhost:5432/trading_bot_v4" \
npx prisma migrate dev
# 4. Verify tables created
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\\dt"
```
### **Schema Updates (After Code Changes)**
```bash
# 1. Generate migration
DATABASE_URL="postgresql://...@localhost:5432/..." \
npx prisma migrate dev --name describe_changes_here
# 2. Regenerate client
npx prisma generate
# 3. Rebuild Docker image (includes new Prisma client)
docker compose build trading-bot
# 4. Restart container
docker compose up -d --force-recreate trading-bot
```
### **Database Backup**
```bash
# Backup to SQL file
docker exec trading-bot-postgres pg_dump -U postgres trading_bot_v4 > backup_$(date +%Y%m%d_%H%M%S).sql
# Restore from backup
cat backup_file.sql | docker exec -i trading-bot-postgres psql -U postgres -d trading_bot_v4
```
---
## 🔐 Environment Variables Setup
### **Required Variables (.env file)**
**Trading Configuration:**
```bash
# Drift Protocol (REQUIRED)
DRIFT_WALLET_PRIVATE_KEY="[91,24,...]" # JSON array or base58
API_SECRET_KEY="your-secret-key-here"
# Solana RPC (REQUIRED - use Helius!)
SOLANA_RPC_URL="https://mainnet.helius-rpc.com/?api-key=YOUR_KEY"
BACKUP_RPC_URL="https://api.mainnet-beta.solana.com"
# Database (REQUIRED)
DATABASE_URL="postgresql://postgres:password@trading-bot-postgres:5432/trading_bot_v4"
POSTGRES_PASSWORD="your-secure-password"
# Telegram (REQUIRED for notifications)
TELEGRAM_BOT_TOKEN="123456789:ABCdefGHIjklMNOpqrsTUVwxyz"
TELEGRAM_CHAT_ID="123456789"
```
**Position Sizing:**
```bash
# Per-Symbol Configuration
SOLANA_ENABLED=true
SOLANA_POSITION_SIZE=100 # Percentage (1-100)
SOLANA_LEVERAGE=15 # Leverage multiplier
ETHEREUM_ENABLED=false
ETHEREUM_POSITION_SIZE=100
ETHEREUM_LEVERAGE=1
# Global Fallback (BTC, etc.)
MAX_POSITION_SIZE_USD=100
LEVERAGE=1
```
**Quality Thresholds:**
```bash
MIN_SIGNAL_QUALITY_SCORE_LONG=90 # LONG signals
MIN_SIGNAL_QUALITY_SCORE_SHORT=80 # SHORT signals (more permissive)
```
**Adaptive Leverage:**
```bash
USE_ADAPTIVE_LEVERAGE=true
HIGH_QUALITY_LEVERAGE=10 # Quality ≥ threshold
LOW_QUALITY_LEVERAGE=5 # Quality < threshold
QUALITY_LEVERAGE_THRESHOLD_LONG=95 # LONG threshold
QUALITY_LEVERAGE_THRESHOLD_SHORT=90 # SHORT threshold
```
**See:** `../.github/copilot-instructions.md` Environment Variable Reference for complete list (100+ variables)
---
## 🐳 Docker Setup
### **Development (docker-compose.dev.yml)**
```bash
# Start all services
docker compose -f docker-compose.dev.yml up -d
# View logs
docker compose -f docker-compose.dev.yml logs -f trading-bot
# Stop all services
docker compose -f docker-compose.dev.yml down
```
### **Production (docker-compose.yml)**
```bash
# Build and start
docker compose up -d --build trading-bot
# Restart with force recreate
docker compose up -d --force-recreate trading-bot
# Check status
docker compose ps
docker logs -f trading-bot-v4
```
### **Telegram Bot (docker-compose.telegram-bot.yml)**
```bash
# Start Telegram bot
docker compose -f docker-compose.telegram-bot.yml up -d
# View logs
docker logs -f trading-bot-telegram
# Test command
# Send "long sol" to Telegram bot
```
---
## 🧪 Testing Setup
### **Test Trade Execution**
```bash
# Via Settings UI
# 1. Open http://localhost:3001/settings
# 2. Scroll to "Test Trading" section
# 3. Click "Test SOL LONG" or "Test SOL SHORT"
# 4. Monitor docker logs for execution
# Via Telegram Bot
# Send message: "long sol --force"
# Watch for confirmation message
```
### **Verify Position Manager**
```bash
# 1. Execute test trade
# 2. Watch logs for monitoring cycle
docker logs -f trading-bot-v4 | grep -E "Position Manager|TP1|TP2|SL"
# 3. Check database
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c \
"SELECT symbol, direction, entryPrice, currentSize, tp1Hit FROM \"Trade\" WHERE exitReason IS NULL;"
```
### **API Health Checks**
```bash
# Drift account health
curl http://localhost:3001/api/drift/account-health
# Market data cache
curl http://localhost:3001/api/trading/market-data
# Cluster status
curl http://localhost:3001/api/cluster/status
# Analytics
curl http://localhost:3001/api/analytics/last-trade
```
---
## 🛠️ Troubleshooting
### **Container Won't Start**
```bash
# Check logs for errors
docker logs trading-bot-v4
# Common issues:
# - DATABASE_URL incorrect (check connection string)
# - DRIFT_WALLET_PRIVATE_KEY format (JSON array or base58)
# - Port 3000 already in use (check with: lsof -i :3000)
```
### **TradingView Alerts Not Received**
```bash
# 1. Check n8n workflow execution log
# 2. Test webhook manually:
curl -X POST http://localhost:5678/webhook/parse-signal \
-H "Content-Type: application/json" \
-d '{"action":"buy","symbol":"SOLUSDT","timeframe":"5",...}'
# 3. Verify webhook URL in TradingView alert settings
# 4. Check firewall allows incoming webhooks
```
### **Database Connection Failed**
```bash
# 1. Verify PostgreSQL running
docker ps | grep postgres
# 2. Test connection
docker exec trading-bot-postgres psql -U postgres -c "SELECT 1"
# 3. Check DATABASE_URL matches container name
# Runtime: postgresql://postgres:pass@trading-bot-postgres:5432/...
# Prisma CLI: postgresql://postgres:pass@localhost:5432/...
```
### **RPC Rate Limiting (429 Errors)**
```bash
# Symptom: Logs show "429 Too Many Requests"
# Fix: Switch to Helius RPC (higher rate limits)
# Update .env:
SOLANA_RPC_URL="https://mainnet.helius-rpc.com/?api-key=YOUR_KEY"
# Restart container
docker compose restart trading-bot
```
---
## 📚 Additional Resources
**External Documentation:**
- Drift Protocol: https://docs.drift.trade
- Pyth Network: https://docs.pyth.network
- Helius RPC: https://docs.helius.dev
- TradingView Alerts: https://www.tradingview.com/support/solutions/43000529348
**Internal Documentation:**
- **Main README:** `../README.md`
- **Architecture:** `../architecture/` directory
- **Roadmaps:** `../roadmaps/` directory
- **Common Pitfalls:** `../.github/copilot-instructions.md`
---
See `../README.md` for overall documentation structure.