- Add new section on auto-restart loop detection and prevention - Include critical debugging commands for automation cycles - Document hardcoded recommendation anti-patterns that cause loops - Add prevention checklist for automation interference - Include order cancellation monitoring commands - Expand debugging strategies for complex automation systems Wisdom gained from resolving rapid order cancellation issue caused by auto-restart loops in position monitor system.
763 lines
27 KiB
Markdown
763 lines
27 KiB
Markdown
# GitHub Copilot Instructions for Trading Bot Development
|
||
|
||
## 🎯 Project Context & Architecture
|
||
|
||
This is an AI-powered trading automation system with advanced learning capabilities built with Next.js 15 App Router, TypeScript, Tailwind CSS, and integrated with Drift Protocol and Jupiter DEX for automated trading execution.
|
||
|
||
### Core System Components
|
||
1. **Superior Parallel Screenshot System** - 60% faster than sequential capture (71s vs 180s)
|
||
2. **AI Learning System** - Adapts trading decisions based on outcomes with pattern recognition
|
||
3. **Orphaned Order Cleanup** - Automatic cleanup when positions close via position monitor
|
||
4. **Position Monitoring** - Frequent checks with integrated cleanup triggers
|
||
5. **Dual-Session Screenshot Automation** - AI and DIY layouts with session persistence
|
||
6. **Robust Cleanup System** - Prevents Chromium process accumulation
|
||
|
||
### Critical File Relationships
|
||
```
|
||
app/api/automation/position-monitor/route.js → Monitors positions + triggers cleanup
|
||
lib/simplified-stop-loss-learner.js → AI learning core with pattern recognition
|
||
lib/superior-screenshot-service.ts → Parallel screenshot capture system
|
||
lib/enhanced-autonomous-risk-manager.js → Risk management with AI integration
|
||
lib/enhanced-screenshot-robust.ts → Guaranteed cleanup with finally blocks
|
||
lib/automated-cleanup-service.ts → Background process monitoring
|
||
```
|
||
|
||
## 🚀 Development Environment (Critical)
|
||
|
||
### Docker Container Development (Required)
|
||
**All development happens inside Docker containers** using Docker Compose v2. Browser automation requires specific system dependencies only available in containerized environment.
|
||
|
||
**IMPORTANT: Use Docker Compose v2 syntax** - All commands use `docker compose` (with space) instead of `docker-compose` (with hyphen).
|
||
|
||
```bash
|
||
# Development environment - Docker Compose v2 dev setup
|
||
npm run docker:dev # Port 9001:3000, hot reload, debug mode
|
||
# Direct v2 command: docker compose -f docker-compose.dev.yml up --build
|
||
|
||
# Production environment
|
||
npm run docker:up # Port 9000:3000, optimized build
|
||
# Direct v2 command: docker compose -f docker-compose.prod.yml up --build
|
||
|
||
# Debugging commands
|
||
npm run docker:logs # View container logs
|
||
npm run docker:exec # Shell access for debugging inside container
|
||
```
|
||
|
||
**Port Configuration:**
|
||
- **Development**: External port `9001` → Internal port `3000` (http://localhost:9001)
|
||
- **Production**: External port `9000` → Internal port `3000` (http://localhost:9000)
|
||
|
||
### Container-First Development Workflow
|
||
**Common Issue**: File edits not reflecting in container due to volume mount sync issues.
|
||
|
||
**Solution - Container Development Workflow:**
|
||
```bash
|
||
# 1. Access running container for immediate edits
|
||
docker compose -f docker-compose.dev.yml exec app bash
|
||
|
||
# 2. Edit files directly in container (immediate effect)
|
||
nano /app/lib/enhanced-screenshot.ts
|
||
echo "console.log('Debug: immediate test');" >> /app/debug.js
|
||
|
||
# 3. Test changes immediately (no rebuild needed)
|
||
# Changes take effect instantly for hot reload
|
||
|
||
# 4. Once everything works, copy changes back to host
|
||
docker cp container_name:/app/modified-file.js ./modified-file.js
|
||
|
||
# 5. Commit successful changes to git BEFORE rebuilding
|
||
git add .
|
||
git commit -m "feat: implement working solution for [specific feature]"
|
||
git push origin development
|
||
|
||
# 6. Rebuild container for persistence
|
||
docker compose -f docker-compose.dev.yml down
|
||
docker compose -f docker-compose.dev.yml up --build -d
|
||
|
||
# 7. Final validation and commit completion
|
||
curl http://localhost:9001 # Verify functionality
|
||
git add . && git commit -m "chore: confirm container persistence" && git push
|
||
```
|
||
|
||
### Git Branch Strategy (Required)
|
||
**Primary development workflow:**
|
||
- **`development` branch**: Use for all active development and feature work
|
||
- **`main` branch**: Stable, production-ready code only
|
||
- **Workflow**: Develop on `development` → test thoroughly → commit progress → merge to `main` when stable
|
||
|
||
```bash
|
||
# Standard development workflow with frequent commits
|
||
git checkout development # Always start here
|
||
git pull origin development # Get latest changes
|
||
|
||
# Make your changes and test in container...
|
||
|
||
# Commit working progress BEFORE rebuilding container
|
||
git add .
|
||
git commit -m "feat: [specific achievement] - tested and working"
|
||
git push origin development
|
||
|
||
# After successful container rebuild and validation
|
||
git add .
|
||
git commit -m "chore: confirm [feature] persistence after rebuild"
|
||
git push origin development
|
||
|
||
# Only merge to main when features are stable and tested
|
||
git checkout main
|
||
git merge development # When ready for production
|
||
git push origin main
|
||
```
|
||
|
||
## 🏗️ System Architecture
|
||
|
||
### Dual-Session Screenshot Automation
|
||
- **AI Layout**: `Z1TzpUrf` - RSI (top), EMAs, MACD (bottom)
|
||
- **DIY Layout**: `vWVvjLhP` - Stochastic RSI (top), VWAP, OBV (bottom)
|
||
- Parallel browser sessions for multi-layout capture in `lib/enhanced-screenshot.ts`
|
||
- TradingView automation with session persistence in `lib/tradingview-automation.ts`
|
||
- Session data stored in `.tradingview-session/` volume mount to avoid captchas
|
||
|
||
### AI Analysis Pipeline
|
||
- OpenAI GPT-4o mini for cost-effective chart analysis (~$0.006 per analysis)
|
||
- Multi-layout comparison and consensus detection in `lib/ai-analysis.ts`
|
||
- Professional trading setups with exact entry/exit levels and risk management
|
||
- Layout-specific indicator analysis (RSI vs Stochastic RSI, MACD vs OBV)
|
||
|
||
### Trading Integration
|
||
- **Drift Protocol**: Perpetual futures trading via `@drift-labs/sdk`
|
||
- **Jupiter DEX**: Spot trading on Solana
|
||
- Position management and P&L tracking in `lib/drift-trading-final.ts`
|
||
- Real-time account balance and collateral monitoring
|
||
|
||
### Browser Process Management & Cleanup System
|
||
**Critical Issue**: Chromium processes accumulate during automated trading, consuming system resources over time.
|
||
|
||
**Robust Cleanup Implementation:**
|
||
1. **Enhanced Screenshot Service** (`lib/enhanced-screenshot-robust.ts`)
|
||
- Guaranteed cleanup via `finally` blocks in all browser operations
|
||
- Active session tracking to prevent orphaned browsers
|
||
- Session cleanup tasks array for systematic teardown
|
||
|
||
2. **Automated Cleanup Service** (`lib/automated-cleanup-service.ts`)
|
||
- Background monitoring service for orphaned processes
|
||
- Multiple kill strategies: graceful → force → system cleanup
|
||
- Periodic cleanup of temporary files and browser data
|
||
|
||
3. **Aggressive Cleanup Utilities** (`lib/aggressive-cleanup.ts`)
|
||
- System-level process killing for stubborn Chromium processes
|
||
- Port cleanup and temporary directory management
|
||
- Emergency cleanup functions for resource recovery
|
||
|
||
**Implementation Patterns:**
|
||
```typescript
|
||
// Always use finally blocks for guaranteed cleanup
|
||
try {
|
||
const browser = await puppeteer.launch(options);
|
||
// ... browser operations
|
||
} finally {
|
||
// Guaranteed cleanup regardless of success/failure
|
||
await ensureBrowserCleanup(browser, sessionId);
|
||
await cleanupSessionTasks(sessionId);
|
||
}
|
||
|
||
// Background monitoring for long-running operations
|
||
const cleanupService = new AutomatedCleanupService();
|
||
cleanupService.startPeriodicCleanup(); // Every 10 minutes
|
||
```
|
||
|
||
|
||
### API Route Structure
|
||
All core functionality exposed via Next.js API routes:
|
||
```typescript
|
||
// Enhanced screenshot with progress tracking and robust cleanup
|
||
POST /api/enhanced-screenshot
|
||
{
|
||
symbol: "SOLUSD",
|
||
timeframe: "240",
|
||
layouts: ["ai", "diy"],
|
||
analyze: true
|
||
}
|
||
// Returns: { screenshots, analysis, sessionId }
|
||
// Note: Includes automatic Chromium process cleanup via finally blocks
|
||
|
||
// Drift trading endpoints
|
||
GET /api/balance # Account balance/collateral
|
||
POST /api/trading # Execute trades
|
||
GET /api/status # Trading status
|
||
GET /api/automation/position-monitor # Position monitoring with orphaned cleanup
|
||
POST /api/drift/cleanup-orders # Manual order cleanup
|
||
```
|
||
|
||
### Progress Tracking System
|
||
Real-time operation tracking for long-running tasks:
|
||
- `lib/progress-tracker.ts` manages EventEmitter-based progress
|
||
- SessionId-based tracking for multi-step operations
|
||
- Steps: init → auth → navigation → loading → capture → analysis
|
||
- Stream endpoint: `/api/progress/[sessionId]/stream`
|
||
|
||
### Page Structure & Multi-Timeframe Implementation
|
||
- `app/analysis/page.js` - Original analysis page with multi-timeframe functionality
|
||
- `app/automation/page.js` - Original automation page (legacy, may have issues)
|
||
- `app/automation-v2/page.js` - **NEW**: Clean automation page with full multi-timeframe support
|
||
- `app/automation/page-v2.js` - Alternative implementation, same functionality as automation-v2
|
||
|
||
**Multi-Timeframe Architecture Pattern:**
|
||
```javascript
|
||
// Standard timeframes array - use this exact format
|
||
const timeframes = ['5m', '15m', '30m', '1h', '2h', '4h', '1d'];
|
||
|
||
// State management for multi-timeframe selection
|
||
const [selectedTimeframes, setSelectedTimeframes] = useState(['1h', '4h']);
|
||
|
||
// Toggle function with proper array handling
|
||
const toggleTimeframe = (tf) => {
|
||
setSelectedTimeframes(prev =>
|
||
prev.includes(tf)
|
||
? prev.filter(t => t !== tf) // Remove if selected
|
||
: [...prev, tf] // Add if not selected
|
||
);
|
||
};
|
||
|
||
// Preset configurations for trading styles
|
||
const presets = {
|
||
scalping: ['5m', '15m', '1h'],
|
||
day: ['1h', '4h', '1d'],
|
||
swing: ['4h', '1d']
|
||
};
|
||
```
|
||
|
||
### Component Architecture
|
||
- `app/layout.js` - Root layout with gradient styling and navigation
|
||
- `components/Navigation.tsx` - Multi-page navigation system
|
||
- `components/AIAnalysisPanel.tsx` - Multi-timeframe analysis interface
|
||
- `components/Dashboard.tsx` - Main trading dashboard with real Drift positions
|
||
- `components/AdvancedTradingPanel.tsx` - Drift Protocol trading interface
|
||
|
||
Critical timeframe handling to avoid TradingView confusion:
|
||
```typescript
|
||
// ALWAYS use minute values first, then alternatives
|
||
'4h': ['240', '240m', '4h', '4H'] // 240 minutes FIRST
|
||
'1h': ['60', '60m', '1h', '1H'] // 60 minutes FIRST
|
||
'15m': ['15', '15m']
|
||
```
|
||
|
||
Layout URL mappings for direct navigation:
|
||
```typescript
|
||
const LAYOUT_URLS = {
|
||
'ai': 'Z1TzpUrf', // RSI + EMAs + MACD
|
||
'diy': 'vWVvjLhP' // Stochastic RSI + VWAP + OBV
|
||
}
|
||
```
|
||
|
||
## 🧠 AI Learning System Patterns
|
||
```javascript
|
||
async generateLearningReport() {
|
||
// Return comprehensive learning status
|
||
return {
|
||
summary: { totalDecisions, systemConfidence, successRate },
|
||
insights: { thresholds, confidenceLevel },
|
||
recommendations: []
|
||
};
|
||
}
|
||
|
||
async getSmartRecommendation(requestData) {
|
||
// Analyze patterns and return AI recommendation
|
||
const { distanceFromSL, symbol, marketConditions } = requestData;
|
||
// Return: { action, confidence, reasoning }
|
||
}
|
||
|
||
async recordDecision(decisionData) {
|
||
// Log decision for learning with unique ID
|
||
const id = `decision_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
|
||
// Store in database for pattern analysis
|
||
}
|
||
|
||
async assessDecisionOutcome(outcomeData) {
|
||
// Update decision with actual result for learning
|
||
// Calculate if decision was correct based on outcome
|
||
}
|
||
```
|
||
|
||
### Database Operations Best Practices:
|
||
```javascript
|
||
// ALWAYS provide unique IDs for Prisma records
|
||
await prisma.ai_learning_data.create({
|
||
data: {
|
||
id: `${prefix}_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`,
|
||
// ... other fields
|
||
}
|
||
});
|
||
|
||
// Use correct import path
|
||
const { getDB } = require('./db'); // NOT './database-util'
|
||
```
|
||
|
||
### Prisma Table Name Debugging (Critical)
|
||
**Problem**: Database queries fail with "Cannot read properties of undefined (reading 'findMany')" despite correct syntax.
|
||
|
||
**Root Cause**: Prisma model names vs database table names can differ, causing silent failures that fall back to alternative APIs (like Binance instead of CoinGecko).
|
||
|
||
**Common Issues**:
|
||
```javascript
|
||
// ❌ Wrong - will cause undefined errors
|
||
await prisma.trade.findMany() // Should be 'trades'
|
||
await prisma.automationSession.findMany() // Should be 'automation_sessions'
|
||
|
||
// ✅ Correct - matches actual database schema
|
||
await prisma.trades.findMany()
|
||
await prisma.automation_sessions.findMany()
|
||
```
|
||
|
||
**Debugging Steps**:
|
||
1. **Check Prisma schema** (`prisma/schema.prisma`) for actual model names
|
||
2. **Verify table names** in database: `PRAGMA table_info(trades);`
|
||
3. **Test queries directly**: `node -e "const prisma = new PrismaClient(); prisma.trades.findMany().then(console.log);"`
|
||
4. **Look for fallback behavior** - API might silently use backup data sources when DB fails
|
||
5. **Monitor logs for price source errors** - DB failures often cause price fetching fallbacks
|
||
|
||
**Impact**: Database errors can cause price monitors to fail and fall back to wrong price sources (Binance instead of CoinGecko), affecting trading accuracy.
|
||
|
||
### Always Include These Functions in Learning Classes:
|
||
```javascript
|
||
async generateLearningReport() {
|
||
// Return comprehensive learning status
|
||
return {
|
||
summary: { totalDecisions, systemConfidence, successRate },
|
||
insights: { thresholds, confidenceLevel },
|
||
recommendations: []
|
||
};
|
||
}
|
||
|
||
async getSmartRecommendation(requestData) {
|
||
// Analyze patterns and return AI recommendation
|
||
const { distanceFromSL, symbol, marketConditions } = requestData;
|
||
// Return: { action, confidence, reasoning }
|
||
}
|
||
|
||
async recordDecision(decisionData) {
|
||
// Log decision for learning with unique ID
|
||
const id = `decision_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
|
||
// Store in database for pattern analysis
|
||
}
|
||
|
||
async assessDecisionOutcome(outcomeData) {
|
||
// Update decision with actual result for learning
|
||
// Calculate if decision was correct based on outcome
|
||
}
|
||
```
|
||
|
||
### Database Operations Best Practices:
|
||
```javascript
|
||
// ALWAYS provide unique IDs for Prisma records
|
||
await prisma.ai_learning_data.create({
|
||
data: {
|
||
id: `${prefix}_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`,
|
||
// ... other fields
|
||
}
|
||
});
|
||
|
||
// Use correct import path
|
||
const { getDB } = require('./db'); // NOT './database-util'
|
||
```
|
||
|
||
## 🔧 Error Handling Patterns
|
||
|
||
## 🔧 Error Handling Patterns
|
||
|
||
### Function Existence Checks:
|
||
```javascript
|
||
// Always check if functions exist before calling
|
||
if (typeof this.learner.generateLearningReport === 'function') {
|
||
const report = await this.learner.generateLearningReport();
|
||
} else {
|
||
// Fallback to alternative method
|
||
const status = await this.learner.getLearningStatus();
|
||
}
|
||
```
|
||
|
||
### Comprehensive Try-Catch:
|
||
```javascript
|
||
try {
|
||
const result = await aiFunction();
|
||
return result;
|
||
} catch (error) {
|
||
await this.log(`❌ AI function error: ${error.message}`);
|
||
return fallbackResult(); // Always provide fallback
|
||
}
|
||
```
|
||
|
||
## 📊 Integration Patterns
|
||
```javascript
|
||
// Always check if functions exist before calling
|
||
if (typeof this.learner.generateLearningReport === 'function') {
|
||
const report = await this.learner.generateLearningReport();
|
||
} else {
|
||
// Fallback to alternative method
|
||
const status = await this.learner.getLearningStatus();
|
||
}
|
||
```
|
||
|
||
### Comprehensive Try-Catch:
|
||
```javascript
|
||
try {
|
||
const result = await aiFunction();
|
||
return result;
|
||
} catch (error) {
|
||
await this.log(`❌ AI function error: ${error.message}`);
|
||
return fallbackResult(); // Always provide fallback
|
||
}
|
||
```
|
||
|
||
## 📊 Integration Patterns
|
||
|
||
### Position Monitor Integration:
|
||
```javascript
|
||
// When no position detected, check for orphaned orders
|
||
if (!result.hasPosition) {
|
||
console.log('📋 No active positions detected - checking for orphaned orders...');
|
||
|
||
try {
|
||
const ordersResponse = await fetch(`${baseUrl}/api/drift/orders`);
|
||
if (ordersResponse.ok) {
|
||
const ordersData = await ordersResponse.json();
|
||
if (ordersData.orders?.length > 0) {
|
||
// Trigger cleanup
|
||
const cleanupResponse = await fetch(`${baseUrl}/api/drift/cleanup-orders`, {
|
||
method: 'POST'
|
||
});
|
||
// Handle cleanup result
|
||
}
|
||
}
|
||
} catch (error) {
|
||
// Handle error gracefully
|
||
}
|
||
}
|
||
```
|
||
|
||
### Parallel Processing for Screenshots:
|
||
```javascript
|
||
// Use Promise.allSettled for parallel processing
|
||
const promises = timeframes.map(timeframe =>
|
||
captureTimeframe(timeframe, symbol, layoutType)
|
||
);
|
||
const results = await Promise.allSettled(promises);
|
||
|
||
// Process results with error isolation
|
||
results.forEach((result, index) => {
|
||
if (result.status === 'fulfilled') {
|
||
// Handle success
|
||
} else {
|
||
// Handle individual failure without breaking others
|
||
}
|
||
});
|
||
```
|
||
|
||
## 🎯 Performance Optimization Rules
|
||
|
||
### Screenshot Capture:
|
||
- Always use parallel processing for multiple timeframes
|
||
- Reuse browser sessions to avoid login/captcha
|
||
- Isolate errors so one failure doesn't break others
|
||
- Prefer `Promise.allSettled` over `Promise.all`
|
||
|
||
### Database Queries:
|
||
- Use indexed fields for frequent searches (symbol, createdAt)
|
||
- Batch operations when possible
|
||
- Include proper error handling for connection issues
|
||
|
||
### Container Optimization:
|
||
- Check syntax before deployment: `node -c filename.js`
|
||
- Use health checks for monitoring
|
||
- Implement graceful shutdown handling
|
||
|
||
## 🧪 Testing Requirements
|
||
|
||
### Always Include These Tests:
|
||
```javascript
|
||
// Test AI learning functions
|
||
const learner = new SimplifiedStopLossLearner();
|
||
const report = await learner.generateLearningReport();
|
||
console.log('Learning report:', report.summary);
|
||
|
||
// Test API endpoints
|
||
const response = await fetch('/api/automation/position-monitor');
|
||
const result = await response.json();
|
||
console.log('Position monitor working:', result.success);
|
||
|
||
// Test error scenarios
|
||
try {
|
||
await riskyFunction();
|
||
} catch (error) {
|
||
console.log('Error handling working:', error.message);
|
||
}
|
||
```
|
||
|
||
## 🎨 UI/UX Patterns
|
||
|
||
### Preset Configuration:
|
||
```javascript
|
||
// Frontend presets MUST match backend exactly
|
||
const TRADING_PRESETS = {
|
||
scalp: ['5m', '15m', '30m'], // NOT ['5m', '15m', '1h']
|
||
day: ['1h', '2h'], // NOT ['1h', '4h', '1d']
|
||
swing: ['4h', '1D'],
|
||
extended: ['1m', '3m', '5m', '15m', '30m', '1h', '4h', '1D']
|
||
};
|
||
```
|
||
|
||
### Status Display:
|
||
```javascript
|
||
// Always provide detailed feedback
|
||
return {
|
||
success: true,
|
||
monitor: {
|
||
hasPosition: false,
|
||
orphanedOrderCleanup: {
|
||
triggered: true,
|
||
success: true,
|
||
message: 'Cleaned up 2 orphaned orders',
|
||
summary: { totalCanceled: 2 }
|
||
}
|
||
}
|
||
};
|
||
```
|
||
|
||
## 🔍 Debugging Strategies
|
||
|
||
### Container Issues:
|
||
```bash
|
||
# Check for syntax errors
|
||
find . -name "*.js" -exec node -c {} \;
|
||
|
||
# Monitor logs for patterns
|
||
docker logs trader_dev --since="1m" | grep -E "(Error|unhandled|crash)"
|
||
|
||
# Test specific components
|
||
node test-learning-system.js
|
||
```
|
||
|
||
### Integration Issues:
|
||
```bash
|
||
# Test API endpoints individually
|
||
curl -s http://localhost:9001/api/automation/position-monitor | jq .
|
||
|
||
# Verify database connectivity
|
||
node -e "const {getDB} = require('./lib/db'); getDB().then(() => console.log('DB OK'));"
|
||
```
|
||
|
||
### Automation Loop Debugging:
|
||
```bash
|
||
# Track automation cycles and recommendations
|
||
docker logs trader_dev --since="5m" | grep -E "(AUTO-RESTART|recommendation|CYCLE)" | tail -10
|
||
|
||
# Monitor order behavior patterns
|
||
curl -s http://localhost:9001/api/drift/orders | jq '.orders | map(select(.status == "CANCELED")) | length'
|
||
|
||
# Check if position detection is working
|
||
curl -s http://localhost:9001/api/drift/positions | jq '.positions | length'
|
||
|
||
# Verify cleanup operations
|
||
curl -s http://localhost:9001/api/automation/position-monitor | jq '.monitor.orphanedOrderCleanup'
|
||
```
|
||
|
||
### Docker Volume Mount Debugging (Critical Learning)
|
||
**Problem**: Code changes don't reflect in container, or container has different file content than host.
|
||
|
||
**Root Cause**: Volume mounts in `docker-compose.dev.yml` synchronize host directories to container:
|
||
```yaml
|
||
volumes:
|
||
- ./app:/app/app:cached # Host ./app → Container /app/app
|
||
- ./lib:/app/lib:cached # Host ./lib → Container /app/lib
|
||
```
|
||
|
||
**Debugging Workflow**:
|
||
```bash
|
||
# 1. Always check what's actually in the container vs host
|
||
docker compose -f docker-compose.dev.yml exec app bash -c "grep -n 'problem_pattern' /app/app/api/file.js"
|
||
grep -n 'problem_pattern' app/api/file.js
|
||
|
||
# 2. Compare file checksums to verify sync
|
||
sha256sum app/api/file.js
|
||
docker compose -f docker-compose.dev.yml exec app bash -c "sha256sum /app/app/api/file.js"
|
||
|
||
# 3. Check if Next.js compiled cache is stale
|
||
docker compose -f docker-compose.dev.yml exec app bash -c "ls -la /app/.next/server/"
|
||
docker compose -f docker-compose.dev.yml exec app bash -c "grep -r 'problem_pattern' /app/.next/server/ || echo 'Not in compiled cache'"
|
||
|
||
# 4. Clear Next.js cache if files match but behavior doesn't
|
||
docker compose -f docker-compose.dev.yml exec app bash -c "rm -rf /app/.next"
|
||
docker compose -f docker-compose.dev.yml restart
|
||
```
|
||
|
||
**Key Insights**:
|
||
- **Host edits sync to container** automatically via volume mounts
|
||
- **Container file copies are overwritten** by volume mounts on restart
|
||
- **Next.js compilation cache** in `.next/` can persist old code even after file changes
|
||
- **Always verify container content** matches expectations before debugging logic
|
||
- **Compiled webpack bundles** may differ from source files - check both
|
||
|
||
**Troubleshooting Steps**:
|
||
1. **Verify host file has expected changes** (`grep`, `cat`, `sed -n '90,100p'`)
|
||
2. **Confirm container file matches host** (checksums, direct comparison)
|
||
3. **Check if .next cache is stale** (search compiled files for old patterns)
|
||
4. **Clear compilation cache and restart** if source is correct but behavior wrong
|
||
5. **Use container logs to trace actual execution** vs expected code paths
|
||
|
||
## <20> Automation Interference Patterns (Critical Learning)
|
||
|
||
### Auto-Restart Loop Detection & Prevention
|
||
**Problem Pattern**: Position monitors with hardcoded "START_TRADING" recommendations create infinite restart loops when no positions are detected, causing rapid order cancellations.
|
||
|
||
**Root Cause Symptoms**:
|
||
```bash
|
||
# Log patterns indicating auto-restart loops
|
||
docker logs trader_dev | grep "AUTO-RESTART.*START_TRADING"
|
||
docker logs trader_dev | grep "No position detected.*recommendation"
|
||
docker logs trader_dev | grep "triggering auto-restart"
|
||
```
|
||
|
||
**Detection Commands**:
|
||
```bash
|
||
# Check for restart loop patterns
|
||
docker logs trader_dev --since="10m" | grep -E "(CYCLE|recommendation|AUTOMATION)" | tail -15
|
||
|
||
# Monitor order cancellation frequency
|
||
curl -s http://localhost:9001/api/drift/orders | jq '.orders | map(select(.status == "CANCELED")) | length'
|
||
|
||
# Check position monitor behavior
|
||
curl -s http://localhost:9001/api/automation/position-monitor | jq '.monitor.recommendation'
|
||
```
|
||
|
||
**Solution Pattern**:
|
||
```javascript
|
||
// ❌ WRONG: Hardcoded recommendation causes loops
|
||
const result = {
|
||
recommendation: 'START_TRADING', // Always triggers restart
|
||
hasPosition: false // When combined, creates infinite loop
|
||
};
|
||
|
||
// ✅ CORRECT: Context-aware recommendations
|
||
const result = {
|
||
recommendation: hasPosition ? 'MONITOR_POSITION' : 'MONITOR_ONLY',
|
||
hasPosition: false // Safe - no auto-restart trigger
|
||
};
|
||
|
||
// ✅ CORRECT: Disable auto-restart entirely for manual control
|
||
/* Auto-restart logic disabled to prevent interference with manual trading */
|
||
```
|
||
|
||
**Prevention Checklist**:
|
||
- [ ] Position monitor recommendations are context-aware, not hardcoded
|
||
- [ ] Auto-restart logic includes manual override capabilities
|
||
- [ ] Order placement doesn't trigger immediate cleanup cycles
|
||
- [ ] System allows manual trading without automation interference
|
||
- [ ] Logs show clean monitoring without constant restart attempts
|
||
|
||
## <20>🚨 Critical Anti-Patterns to Avoid
|
||
|
||
### ❌ Don't Do This:
|
||
```javascript
|
||
// Missing error handling
|
||
const report = await this.learner.generateLearningReport(); // Will crash if function missing
|
||
|
||
// Redundant polling
|
||
setInterval(checkOrders, 60000); // When position monitor already runs frequently
|
||
|
||
// Auto-restart loops that interfere with trading
|
||
recommendation: 'START_TRADING', // Hardcoded - causes constant restart triggers
|
||
if (!hasPosition && recommendation === 'START_TRADING') {
|
||
// Auto-restart logic that triggers rapid cleanup cycles
|
||
}
|
||
|
||
// Frontend/backend preset mismatch
|
||
backend: ['5m', '15m', '1h']
|
||
frontend: ['5m', '15m', '30m'] // Will cause confusion
|
||
|
||
// Missing unique IDs
|
||
await prisma.create({ data: { symbol, timeframe } }); // Will fail validation
|
||
```
|
||
|
||
### ✅ Do This Instead:
|
||
```javascript
|
||
// Defensive programming
|
||
if (typeof this.learner.generateLearningReport === 'function') {
|
||
try {
|
||
const report = await this.learner.generateLearningReport();
|
||
} catch (error) {
|
||
await this.log(`Report generation failed: ${error.message}`);
|
||
}
|
||
}
|
||
|
||
// Leverage existing infrastructure
|
||
// Add cleanup to existing position monitor instead of new polling
|
||
|
||
// Smart recommendations that don't trigger loops
|
||
recommendation: hasPosition ? 'MONITOR_POSITION' : 'MONITOR_ONLY', // Context-aware
|
||
// Disable auto-restart for manual control
|
||
/* Auto-restart logic disabled to prevent interference */
|
||
|
||
// Ensure consistency
|
||
const PRESETS = { scalp: ['5m', '15m', '30m'] }; // Same in frontend and backend
|
||
|
||
// Always provide unique IDs
|
||
const id = `${type}_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
|
||
```
|
||
|
||
## 🎯 Configuration Standards
|
||
|
||
### Environment Variables:
|
||
```javascript
|
||
// Always provide fallbacks
|
||
const apiKey = process.env.OPENAI_API_KEY || '';
|
||
if (!apiKey) {
|
||
throw new Error('OPENAI_API_KEY is required');
|
||
}
|
||
```
|
||
|
||
### Next.js Configuration:
|
||
```typescript
|
||
// Use new format, not deprecated
|
||
const nextConfig: NextConfig = {
|
||
serverExternalPackages: ['puppeteer-core'], // NOT experimental.serverComponentsExternalPackages
|
||
transpilePackages: ['next-font'],
|
||
typescript: { ignoreBuildErrors: true },
|
||
eslint: { ignoreDuringBuilds: true }
|
||
};
|
||
```
|
||
|
||
## 📈 Enhancement Guidelines
|
||
|
||
When adding new features:
|
||
|
||
1. **Check Existing Infrastructure** - Can it be integrated vs creating new?
|
||
2. **Add Comprehensive Error Handling** - Assume functions may not exist
|
||
3. **Include Fallback Mechanisms** - System should work without AI/learning
|
||
4. **Test in Isolation** - Create test scripts for new components
|
||
5. **Document Integration Points** - How does it connect to existing systems?
|
||
6. **Maintain Consistency** - Frontend and backend must match exactly
|
||
7. **Use Defensive Programming** - Check before calling, handle gracefully
|
||
|
||
## 📚 Documentation References
|
||
|
||
### Technical Documentation
|
||
- **`ADVANCED_SYSTEM_KNOWLEDGE.md`** - Deep technical architecture, session management, cleanup systems
|
||
- **`README.md`** - Main project overview with current feature status and setup
|
||
- **`AI_LEARNING_EXPLAINED.md`** - AI learning system implementation details
|
||
- **`DRIFT_FEEDBACK_LOOP_COMPLETE.md`** - Complete Drift trading integration
|
||
- **`ROBUST_CLEANUP_IMPLEMENTATION.md`** - Browser process cleanup system details
|
||
|
||
### Implementation Guides
|
||
- **`MULTI_LAYOUT_IMPLEMENTATION.md`** - Dual-session screenshot system
|
||
- **`SESSION_PERSISTENCE.md`** - TradingView session management
|
||
- **`DOCKER_AUTOMATION.md`** - Container development workflow
|
||
- **`DEVELOPMENT_GUIDE.md`** - Complete development setup instructions
|
||
|
||
### Analysis & Troubleshooting
|
||
- **`MULTI_LAYOUT_TROUBLESHOOTING.md`** - Screenshot automation debugging
|
||
- **`CLEANUP_IMPROVEMENTS.md`** - Process management enhancements
|
||
- **`SCREENSHOT_PATH_FIXES.md`** - Screenshot capture issue resolution
|
||
|
||
---
|
||
|
||
**Follow these patterns to maintain system stability and avoid the complex debugging issues that were resolved in this session.**
|