From 11aec95d473d4c39a7512668af61c2d1df9cc621 Mon Sep 17 00:00:00 2001
From: mindesbunister <github_service@egonetix.de>
Date: Sat, 26 Jul 2025 15:19:36 +0200
Subject: [PATCH] docs: consolidate copilot instructions into single
 comprehensive file

- Merged duplicate .github/copilot-instructions.instructions.md into main copilot-instructions.md
- Combined development patterns, architecture details, and AI learning system docs
- Added comprehensive references to all technical documentation files
- Single source of truth for GitHub Copilot development guidance
- Includes Docker workflow, cleanup systems, error handling patterns
---
 .github/copilot-instructions.instructions.md | 589 -------------------
 .github/copilot-instructions.md              | 324 +++++++++-
 2 files changed, 320 insertions(+), 593 deletions(-)
 delete mode 100644 .github/copilot-instructions.instructions.md

diff --git a/.github/copilot-instructions.instructions.md b/.github/copilot-instructions.instructions.md
deleted file mode 100644
index c3c5fa7..0000000
--- a/.github/copilot-instructions.instructions.md
+++ /dev/null
@@ -1,589 +0,0 @@
-# AI-Powered Trading Bot Dashboard
-
-This is a Next.js 15 App Router application with TypeScript, Tailwind CSS, and API routes. It's a production-ready trading bot with AI analysis, automated screenshot capture, and real-time trading execution via Drift Protocol and Jupiter DEX.
-
-**Prerequisites:**
-- Docker and Docker Compose v2 (uses `docker compose` command syntax)
-- All development must be done inside Docker containers for browser automation compatibility
-
-## Core Architecture
-
-### Dual-Session Screenshot Automation
-- **AI Layout**: `Z1TzpUrf` - RSI (top), EMAs, MACD (bottom)
-- **DIY Layout**: `vWVvjLhP` - Stochastic RSI (top), VWAP, OBV (bottom)
-- Parallel browser sessions for multi-layout capture in `lib/enhanced-screenshot.ts`
-- TradingView automation with session persistence in `lib/tradingview-automation.ts`
-- Session data stored in `.tradingview-session/` volume mount to avoid captchas
-
-### AI Analysis Pipeline  
-- OpenAI GPT-4o mini for cost-effective chart analysis (~$0.006 per analysis)
-- Multi-layout comparison and consensus detection in `lib/ai-analysis.ts`
-- Professional trading setups with exact entry/exit levels and risk management
-- Layout-specific indicator analysis (RSI vs Stochastic RSI, MACD vs OBV)
-
-### Trading Integration
-- **Drift Protocol**: Perpetual futures trading via `@drift-labs/sdk`
-- **Jupiter DEX**: Spot trading on Solana
-- Position management and P&L tracking in `lib/drift-trading-final.ts`
-- Real-time account balance and collateral monitoring
-
-## Critical Development Patterns
-
-### Docker Container Development (Required)
-**All development happens inside Docker containers** using Docker Compose v2. Browser automation requires specific system dependencies that are only available in the containerized environment:
-
-**IMPORTANT: Use Docker Compose v2 syntax** - All commands use `docker compose` (with space) instead of `docker-compose` (with hyphen).
-
-```bash
-# Development environment - Docker Compose v2 dev setup
-npm run docker:dev        # Port 9001:3000, hot reload, debug mode
-# Direct v2 command: docker compose -f docker-compose.dev.yml up --build
-
-# Production environment  
-npm run docker:up         # Port 9000:3000, optimized build
-# Direct v2 command: docker compose -f docker-compose.prod.yml up --build
-
-# Debugging commands
-npm run docker:logs       # View container logs
-# Direct v2 command: docker compose -f docker-compose.dev.yml logs -f
-
-npm run docker:exec       # Shell access for debugging inside container
-# Direct v2 command: docker compose -f docker-compose.dev.yml exec app bash
-```
-
-**Port Configuration:**
-- **Development**: External port `9001` → Internal port `3000` (http://localhost:9001)
-- **Production**: External port `9000` → Internal port `3000` (http://localhost:9000)
-
-### Docker Volume Mount Troubleshooting & Direct Container Development
-**Common Issue**: File edits not reflecting in container due to volume mount sync issues.
-
-**Container-First Development Workflow:**
-For immediate results and faster iteration, edit files directly inside the running container first, then rebuild for persistence:
-
-```bash
-# 1. Access running container for immediate edits
-docker compose -f docker-compose.dev.yml exec app bash
-
-# 2. Edit files directly in container (immediate effect)
-# Use nano, vi, or echo for quick changes
-nano /app/lib/enhanced-screenshot.ts
-echo "console.log('Debug: immediate test');" >> /app/debug.js
-
-# 3. Test changes immediately (no rebuild needed)
-# Changes take effect instantly for hot reload
-
-# 4. Once everything works, copy changes back to host
-docker cp container_name:/app/modified-file.js ./modified-file.js
-
-# 5. Commit successful changes to git BEFORE rebuilding
-git add .
-git commit -m "feat: implement working solution for [specific feature]"
-git push origin development
-
-# 6. Rebuild container for persistence
-docker compose -f docker-compose.dev.yml down
-docker compose -f docker-compose.dev.yml up --build -d
-
-# 7. Final validation and commit completion
-# Test that changes persist after rebuild
-curl http://localhost:9001  # Verify functionality
-git add . && git commit -m "chore: confirm container persistence" && git push
-```
-
-**Alternative Solutions:**
-1. **Fresh Implementation Approach**: When modifying existing files fails, create new files (e.g., `page-v2.js`) instead of editing corrupted files
-2. **Container Restart**: `docker compose -f docker-compose.dev.yml restart app`
-3. **Full Rebuild**: `docker compose -f docker-compose.dev.yml down && docker compose -f docker-compose.dev.yml up --build`
-4. **Manual Copy**: Use `docker cp` to copy files directly into container for immediate testing
-5. **Avoid sed/awk**: Direct text manipulation commands often corrupt JSX syntax - prefer file replacement
-
-**Volume Mount Verification:**
-```bash
-# Test volume mount sync
-echo "test-$(date)" > test-volume-mount.txt
-docker compose -f docker-compose.dev.yml exec app cat test-volume-mount.txt
-```
-
-**Container Development Best Practices:**
-- **Speed**: Direct container edits = immediate testing
-- **Persistence**: Always rebuild container after successful tests
-- **Backup**: Use `docker cp` to extract working changes before rebuild
-- **Debugging**: Use container shell for real-time log inspection and debugging
-
-### Multi-Timeframe Feature Copy Pattern
-When copying multi-timeframe functionality between pages:
-
-**Step 1: Identify Source Implementation**
-```bash
-# Search for existing timeframe patterns
-grep -r "timeframes.*=.*\[" app/ --include="*.js" --include="*.jsx"
-grep -r "selectedTimeframes" app/ --include="*.js" --include="*.jsx"
-```
-
-**Step 2: Copy Core State Management**
-```javascript
-// Always include these state hooks
-const [selectedTimeframes, setSelectedTimeframes] = useState(['1h', '4h']);
-const [balance, setBalance] = useState({ balance: 0, collateral: 0 });
-
-// Essential toggle function
-const toggleTimeframe = (tf) => {
-  setSelectedTimeframes(prev => 
-    prev.includes(tf) ? prev.filter(t => t !== tf) : [...prev, tf]
-  );
-};
-```
-
-**Step 3: Copy UI Components**
-- Timeframe checkbox grid
-- Preset buttons (Scalping, Day Trading, Swing Trading)
-- Auto-sizing position calculator
-- Formatted balance display
-
-**Step 4: Avoid Docker Issues**
-- Create new file instead of editing existing if volume mount issues persist
-- Use fresh filename like `page-v2.js` or `automation-v2/page.js`
-- Test in container before committing
-
-### API Route Structure
-All core functionality exposed via Next.js API routes:
-```typescript
-// Enhanced screenshot with progress tracking and robust cleanup
-POST /api/enhanced-screenshot
-{
-  symbol: "SOLUSD", 
-  timeframe: "240", 
-  layouts: ["ai", "diy"],
-  analyze: true
-}
-// Returns: { screenshots, analysis, sessionId }
-// Note: Includes automatic Chromium process cleanup via finally blocks
-
-// Drift trading endpoints
-GET /api/balance          # Account balance/collateral
-POST /api/trading         # Execute trades
-GET /api/status          # Trading status
-```
-
-**API Development Tips:**
-- All browser automation APIs include guaranteed cleanup via finally blocks
-- Use session tracking for long-running operations
-- Test API endpoints directly with curl before UI integration
-- Monitor Chromium processes during API testing: `pgrep -f chrome | wc -l`
-
-### Progress Tracking System
-Real-time operation tracking for long-running tasks:
-- `lib/progress-tracker.ts` manages EventEmitter-based progress
-- SessionId-based tracking for multi-step operations
-- Steps: init → auth → navigation → loading → capture → analysis
-- Stream endpoint: `/api/progress/[sessionId]/stream`
-
-### Browser Process Management & Cleanup System
-**Critical Issue**: Chromium processes accumulate during automated trading, consuming system resources over time.
-
-**Robust Cleanup Implementation:**
-The trading bot includes a comprehensive cleanup system to prevent Chromium process accumulation:
-
-**Core Cleanup Components:**
-1. **Enhanced Screenshot Service** (`lib/enhanced-screenshot-robust.ts`)
-   - Guaranteed cleanup via `finally` blocks in all browser operations
-   - Active session tracking to prevent orphaned browsers
-   - Session cleanup tasks array for systematic teardown
-
-2. **Automated Cleanup Service** (`lib/automated-cleanup-service.ts`)
-   - Background monitoring service for orphaned processes
-   - Multiple kill strategies: graceful → force → system cleanup
-   - Periodic cleanup of temporary files and browser data
-
-3. **Aggressive Cleanup Utilities** (`lib/aggressive-cleanup.ts`)
-   - System-level process killing for stubborn Chromium processes
-   - Port cleanup and temporary directory management
-   - Emergency cleanup functions for resource recovery
-
-**Implementation Patterns:**
-```typescript
-// Always use finally blocks for guaranteed cleanup
-try {
-  const browser = await puppeteer.launch(options);
-  // ... browser operations
-} finally {
-  // Guaranteed cleanup regardless of success/failure
-  await ensureBrowserCleanup(browser, sessionId);
-  await cleanupSessionTasks(sessionId);
-}
-
-// Background monitoring for long-running operations
-const cleanupService = new AutomatedCleanupService();
-cleanupService.startPeriodicCleanup(); // Every 10 minutes
-```
-
-**Cleanup Testing:**
-```bash
-# Test cleanup system functionality
-node test-cleanup-system.js
-
-# Monitor Chromium processes during automation
-watch 'pgrep -f "chrome|chromium" | wc -l'
-
-# Manual cleanup if needed
-node -e "require('./lib/aggressive-cleanup.ts').performAggressiveCleanup()"
-```
-
-**Prevention Strategies:**
-- Use session tracking for all browser instances
-- Implement timeout protection for long-running operations
-- Monitor resource usage during extended automation cycles
-- Restart containers periodically for fresh environment
-Critical timeframe handling to avoid TradingView confusion:
-```typescript
-// ALWAYS use minute values first, then alternatives
-'4h': ['240', '240m', '4h', '4H'] // 240 minutes FIRST
-'1h': ['60', '60m', '1h', '1H']   // 60 minutes FIRST
-'15m': ['15', '15m']
-```
-
-Layout URL mappings for direct navigation:
-```typescript
-const LAYOUT_URLS = {
-  'ai': 'Z1TzpUrf',    // RSI + EMAs + MACD
-  'diy': 'vWVvjLhP'    // Stochastic RSI + VWAP + OBV
-}
-```
-
-### Component Architecture
-- `app/layout.js` - Root layout with gradient styling and navigation
-- `components/Navigation.tsx` - Multi-page navigation system
-- `components/AIAnalysisPanel.tsx` - Multi-timeframe analysis interface
-- `components/Dashboard.tsx` - Main trading dashboard with real Drift positions
-- `components/AdvancedTradingPanel.tsx` - Drift Protocol trading interface
-
-### Cleanup System Architecture
-**Critical Production Issue**: Chromium processes accumulate during automated trading, leading to resource exhaustion after several hours of operation.
-
-**Solution Components:**
-1. **Enhanced Screenshot Service** (`lib/enhanced-screenshot-robust.ts`)
-   - Replaces original screenshot service with guaranteed cleanup
-   - Uses `finally` blocks to ensure browser cleanup regardless of success/failure
-   - Active session tracking with cleanup task arrays
-   - Force-kill functionality for stubborn processes
-
-2. **Automated Cleanup Service** (`lib/automated-cleanup-service.ts`)
-   - Background monitoring service that runs every 10 minutes
-   - Multiple cleanup strategies: graceful → force → system-level cleanup
-   - Temporary file cleanup and browser data directory management
-   - Orphaned process detection and elimination
-
-3. **Aggressive Cleanup Utilities** (`lib/aggressive-cleanup.ts`)
-   - Emergency cleanup functions for critical resource recovery
-   - System-level process management with multiple kill strategies
-   - Port cleanup and zombie process elimination
-   - Used by both automated service and manual intervention
-
-**Integration Pattern:**
-```typescript
-// In API routes - always use finally blocks
-app/api/enhanced-screenshot/route.js:
-try {
-  const result = await enhancedScreenshot.captureAndAnalyze(...);
-  return NextResponse.json(result);
-} finally {
-  // Guaranteed cleanup execution
-  await enhancedScreenshot.cleanup();
-}
-
-// Background monitoring
-lib/automated-cleanup-service.ts:
-setInterval(async () => {
-  await this.performCleanup();
-}, 10 * 60 * 1000); // Every 10 minutes
-```
-
-**Testing Cleanup System:**
-```bash
-# Monitor process count during operation
-watch 'pgrep -f "chrome|chromium" | wc -l'
-
-# Test cleanup functionality
-node test-cleanup-system.js
-
-# Manual cleanup if needed
-docker compose exec app node -e "require('./lib/aggressive-cleanup.ts').forceKillAllChromium()"
-```
-
-### Page Structure & Multi-Timeframe Implementation
-- `app/analysis/page.js` - Original analysis page with multi-timeframe functionality
-- `app/automation/page.js` - Original automation page (legacy, may have issues)
-- `app/automation-v2/page.js` - **NEW**: Clean automation page with full multi-timeframe support
-- `app/automation/page-v2.js` - Alternative implementation, same functionality as automation-v2
-
-**Multi-Timeframe Architecture Pattern:**
-```javascript
-// Standard timeframes array - use this exact format
-const timeframes = ['5m', '15m', '30m', '1h', '2h', '4h', '1d'];
-
-// State management for multi-timeframe selection
-const [selectedTimeframes, setSelectedTimeframes] = useState(['1h', '4h']);
-
-// Toggle function with proper array handling
-const toggleTimeframe = (tf) => {
-  setSelectedTimeframes(prev => 
-    prev.includes(tf) 
-      ? prev.filter(t => t !== tf)  // Remove if selected
-      : [...prev, tf]                // Add if not selected
-  );
-};
-
-// Preset configurations for trading styles
-const presets = {
-  scalping: ['5m', '15m', '1h'],
-  day: ['1h', '4h', '1d'],
-  swing: ['4h', '1d']
-};
-```
-
-**UI Pattern for Timeframe Selection:**
-```jsx
-// Checkbox grid layout with visual feedback
-<div className="grid grid-cols-4 gap-2 mb-4">
-  {timeframes.map(tf => (
-    <button
-      key={tf}
-      onClick={() => toggleTimeframe(tf)}
-      className={`p-2 rounded border transition-all ${
-        selectedTimeframes.includes(tf)
-          ? 'bg-blue-600 border-blue-500 text-white'
-          : 'bg-gray-700 border-gray-600 text-gray-300 hover:bg-gray-600'
-      }`}
-    >
-      {tf}
-    </button>
-  ))}
-</div>
-
-// Preset buttons for quick selection
-<div className="flex gap-2 mb-4">
-  {Object.entries(presets).map(([name, tfs]) => (
-    <button
-      key={name}
-      onClick={() => setSelectedTimeframes(tfs)}
-      className="px-3 py-1 bg-purple-600 hover:bg-purple-700 rounded text-sm"
-    >
-      {name.charAt(0).toUpperCase() + name.slice(1)}
-    </button>
-  ))}
-</div>
-```
-
-## Environment Variables
-```bash
-# AI Analysis (Required)
-OPENAI_API_KEY=sk-...     # OpenAI API key for chart analysis
-
-# TradingView Automation (Required)
-TRADINGVIEW_EMAIL=        # TradingView account email
-TRADINGVIEW_PASSWORD=     # TradingView account password
-
-# Trading Integration (Optional)
-SOLANA_RPC_URL=https://api.mainnet-beta.solana.com
-DRIFT_PRIVATE_KEY=        # Base58 encoded Solana private key
-SOLANA_PRIVATE_KEY=       # JSON array format for Jupiter DEX
-
-# Docker Environment Detection
-DOCKER_ENV=true           # Auto-set in containers
-PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium
-```
-
-## Testing & Debugging Workflow
-Test files follow specific patterns - use them to validate changes:
-```bash
-# Test dual-session screenshot capture
-node test-enhanced-screenshot.js
-
-# Test robust cleanup system
-node test-cleanup-system.js
-
-# Test Docker environment (requires Docker Compose v2)
-./test-docker-comprehensive.sh
-
-# Test API endpoints directly
-node test-analysis-api.js
-
-# Test Drift trading integration
-node test-drift-trading.js
-
-# Monitor resource usage during automation
-watch 'pgrep -f "chrome|chromium" | wc -l'
-```
-
-**Container-First Development Workflow:**
-```bash
-# 1. Start development container
-npm run docker:dev  # Port 9001
-
-# 2. For immediate testing, edit directly in container
-docker compose -f docker-compose.dev.yml exec app bash
-# Edit files in /app/ directory for instant results
-
-# 3. Test changes in real-time (hot reload active)
-curl http://localhost:9001/api/enhanced-screenshot
-
-# 4. Once working, commit progress to git
-git add .
-git commit -m "feat: implement [feature] - tested and working"
-git push origin development
-
-# 5. Rebuild container for persistence
-docker compose -f docker-compose.dev.yml down
-docker compose -f docker-compose.dev.yml up --build -d
-
-# 6. Validate persistent changes and final commit
-curl http://localhost:9001  # Should show updated functionality
-git add . && git commit -m "chore: confirm persistence after container rebuild" && git push
-```
-
-Browser automation debugging:
-- Screenshots automatically saved to `screenshots/` with timestamps
-- Debug screenshots: `takeDebugScreenshot('prefix')` 
-- Session persistence prevents repeated logins/captchas
-- Use `npm run docker:logs` to view real-time automation logs
-- All Docker commands use v2 syntax: `docker compose` (not `docker-compose`)
-- Monitor Chromium processes: `docker compose exec app pgrep -f chrome`
-
-## Code Style & Architecture Patterns
-- **Client Components**: Use `"use client"` for state/effects, server components by default
-- **Styling**: Tailwind with gradient backgrounds (`bg-gradient-to-br from-gray-900 via-blue-900 to-purple-900`)
-- **Error Handling**: Detailed logging for browser automation with fallbacks
-- **File Structure**: Mixed `.js`/`.tsx` - components in TypeScript, API routes in JavaScript
-- **Database**: Prisma with SQLite (`DATABASE_URL=file:./prisma/dev.db`)
-
-## Key Integration Points
-- **Session Persistence**: `.tradingview-session/` directory volume-mounted
-- **Screenshots**: `screenshots/` directory for chart captures
-- **Progress Tracking**: EventEmitter-based real-time updates via SSE
-- **Multi-Stage Docker**: Development vs production builds with browser optimization
-- **CAPTCHA Handling**: Manual CAPTCHA mode with X11 forwarding (`ALLOW_MANUAL_CAPTCHA=true`)
-- **Process Management**: Robust cleanup system prevents Chromium accumulation
-- **Container Development**: Direct in-container editing for immediate testing, rebuild for persistence
-
-## Development vs Production Modes
-- **Development**: Port 9001:3000, hot reload, debug logging, headless: false option
-- **Production**: Port 9000:3000, optimized build, minimal logging, always headless
-
-**Development Container Features:**
-- **Hot Reload**: File changes reflect immediately (when volume mounts work)
-- **Process Monitoring**: Real-time Chromium process tracking
-- **Debug Access**: Shell access via `docker compose exec app bash`
-- **Immediate Testing**: Edit files in `/app/` directory for instant results
-- **Resource Cleanup**: Automated cleanup services running in background
-
-### Git Branch Strategy (Required)
-**Primary development workflow:**
-- **`development` branch**: Use for all active development and feature work
-- **`main` branch**: Stable, production-ready code only
-- **Workflow**: Develop on `development` → test thoroughly → commit progress → merge to `main` when stable
-
-```bash
-# Standard development workflow with frequent commits
-git checkout development        # Always start here
-git pull origin development     # Get latest changes
-
-# Make your changes and test in container...
-
-# Commit working progress BEFORE rebuilding container
-git add .
-git commit -m "feat: [specific achievement] - tested and working"
-git push origin development
-
-# After successful container rebuild and validation
-git add .
-git commit -m "chore: confirm [feature] persistence after rebuild"
-git push origin development
-
-# Only merge to main when features are stable and tested and you have asked the user to merge to main
-git checkout main
-git merge development          # When ready for production
-git push origin main
-```
-
-**Git Commit Best Practices:**
-- **Commit Early**: Save working progress before container rebuilds
-- **Commit Often**: After each successful test or implementation step
-- **Descriptive Messages**: Include what was accomplished and tested
-- **Final Commits**: Always commit after confirming container persistence
-
-**Container Persistence & Git Strategy:**
-- Git changes only persist after container rebuild with `--build` flag
-- **CRITICAL**: Commit working changes BEFORE rebuilding container
-- Test changes in container first, then commit and rebuild
-- Use descriptive commit messages for cleanup system improvements
-- Example commits from robust cleanup implementation:
-  - `feat: implement robust cleanup system with finally blocks - tested in container`
-  - `fix: restore automation-v2 page with balance slider - confirmed working`
-  - `chore: confirm cleanup system persistence after container rebuild`
-  - `docs: update instructions with container development workflow`
-
-When working with this codebase, prioritize Docker consistency, understand the dual-session architecture, leverage the comprehensive test suite to validate changes, and always implement proper cleanup patterns for browser automation to prevent resource exhaustion.
-
-## Common Issues & Troubleshooting
-
-### Chromium Process Accumulation
-**Symptoms**: System becomes slow after hours of automation, high CPU/memory usage, many chrome processes running
-**Diagnosis**: `pgrep -f "chrome|chromium" | wc -l` shows increasing process count
-**Solutions**:
-1. Ensure all browser automation uses finally blocks for cleanup
-2. Restart automated cleanup service: `docker compose exec app node -e "require('./lib/automated-cleanup-service.ts').startPeriodicCleanup()"`
-3. Manual cleanup: `docker compose exec app node test-cleanup-system.js`
-4. Container restart: `docker compose restart app`
-
-### Volume Mount Sync Issues
-**Symptoms**: File changes not reflecting in running container
-**Diagnosis**: Edit test file and check if visible in container
-**Solutions**:
-1. **Quick Fix**: Edit directly in container for immediate testing
-2. **Commit Progress**: Save working changes to git before rebuilding
-3. **Persistence**: Always rebuild container after confirming changes work
-4. **Final Commit**: Validate and commit after successful rebuild
-5. **Manual Copy**: Use `docker cp` to transfer working files
-6. **Fresh Start**: Create new files instead of editing problematic ones
-
-### Automation Page Restoration
-**Symptoms**: Automation page shows old version after container rebuild
-**Issue**: Git history not properly maintained in container
-**Solutions**:
-1. Check current branch: `git branch`
-2. Restore from git: `git checkout HEAD -- app/automation-v2/page.js`
-3. Verify features: Check for balance slider and multi-timeframe selection
-4. Commit restoration: `git add . && git commit -m "fix: restore automation-v2 functionality" && git push`
-5. Rebuild container to persist restoration
-
-### Successful Implementation Workflow
-**After completing any feature or fix:**
-```bash
-# 1. Test functionality thoroughly
-curl http://localhost:9001/api/test-endpoint
-
-# 2. Commit successful implementation
-git add .
-git commit -m "feat: [specific achievement] - fully tested and working"
-git push origin development
-
-# 3. Rebuild container for persistence  
-docker compose down && docker compose up --build -d
-
-# 4. Final validation and completion commit
-curl http://localhost:9001  # Verify persistent functionality
-git add . && git commit -m "chore: confirm [feature] persistence - implementation complete" && git push
-
-# 5. Consider merge to main if ready for production
-# (Ask user first before merging to main branch)
-```
-
-### API Endpoint Not Responding
-**Symptoms**: API calls timeout or return errors
-**Diagnosis**: Check container logs: `docker compose logs -f app`
-**Solutions**:
-1. Verify container is running: `docker ps`
-2. Check port mapping: Development=9001:3000, Production=9000:3000
-3. Test direct access: `curl localhost:9001/api/status`
-4. Restart if needed: `docker compose restart app`
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
index 2eb9163..da28100 100644
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -2,13 +2,15 @@
 
 ## 🎯 Project Context & Architecture
 
-This is an AI-powered trading automation system with advanced learning capabilities. When working on this codebase:
+This is an AI-powered trading automation system with advanced learning capabilities built with Next.js 15 App Router, TypeScript, Tailwind CSS, and integrated with Drift Protocol and Jupiter DEX for automated trading execution.
 
 ### Core System Components
-1. **Superior Parallel Screenshot System** - 60% faster than sequential capture
-2. **AI Learning System** - Adapts trading decisions based on outcomes  
-3. **Orphaned Order Cleanup** - Automatic cleanup when positions close
+1. **Superior Parallel Screenshot System** - 60% faster than sequential capture (71s vs 180s)
+2. **AI Learning System** - Adapts trading decisions based on outcomes with pattern recognition  
+3. **Orphaned Order Cleanup** - Automatic cleanup when positions close via position monitor
 4. **Position Monitoring** - Frequent checks with integrated cleanup triggers
+5. **Dual-Session Screenshot Automation** - AI and DIY layouts with session persistence
+6. **Robust Cleanup System** - Prevents Chromium process accumulation
 
 ### Critical File Relationships
 ```
@@ -16,9 +18,279 @@ app/api/automation/position-monitor/route.js → Monitors positions + triggers c
 lib/simplified-stop-loss-learner.js → AI learning core with pattern recognition
 lib/superior-screenshot-service.ts → Parallel screenshot capture system
 lib/enhanced-autonomous-risk-manager.js → Risk management with AI integration
+lib/enhanced-screenshot-robust.ts → Guaranteed cleanup with finally blocks
+lib/automated-cleanup-service.ts → Background process monitoring
+```
+
+## 🚀 Development Environment (Critical)
+
+### Docker Container Development (Required)
+**All development happens inside Docker containers** using Docker Compose v2. Browser automation requires specific system dependencies only available in containerized environment.
+
+**IMPORTANT: Use Docker Compose v2 syntax** - All commands use `docker compose` (with space) instead of `docker-compose` (with hyphen).
+
+```bash
+# Development environment - Docker Compose v2 dev setup
+npm run docker:dev        # Port 9001:3000, hot reload, debug mode
+# Direct v2 command: docker compose -f docker-compose.dev.yml up --build
+
+# Production environment  
+npm run docker:up         # Port 9000:3000, optimized build
+# Direct v2 command: docker compose -f docker-compose.prod.yml up --build
+
+# Debugging commands
+npm run docker:logs       # View container logs
+npm run docker:exec       # Shell access for debugging inside container
+```
+
+**Port Configuration:**
+- **Development**: External port `9001` → Internal port `3000` (http://localhost:9001)
+- **Production**: External port `9000` → Internal port `3000` (http://localhost:9000)
+
+### Container-First Development Workflow
+**Common Issue**: File edits not reflecting in container due to volume mount sync issues.
+
+**Solution - Container Development Workflow:**
+```bash
+# 1. Access running container for immediate edits
+docker compose -f docker-compose.dev.yml exec app bash
+
+# 2. Edit files directly in container (immediate effect)
+nano /app/lib/enhanced-screenshot.ts
+echo "console.log('Debug: immediate test');" >> /app/debug.js
+
+# 3. Test changes immediately (no rebuild needed)
+# Changes take effect instantly for hot reload
+
+# 4. Once everything works, copy changes back to host
+docker cp container_name:/app/modified-file.js ./modified-file.js
+
+# 5. Commit successful changes to git BEFORE rebuilding
+git add .
+git commit -m "feat: implement working solution for [specific feature]"
+git push origin development
+
+# 6. Rebuild container for persistence
+docker compose -f docker-compose.dev.yml down
+docker compose -f docker-compose.dev.yml up --build -d
+
+# 7. Final validation and commit completion
+curl http://localhost:9001  # Verify functionality
+git add . && git commit -m "chore: confirm container persistence" && git push
+```
+
+### Git Branch Strategy (Required)
+**Primary development workflow:**
+- **`development` branch**: Use for all active development and feature work
+- **`main` branch**: Stable, production-ready code only
+- **Workflow**: Develop on `development` → test thoroughly → commit progress → merge to `main` when stable
+
+```bash
+# Standard development workflow with frequent commits
+git checkout development        # Always start here
+git pull origin development     # Get latest changes
+
+# Make your changes and test in container...
+
+# Commit working progress BEFORE rebuilding container
+git add .
+git commit -m "feat: [specific achievement] - tested and working"
+git push origin development
+
+# After successful container rebuild and validation
+git add .
+git commit -m "chore: confirm [feature] persistence after rebuild"
+git push origin development
+
+# Only merge to main when features are stable and tested
+git checkout main
+git merge development          # When ready for production
+git push origin main
+```
+
+## 🏗️ System Architecture
+
+### Dual-Session Screenshot Automation
+- **AI Layout**: `Z1TzpUrf` - RSI (top), EMAs, MACD (bottom)
+- **DIY Layout**: `vWVvjLhP` - Stochastic RSI (top), VWAP, OBV (bottom)
+- Parallel browser sessions for multi-layout capture in `lib/enhanced-screenshot.ts`
+- TradingView automation with session persistence in `lib/tradingview-automation.ts`
+- Session data stored in `.tradingview-session/` volume mount to avoid captchas
+
+### AI Analysis Pipeline  
+- OpenAI GPT-4o mini for cost-effective chart analysis (~$0.006 per analysis)
+- Multi-layout comparison and consensus detection in `lib/ai-analysis.ts`
+- Professional trading setups with exact entry/exit levels and risk management
+- Layout-specific indicator analysis (RSI vs Stochastic RSI, MACD vs OBV)
+
+### Trading Integration
+- **Drift Protocol**: Perpetual futures trading via `@drift-labs/sdk`
+- **Jupiter DEX**: Spot trading on Solana
+- Position management and P&L tracking in `lib/drift-trading-final.ts`
+- Real-time account balance and collateral monitoring
+
+### Browser Process Management & Cleanup System
+**Critical Issue**: Chromium processes accumulate during automated trading, consuming system resources over time.
+
+**Robust Cleanup Implementation:**
+1. **Enhanced Screenshot Service** (`lib/enhanced-screenshot-robust.ts`)
+   - Guaranteed cleanup via `finally` blocks in all browser operations
+   - Active session tracking to prevent orphaned browsers
+   - Session cleanup tasks array for systematic teardown
+
+2. **Automated Cleanup Service** (`lib/automated-cleanup-service.ts`)
+   - Background monitoring service for orphaned processes
+   - Multiple kill strategies: graceful → force → system cleanup
+   - Periodic cleanup of temporary files and browser data
+
+3. **Aggressive Cleanup Utilities** (`lib/aggressive-cleanup.ts`)
+   - System-level process killing for stubborn Chromium processes
+   - Port cleanup and temporary directory management
+   - Emergency cleanup functions for resource recovery
+
+**Implementation Patterns:**
+```typescript
+// Always use finally blocks for guaranteed cleanup
+try {
+  const browser = await puppeteer.launch(options);
+  // ... browser operations
+} finally {
+  // Guaranteed cleanup regardless of success/failure
+  await ensureBrowserCleanup(browser, sessionId);
+  await cleanupSessionTasks(sessionId);
+}
+
+// Background monitoring for long-running operations
+const cleanupService = new AutomatedCleanupService();
+cleanupService.startPeriodicCleanup(); // Every 10 minutes
+```
+
+
+### API Route Structure
+All core functionality exposed via Next.js API routes:
+```typescript
+// Enhanced screenshot with progress tracking and robust cleanup
+POST /api/enhanced-screenshot
+{
+  symbol: "SOLUSD", 
+  timeframe: "240", 
+  layouts: ["ai", "diy"],
+  analyze: true
+}
+// Returns: { screenshots, analysis, sessionId }
+// Note: Includes automatic Chromium process cleanup via finally blocks
+
+// Drift trading endpoints
+GET /api/balance          # Account balance/collateral
+POST /api/trading         # Execute trades
+GET /api/status          # Trading status
+GET /api/automation/position-monitor  # Position monitoring with orphaned cleanup
+POST /api/drift/cleanup-orders        # Manual order cleanup
+```
+
+### Progress Tracking System
+Real-time operation tracking for long-running tasks:
+- `lib/progress-tracker.ts` manages EventEmitter-based progress
+- SessionId-based tracking for multi-step operations
+- Steps: init → auth → navigation → loading → capture → analysis
+- Stream endpoint: `/api/progress/[sessionId]/stream`
+
+### Page Structure & Multi-Timeframe Implementation
+- `app/analysis/page.js` - Original analysis page with multi-timeframe functionality
+- `app/automation/page.js` - Original automation page (legacy, may have issues)
+- `app/automation-v2/page.js` - **NEW**: Clean automation page with full multi-timeframe support
+- `app/automation/page-v2.js` - Alternative implementation, same functionality as automation-v2
+
+**Multi-Timeframe Architecture Pattern:**
+```javascript
+// Standard timeframes array - use this exact format
+const timeframes = ['5m', '15m', '30m', '1h', '2h', '4h', '1d'];
+
+// State management for multi-timeframe selection
+const [selectedTimeframes, setSelectedTimeframes] = useState(['1h', '4h']);
+
+// Toggle function with proper array handling
+const toggleTimeframe = (tf) => {
+  setSelectedTimeframes(prev => 
+    prev.includes(tf) 
+      ? prev.filter(t => t !== tf)  // Remove if selected
+      : [...prev, tf]                // Add if not selected
+  );
+};
+
+// Preset configurations for trading styles
+const presets = {
+  scalping: ['5m', '15m', '1h'],
+  day: ['1h', '4h', '1d'],
+  swing: ['4h', '1d']
+};
+```
+
+### Component Architecture
+- `app/layout.js` - Root layout with gradient styling and navigation
+- `components/Navigation.tsx` - Multi-page navigation system
+- `components/AIAnalysisPanel.tsx` - Multi-timeframe analysis interface
+- `components/Dashboard.tsx` - Main trading dashboard with real Drift positions
+- `components/AdvancedTradingPanel.tsx` - Drift Protocol trading interface
+
+Critical timeframe handling to avoid TradingView confusion:
+```typescript
+// ALWAYS use minute values first, then alternatives
+'4h': ['240', '240m', '4h', '4H'] // 240 minutes FIRST
+'1h': ['60', '60m', '1h', '1H']   // 60 minutes FIRST
+'15m': ['15', '15m']
+```
+
+Layout URL mappings for direct navigation:
+```typescript
+const LAYOUT_URLS = {
+  'ai': 'Z1TzpUrf',    // RSI + EMAs + MACD
+  'diy': 'vWVvjLhP'    // Stochastic RSI + VWAP + OBV
+}
 ```
 
 ## 🧠 AI Learning System Patterns
+```javascript
+async generateLearningReport() {
+  // Return comprehensive learning status
+  return {
+    summary: { totalDecisions, systemConfidence, successRate },
+    insights: { thresholds, confidenceLevel },
+    recommendations: []
+  };
+}
+
+async getSmartRecommendation(requestData) {
+  // Analyze patterns and return AI recommendation
+  const { distanceFromSL, symbol, marketConditions } = requestData;
+  // Return: { action, confidence, reasoning }
+}
+
+async recordDecision(decisionData) {
+  // Log decision for learning with unique ID
+  const id = `decision_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
+  // Store in database for pattern analysis
+}
+
+async assessDecisionOutcome(outcomeData) {
+  // Update decision with actual result for learning
+  // Calculate if decision was correct based on outcome
+}
+```
+
+### Database Operations Best Practices:
+```javascript
+// ALWAYS provide unique IDs for Prisma records
+await prisma.ai_learning_data.create({
+  data: {
+    id: `${prefix}_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`,
+    // ... other fields
+  }
+});
+
+// Use correct import path
+const { getDB } = require('./db');  // NOT './database-util'
+```
 
 ### Always Include These Functions in Learning Classes:
 ```javascript
@@ -65,6 +337,8 @@ const { getDB } = require('./db');  // NOT './database-util'
 
 ## 🔧 Error Handling Patterns
 
+## 🔧 Error Handling Patterns
+
 ### Function Existence Checks:
 ```javascript
 // Always check if functions exist before calling
@@ -87,6 +361,28 @@ try {
 }
 ```
 
+## 📊 Integration Patterns
+```javascript
+// Always check if functions exist before calling
+if (typeof this.learner.generateLearningReport === 'function') {
+  const report = await this.learner.generateLearningReport();
+} else {
+  // Fallback to alternative method
+  const status = await this.learner.getLearningStatus();
+}
+```
+
+### Comprehensive Try-Catch:
+```javascript
+try {
+  const result = await aiFunction();
+  return result;
+} catch (error) {
+  await this.log(`❌ AI function error: ${error.message}`);
+  return fallbackResult(); // Always provide fallback
+}
+```
+
 ## 📊 Integration Patterns
 
 ### Position Monitor Integration:
@@ -297,6 +593,26 @@ When adding new features:
 6. **Maintain Consistency** - Frontend and backend must match exactly
 7. **Use Defensive Programming** - Check before calling, handle gracefully
 
+## 📚 Documentation References
+
+### Technical Documentation
+- **`ADVANCED_SYSTEM_KNOWLEDGE.md`** - Deep technical architecture, session management, cleanup systems
+- **`README.md`** - Main project overview with current feature status and setup
+- **`AI_LEARNING_EXPLAINED.md`** - AI learning system implementation details
+- **`DRIFT_FEEDBACK_LOOP_COMPLETE.md`** - Complete Drift trading integration
+- **`ROBUST_CLEANUP_IMPLEMENTATION.md`** - Browser process cleanup system details
+
+### Implementation Guides
+- **`MULTI_LAYOUT_IMPLEMENTATION.md`** - Dual-session screenshot system
+- **`SESSION_PERSISTENCE.md`** - TradingView session management
+- **`DOCKER_AUTOMATION.md`** - Container development workflow
+- **`DEVELOPMENT_GUIDE.md`** - Complete development setup instructions
+
+### Analysis & Troubleshooting
+- **`MULTI_LAYOUT_TROUBLESHOOTING.md`** - Screenshot automation debugging
+- **`CLEANUP_IMPROVEMENTS.md`** - Process management enhancements
+- **`SCREENSHOT_PATH_FIXES.md`** - Screenshot capture issue resolution
+
 ---
 
 **Follow these patterns to maintain system stability and avoid the complex debugging issues that were resolved in this session.**