trading_bot_v3/.github/copilot-instructions.instructions.md

# AI-Powered Trading Bot Dashboard

This is a Next.js 15 App Router application with TypeScript, Tailwind CSS, and API routes. It's a production-ready trading bot with AI analysis, automated screenshot capture, and real-time trading execution via Drift Protocol and Jupiter DEX.

**Prerequisites:**
- Docker and Docker Compose v2 (uses `docker compose` command syntax)
- All development must be done inside Docker containers for browser automation compatibility

## Core Architecture

### Dual-Session Screenshot Automation
- **AI Layout**: `Z1TzpUrf` - RSI (top), EMAs, MACD (bottom)
- **DIY Layout**: `vWVvjLhP` - Stochastic RSI (top), VWAP, OBV (bottom)
- Parallel browser sessions for multi-layout capture in `lib/enhanced-screenshot.ts`
- TradingView automation with session persistence in `lib/tradingview-automation.ts`
- Session data stored in `.tradingview-session/` volume mount to avoid captchas

### AI Analysis Pipeline  
- OpenAI GPT-4o mini for cost-effective chart analysis (~$0.006 per analysis)
- Multi-layout comparison and consensus detection in `lib/ai-analysis.ts`
- Professional trading setups with exact entry/exit levels and risk management
- Layout-specific indicator analysis (RSI vs Stochastic RSI, MACD vs OBV)

### Trading Integration
- **Drift Protocol**: Perpetual futures trading via `@drift-labs/sdk`
- **Jupiter DEX**: Spot trading on Solana
- Position management and P&L tracking in `lib/drift-trading-final.ts`
- Real-time account balance and collateral monitoring

## Critical Development Patterns

### Docker Container Development (Required)
**All development happens inside Docker containers** using Docker Compose v2. Browser automation requires specific system dependencies that are only available in the containerized environment:

**IMPORTANT: Use Docker Compose v2 syntax** - All commands use `docker compose` (with space) instead of `docker-compose` (with hyphen).

```bash
# Development environment - Docker Compose v2 dev setup
npm run docker:dev        # Port 9001:3000, hot reload, debug mode
# Direct v2 command: docker compose -f docker-compose.dev.yml up --build

# Production environment  
npm run docker:up         # Port 9000:3000, optimized build
# Direct v2 command: docker compose -f docker-compose.prod.yml up --build

# Debugging commands
npm run docker:logs       # View container logs
# Direct v2 command: docker compose -f docker-compose.dev.yml logs -f

npm run docker:exec       # Shell access for debugging inside container
# Direct v2 command: docker compose -f docker-compose.dev.yml exec app bash
```

**Port Configuration:**
- **Development**: External port `9001` → Internal port `3000` (http://localhost:9001)
- **Production**: External port `9000` → Internal port `3000` (http://localhost:9000)

### Docker Volume Mount Troubleshooting
**Common Issue**: File edits not reflecting in container due to volume mount sync issues.

**Solutions:**
1. **Fresh Implementation Approach**: When modifying existing files fails, create new files (e.g., `page-v2.js`) instead of editing corrupted files
2. **Container Restart**: `docker compose -f docker-compose.dev.yml restart app`
3. **Full Rebuild**: `docker compose -f docker-compose.dev.yml down && docker compose -f docker-compose.dev.yml up --build`
4. **Manual Copy**: Use `docker cp` to copy files directly into container for immediate testing
5. **Avoid sed/awk**: Direct text manipulation commands often corrupt JSX syntax - prefer file replacement

**Volume Mount Verification:**
```bash
# Test volume mount sync
echo "test-$(date)" > test-volume-mount.txt
docker compose -f docker-compose.dev.yml exec app cat test-volume-mount.txt
```

### Multi-Timeframe Feature Copy Pattern
When copying multi-timeframe functionality between pages:

**Step 1: Identify Source Implementation**
```bash
# Search for existing timeframe patterns
grep -r "timeframes.*=.*\[" app/ --include="*.js" --include="*.jsx"
grep -r "selectedTimeframes" app/ --include="*.js" --include="*.jsx"
```

**Step 2: Copy Core State Management**
```javascript
// Always include these state hooks
const [selectedTimeframes, setSelectedTimeframes] = useState(['1h', '4h']);
const [balance, setBalance] = useState({ balance: 0, collateral: 0 });

// Essential toggle function
const toggleTimeframe = (tf) => {
  setSelectedTimeframes(prev => 
    prev.includes(tf) ? prev.filter(t => t !== tf) : [...prev, tf]
  );
};
```

**Step 3: Copy UI Components**
- Timeframe checkbox grid
- Preset buttons (Scalping, Day Trading, Swing Trading)
- Auto-sizing position calculator
- Formatted balance display

**Step 4: Avoid Docker Issues**
- Create new file instead of editing existing if volume mount issues persist
- Use fresh filename like `page-v2.js` or `automation-v2/page.js`
- Test in container before committing

### API Route Structure
All core functionality exposed via Next.js API routes:
```typescript
// Enhanced screenshot with progress tracking
POST /api/enhanced-screenshot
{
  symbol: "SOLUSD", 
  timeframe: "240", 
  layouts: ["ai", "diy"],
  analyze: true
}
// Returns: { screenshots, analysis, sessionId }

// Drift trading endpoints
GET /api/balance          # Account balance/collateral
POST /api/trading         # Execute trades
GET /api/status          # Trading status
```

### Progress Tracking System
Real-time operation tracking for long-running tasks:
- `lib/progress-tracker.ts` manages EventEmitter-based progress
- SessionId-based tracking for multi-step operations
- Steps: init → auth → navigation → loading → capture → analysis
- Stream endpoint: `/api/progress/[sessionId]/stream`

### TradingView Automation Patterns
Critical timeframe handling to avoid TradingView confusion:
```typescript
// ALWAYS use minute values first, then alternatives
'4h': ['240', '240m', '4h', '4H'] // 240 minutes FIRST
'1h': ['60', '60m', '1h', '1H']   // 60 minutes FIRST
'15m': ['15', '15m']
```

Layout URL mappings for direct navigation:
```typescript
const LAYOUT_URLS = {
  'ai': 'Z1TzpUrf',    // RSI + EMAs + MACD
  'diy': 'vWVvjLhP'    // Stochastic RSI + VWAP + OBV
}
```

### Component Architecture
- `app/layout.js` - Root layout with gradient styling and navigation
- `components/Navigation.tsx` - Multi-page navigation system
- `components/AIAnalysisPanel.tsx` - Multi-timeframe analysis interface
- `components/Dashboard.tsx` - Main trading dashboard with real Drift positions
- `components/AdvancedTradingPanel.tsx` - Drift Protocol trading interface

### Page Structure & Multi-Timeframe Implementation
- `app/analysis/page.js` - Original analysis page with multi-timeframe functionality
- `app/automation/page.js` - Original automation page (legacy, may have issues)
- `app/automation-v2/page.js` - **NEW**: Clean automation page with full multi-timeframe support
- `app/automation/page-v2.js` - Alternative implementation, same functionality as automation-v2

**Multi-Timeframe Architecture Pattern:**
```javascript
// Standard timeframes array - use this exact format
const timeframes = ['5m', '15m', '30m', '1h', '2h', '4h', '1d'];

// State management for multi-timeframe selection
const [selectedTimeframes, setSelectedTimeframes] = useState(['1h', '4h']);

// Toggle function with proper array handling
const toggleTimeframe = (tf) => {
  setSelectedTimeframes(prev => 
    prev.includes(tf) 
      ? prev.filter(t => t !== tf)  // Remove if selected
      : [...prev, tf]                // Add if not selected
  );
};

// Preset configurations for trading styles
const presets = {
  scalping: ['5m', '15m', '1h'],
  day: ['1h', '4h', '1d'],
  swing: ['4h', '1d']
};
```

**UI Pattern for Timeframe Selection:**
```jsx
// Checkbox grid layout with visual feedback
<div className="grid grid-cols-4 gap-2 mb-4">
  {timeframes.map(tf => (
    <button
      key={tf}
      onClick={() => toggleTimeframe(tf)}
      className={`p-2 rounded border transition-all ${
        selectedTimeframes.includes(tf)
          ? 'bg-blue-600 border-blue-500 text-white'
          : 'bg-gray-700 border-gray-600 text-gray-300 hover:bg-gray-600'
      }`}
    >
      {tf}
    </button>
  ))}
</div>

// Preset buttons for quick selection
<div className="flex gap-2 mb-4">
  {Object.entries(presets).map(([name, tfs]) => (
    <button
      key={name}
      onClick={() => setSelectedTimeframes(tfs)}
      className="px-3 py-1 bg-purple-600 hover:bg-purple-700 rounded text-sm"
    >
      {name.charAt(0).toUpperCase() + name.slice(1)}
    </button>
  ))}
</div>
```

## Environment Variables
```bash
# AI Analysis (Required)
OPENAI_API_KEY=sk-...     # OpenAI API key for chart analysis

# TradingView Automation (Required)
TRADINGVIEW_EMAIL=        # TradingView account email
TRADINGVIEW_PASSWORD=     # TradingView account password

# Trading Integration (Optional)
SOLANA_RPC_URL=https://api.mainnet-beta.solana.com
DRIFT_PRIVATE_KEY=        # Base58 encoded Solana private key
SOLANA_PRIVATE_KEY=       # JSON array format for Jupiter DEX

# Docker Environment Detection
DOCKER_ENV=true           # Auto-set in containers
PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium
```

## Testing & Debugging Workflow
Test files follow specific patterns - use them to validate changes:
```bash
# Test dual-session screenshot capture
node test-enhanced-screenshot.js

# Test Docker environment (requires Docker Compose v2)
./test-docker-comprehensive.sh

# Test API endpoints directly
node test-analysis-api.js

# Test Drift trading integration
node test-drift-trading.js
```

Browser automation debugging:
- Screenshots automatically saved to `screenshots/` with timestamps
- Debug screenshots: `takeDebugScreenshot('prefix')` 
- Session persistence prevents repeated logins/captchas
- Use `npm run docker:logs` to view real-time automation logs
- All Docker commands use v2 syntax: `docker compose` (not `docker-compose`)

## Code Style & Architecture Patterns
- **Client Components**: Use `"use client"` for state/effects, server components by default
- **Styling**: Tailwind with gradient backgrounds (`bg-gradient-to-br from-gray-900 via-blue-900 to-purple-900`)
- **Error Handling**: Detailed logging for browser automation with fallbacks
- **File Structure**: Mixed `.js`/`.tsx` - components in TypeScript, API routes in JavaScript
- **Database**: Prisma with SQLite (`DATABASE_URL=file:./prisma/dev.db`)

## Key Integration Points
- **Session Persistence**: `.tradingview-session/` directory volume-mounted
- **Screenshots**: `screenshots/` directory for chart captures
- **Progress Tracking**: EventEmitter-based real-time updates via SSE
- **Multi-Stage Docker**: Development vs production builds with browser optimization
- **CAPTCHA Handling**: Manual CAPTCHA mode with X11 forwarding (`ALLOW_MANUAL_CAPTCHA=true`)

## Development vs Production Modes
- **Development**: Port 9001:3000, hot reload, debug logging, headless: false option
- **Production**: Port 9000:3000, optimized build, minimal logging, always headless

### Git Branch Strategy (Required)
**Primary development workflow:**
- **`development` branch**: Use for all active development and feature work
- **`main` branch**: Stable, production-ready code only
- **Workflow**: Develop on `development` → test thoroughly → merge to `main` when stable

```bash
# Standard development workflow
git checkout development        # Always start here
git pull origin development     # Get latest changes
# Make your changes...
git add . && git commit -m "feat: description"
git push origin development

# Only merge to main when features are stable and tested and you have asked the user to merge to main
git checkout main
git merge development          # When ready for production
git push origin main
```

When working with this codebase, prioritize Docker consistency, understand the dual-session architecture, and leverage the comprehensive test suite to validate changes.