trading_bot_v4/.github/copilot-instructions.md

AI Agent Instructions for Trading Bot v4

Architecture Overview

Type: Autonomous cryptocurrency trading bot with Next.js 15 frontend + Solana/Drift Protocol backend

Data Flow: TradingView → n8n webhook → Next.js API → Drift Protocol (Solana DEX) → Real-time monitoring → Auto-exit

Key Design Principle: Dual-layer redundancy - every trade has both on-chain orders (Drift) AND software monitoring (Position Manager) as backup.

Exit Strategy: Three-tier scaling system:

• TP1 at +0.4%: Close 75% (configurable via TAKE_PROFIT_1_SIZE_PERCENT)
• TP2 at +0.7%: Close 80% of remaining = 20% total (configurable via TAKE_PROFIT_2_SIZE_PERCENT)
• Runner: 5% remaining with 0.3% trailing stop (configurable via TRAILING_STOP_PERCENT)

Per-Symbol Configuration: SOL and ETH have independent enable/disable toggles and position sizing:

• SOLANA_ENABLED, SOLANA_POSITION_SIZE, SOLANA_LEVERAGE (defaults: true, $210, 10x)
• ETHEREUM_ENABLED, ETHEREUM_POSITION_SIZE, ETHEREUM_LEVERAGE (defaults: true, $4, 1x)
• BTC and other symbols fall back to global settings (MAX_POSITION_SIZE_USD, LEVERAGE)
• Priority: Per-symbol ENV → Market config → Global ENV → Defaults

Signal Quality System: Filters trades based on 5 metrics (ATR, ADX, RSI, volumeRatio, pricePosition) scored 0-100. Only trades scoring 60+ are executed. Scores stored in database for future optimization.

Timeframe-Aware Scoring: Signal quality thresholds adjust based on timeframe (5min vs daily):

• 5min: ADX 12+ trending (vs 18+ for daily), ATR 0.2-0.7% healthy (vs 0.4%+ for daily)
• Anti-chop filter: -20 points for extreme sideways (ADX <10, ATR <0.25%, Vol <0.9x)
• Pass timeframe param to scoreSignalQuality() from TradingView alerts (e.g., timeframe: "5")

MAE/MFE Tracking: Every trade tracks Maximum Favorable Excursion (best profit %) and Maximum Adverse Excursion (worst loss %) updated every 2s. Used for data-driven optimization of TP/SL levels.

Manual Trading via Telegram: Send plain-text messages like long sol, short eth, long btc to open positions instantly (bypasses n8n, calls /api/trading/execute directly with preset healthy metrics).

Critical Components

1. Signal Quality Scoring (lib/trading/signal-quality.ts)

Purpose: Unified quality validation system that scores trading signals 0-100 based on 5 market metrics

Timeframe-aware thresholds:

scoreSignalQuality({ 
  atr, adx, rsi, volumeRatio, pricePosition, 
  timeframe?: string // "5" for 5min, undefined for higher timeframes
})

5min chart adjustments:

• ADX healthy range: 12-22 (vs 18-30 for daily)
• ATR healthy range: 0.2-0.7% (vs 0.4%+ for daily)
• Anti-chop filter: -20 points for extreme sideways (ADX <10, ATR <0.25%, Vol <0.9x)

Price position penalties (all timeframes):

• Long at 90-95%+ range: -15 to -30 points (chasing highs)
• Short at <5-10% range: -15 to -30 points (chasing lows)
• Prevents flip-flop losses from entering range extremes

Key behaviors:

• Returns score 0-100 and detailed breakdown object
• Minimum score 60 required to execute trade
• Called by both /api/trading/check-risk and /api/trading/execute
• Scores saved to database for post-trade analysis

2. Position Manager (lib/trading/position-manager.ts)

Purpose: Software-based monitoring loop that checks prices every 2 seconds and closes positions via market orders

Singleton pattern: Always use getInitializedPositionManager() - never instantiate directly

const positionManager = await getInitializedPositionManager()
await positionManager.addTrade(activeTrade)

Key behaviors:

• Tracks ActiveTrade objects in a Map
• Three-tier exits: TP1 (75%), TP2 (80% of remaining), Runner (with trailing stop)
• Dynamic SL adjustments: Moves to breakeven after TP1, locks profit at +1.2%
• On-chain order synchronization: After TP1 hits, calls cancelAllOrders() then placeExitOrders() with updated SL price at breakeven
• Trailing stop: Activates after TP2, tracks peakPrice and trails by configured %
• Closes positions via closePosition() market orders when targets hit
• Acts as backup if on-chain orders don't fill
• State persistence: Saves to database, restores on restart via configSnapshot.positionManagerState
• Grace period for new trades: Skips "external closure" detection for positions <30 seconds old (Drift positions take 5-10s to propagate)
• Exit reason detection: Uses trade state flags (tp1Hit, tp2Hit) and realized P&L to determine exit reason, NOT current price (avoids misclassification when price moves after order fills)

3. Telegram Bot (telegram_command_bot.py)

Purpose: Python-based Telegram bot for manual trading commands and position status monitoring

Manual trade commands via plain text:

# User sends plain text message (not slash commands)
"long sol"   → Opens SOL-PERP long position
"short eth"  → Opens ETH-PERP short position
"long btc"   → Opens BTC-PERP long position

Key behaviors:

• MessageHandler processes all text messages (not just commands)
• Maps user-friendly symbols (sol, eth, btc) to Drift format (SOL-PERP, etc.)
• Calls /api/trading/execute directly with preset healthy metrics (ATR=1.0, ADX=25, RSI=50, volumeRatio=1.2)
• Bypasses n8n workflow and TradingView requirements
• 60-second timeout for API calls
• Responds with trade confirmation or error message

Status command:

/status → Returns JSON of open positions from Drift

Implementation details:

• Uses python-telegram-bot library
• Deployed via docker-compose.telegram-bot.yml
• Requires TELEGRAM_BOT_TOKEN and TELEGRAM_CHANNEL_ID in .env
• API calls to http://trading-bot:3000/api/trading/execute

Drift client integration:

• Singleton pattern: Use initializeDriftService() and getDriftService() - maintains single connection

const driftService = await initializeDriftService()
const health = await driftService.getAccountHealth()

• Wallet handling: Supports both JSON array [91,24,...] and base58 string formats from Phantom wallet

4. Order Placement (lib/drift/orders.ts)

Critical functions:

• openPosition() - Opens market position with transaction confirmation
• closePosition() - Closes position with transaction confirmation
• placeExitOrders() - Places TP/SL orders on-chain

CRITICAL: Transaction Confirmation Pattern Both openPosition() and closePosition() MUST confirm transactions on-chain:

const txSig = await driftClient.placePerpOrder(orderParams)
console.log('⏳ Confirming transaction on-chain...')
const connection = driftService.getConnection()
const confirmation = await connection.confirmTransaction(txSig, 'confirmed')

if (confirmation.value.err) {
  throw new Error(`Transaction failed: ${JSON.stringify(confirmation.value.err)}`)
}
console.log('✅ Transaction confirmed on-chain')

Without this, the SDK returns signatures for transactions that never execute, causing phantom trades/closes.

Dual Stop System (USE_DUAL_STOPS=true):

// Soft stop: TRIGGER_LIMIT at -1.5% (avoids wicks)
// Hard stop: TRIGGER_MARKET at -2.5% (guarantees exit)

Order types:

• Entry: MARKET (immediate execution)
• TP1/TP2: LIMIT reduce-only orders
• Soft SL: TRIGGER_LIMIT reduce-only
• Hard SL: TRIGGER_MARKET reduce-only

5. Database (lib/database/trades.ts + prisma/schema.prisma)

Purpose: PostgreSQL via Prisma ORM for trade history and analytics

Models: Trade, PriceUpdate, SystemEvent, DailyStats

Singleton pattern: Use getPrismaClient() - never instantiate PrismaClient directly

Key functions:

• createTrade() - Save trade after execution (includes dual stop TX signatures + signalQualityScore)
• updateTradeExit() - Record exit with P&L
• addPriceUpdate() - Track price movements (called by Position Manager)
• getTradeStats() - Win rate, profit factor, avg win/loss
• getLastTrade() - Fetch most recent trade for analytics dashboard

Important fields:

• signalQualityScore (Int?) - 0-100 score for data-driven optimization
• maxFavorableExcursion / maxAdverseExcursion - Track best/worst P&L during trade lifetime
• maxFavorablePrice / maxAdversePrice - Track prices at MFE/MAE points
• configSnapshot (Json) - Stores Position Manager state for crash recovery
• atr, adx, rsi, volumeRatio, pricePosition - Context metrics from TradingView

Per-symbol functions:

• getLastTradeTimeForSymbol(symbol) - Get last trade time for specific coin (enables per-symbol cooldown)
• Each coin (SOL/ETH/BTC) has independent cooldown timer to avoid missed opportunities

Configuration System

Three-layer merge:

1. DEFAULT_TRADING_CONFIG (config/trading.ts)
2. Environment variables (.env) via getConfigFromEnv()
3. Runtime overrides via getMergedConfig(overrides)

Always use: getMergedConfig() to get final config - never read env vars directly in business logic

Per-symbol position sizing: Use getPositionSizeForSymbol(symbol, config) which returns { size, leverage, enabled }

const { size, leverage, enabled } = getPositionSizeForSymbol('SOL-PERP', config)
if (!enabled) {
  return NextResponse.json({ success: false, error: 'Symbol trading disabled' }, { status: 400 })
}

Symbol normalization: TradingView sends "SOLUSDT" → must convert to "SOL-PERP" for Drift

const driftSymbol = normalizeTradingViewSymbol(body.symbol)

API Endpoints Architecture

Authentication: All /api/trading/* endpoints (except /test) require Authorization: Bearer API_SECRET_KEY

Pattern: Each endpoint follows same flow:

1. Auth check
2. Get config via getMergedConfig()
3. Initialize Drift service
4. Check account health
5. Execute operation
6. Save to database
7. Add to Position Manager if applicable

Key endpoints:

• /api/trading/execute - Main entry point from n8n (production, requires auth)
• /api/trading/check-risk - Pre-execution validation (duplicate check, quality score, per-symbol cooldown, rate limits, symbol enabled check)
• /api/trading/test - Test trades from settings UI (no auth required, respects symbol enable/disable)
• /api/trading/close - Manual position closing
• /api/trading/positions - Query open positions from Drift
• /api/settings - Get/update config (writes to .env file, includes per-symbol settings)
• /api/analytics/last-trade - Fetch most recent trade details for dashboard (includes quality score)
• /api/restart - Create restart flag for watch-restart.sh script

Critical Workflows

Execute Trade (Production)

TradingView alert → n8n Parse Signal Enhanced (extracts metrics + timeframe)
  ↓ /api/trading/check-risk [validates quality score ≥60, checks duplicates, per-symbol cooldown]
  ↓ /api/trading/execute
  ↓ normalize symbol (SOLUSDT → SOL-PERP)
  ↓ getMergedConfig()
  ↓ getPositionSizeForSymbol() [check if symbol enabled + get sizing]
  ↓ openPosition() [MARKET order]
  ↓ calculate dual stop prices if enabled
  ↓ placeExitOrders() [on-chain TP1/TP2/SL orders]
  ↓ scoreSignalQuality({ ..., timeframe }) [compute 0-100 score with timeframe-aware thresholds]
  ↓ createTrade() [save to database with signalQualityScore]
  ↓ positionManager.addTrade() [start monitoring]

Position Monitoring Loop

Position Manager every 2s:
  ↓ Verify on-chain position still exists (detect external closures)
  ↓ getPythPriceMonitor().getLatestPrice()
  ↓ Calculate current P&L and update MAE/MFE metrics
  ↓ Check emergency stop (-2%) → closePosition(100%)
  ↓ Check SL hit → closePosition(100%)
  ↓ Check TP1 hit → closePosition(75%), cancelAllOrders(), placeExitOrders() with SL at breakeven
  ↓ Check profit lock trigger (+1.2%) → move SL to +configured%
  ↓ Check TP2 hit → closePosition(80% of remaining), activate runner
  ↓ Check trailing stop (if runner active) → adjust SL dynamically based on peakPrice
  ↓ addPriceUpdate() [save to database every N checks]
  ↓ saveTradeState() [persist Position Manager state + MAE/MFE for crash recovery]

Settings Update

Web UI → /api/settings POST
  ↓ Validate new settings
  ↓ Write to .env file using string replacement
  ↓ Return success
  ↓ User clicks "Restart Bot" → /api/restart
  ↓ Creates /tmp/trading-bot-restart.flag
  ↓ watch-restart.sh detects flag
  ↓ Executes: docker restart trading-bot-v4

Docker Context

Multi-stage build: deps → builder → runner (Node 20 Alpine)

Critical Dockerfile steps:

1. Install deps with npm install --production
2. Copy source and npx prisma generate (MUST happen before build)
3. npm run build (Next.js standalone output)
4. Runner stage copies standalone + static + node_modules + Prisma client

Container networking:

• External: trading-bot-v4 on port 3001
• Internal: Next.js on port 3000
• Database: trading-bot-postgres on 172.28.0.0/16 network

DATABASE_URL caveat: Use trading-bot-postgres (container name) in .env for runtime, but localhost:5432 for Prisma CLI migrations from host

Project-Specific Patterns

1. Singleton Services

Never create multiple instances - always use getter functions:

const driftService = await initializeDriftService() // NOT: new DriftService()
const positionManager = getPositionManager()        // NOT: new PositionManager()
const prisma = getPrismaClient()                     // NOT: new PrismaClient()

2. Price Calculations

Direction matters for long vs short:

function calculatePrice(entry: number, percent: number, direction: 'long' | 'short') {
  if (direction === 'long') {
    return entry * (1 + percent / 100)  // Long: +1% = higher price
  } else {
    return entry * (1 - percent / 100)  // Short: +1% = lower price
  }
}

3. Error Handling

Database failures should not fail trades - always wrap in try/catch:

try {
  await createTrade(params)
  console.log('💾 Trade saved to database')
} catch (dbError) {
  console.error('❌ Failed to save trade:', dbError)
  // Don't fail the trade if database save fails
}

4. Reduce-Only Orders

All exit orders MUST be reduce-only (can only close, not open positions):

const orderParams = {
  reduceOnly: true,  // CRITICAL for TP/SL orders
  // ... other params
}

Testing Commands

# Local development
npm run dev

# Build production
npm run build && npm start

# Docker build and restart
docker compose build trading-bot
docker compose up -d --force-recreate trading-bot
docker logs -f trading-bot-v4

# Database operations
npx prisma generate                                    # Generate client
DATABASE_URL="postgresql://...@localhost:5432/..." npx prisma migrate dev
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\dt"

# Test trade from UI
# Go to http://localhost:3001/settings
# Click "Test LONG" or "Test SHORT"

Common Pitfalls

1. Prisma not generated in Docker: Must run npx prisma generate in Dockerfile BEFORE npm run build

2. Wrong DATABASE_URL: Container runtime needs trading-bot-postgres, Prisma CLI from host needs localhost:5432

3. Symbol format mismatch: Always normalize with normalizeTradingViewSymbol() before calling Drift (applies to ALL endpoints including /api/trading/close)

4. Missing reduce-only flag: Exit orders without reduceOnly: true can accidentally open new positions

5. Singleton violations: Creating multiple DriftClient or Position Manager instances causes connection/state issues

6. Type errors with Prisma: The Trade type from Prisma is only available AFTER npx prisma generate - use explicit types or // @ts-ignore carefully

7. Quality score duplication: Signal quality calculation exists in BOTH check-risk and execute endpoints - keep logic synchronized

8. Runner configuration confusion:

   • TAKE_PROFIT_1_SIZE_PERCENT=75 means "close 75% at TP1" (not "keep 75%")
   • TAKE_PROFIT_2_SIZE_PERCENT=80 means "close 80% of REMAINING" (not of original)
   • Actual runner size = (100 - TP1%) × (100 - TP2%) / 100 = 5% with defaults

9. Transaction confirmation CRITICAL: Both openPosition() AND closePosition() MUST call connection.confirmTransaction() after placePerpOrder(). Without this, the SDK returns transaction signatures that aren't confirmed on-chain, causing "phantom trades" or "phantom closes". Always check confirmation.value.err before proceeding.

10. Execution order matters: When creating trades via API endpoints, the order MUST be:

    1. Open position + place exit orders
    2. Save to database (createTrade())
    3. Add to Position Manager (positionManager.addTrade())

    If Position Manager is added before database save, race conditions occur where monitoring checks before the trade exists in DB.

11. New trade grace period: Position Manager skips "external closure" detection for trades <30 seconds old because Drift positions take 5-10 seconds to propagate after opening. Without this grace period, new positions are immediately detected as "closed externally" and cancelled.

12. Drift minimum position sizes: Actual minimums differ from documentation:

    • SOL-PERP: 0.1 SOL (~$5-15 depending on price)
    • ETH-PERP: 0.01 ETH (~$38-40 at $4000/ETH)
    • BTC-PERP: 0.0001 BTC (~$10-12 at $100k/BTC)

    Always calculate: minOrderSize × currentPrice must exceed Drift's $4 minimum. Add buffer for price movement.

13. Exit reason detection bug: Position Manager was using current price to determine exit reason, but on-chain orders filled at a DIFFERENT price in the past. Now uses trade.tp1Hit / trade.tp2Hit flags and realized P&L to correctly identify whether TP1, TP2, or SL triggered. Prevents profitable trades being mislabeled as "SL" exits.

14. Per-symbol cooldown: Cooldown period is per-symbol, NOT global. ETH trade at 10:00 does NOT block SOL trade at 10:01. Each coin (SOL/ETH/BTC) has independent cooldown timer to avoid missing opportunities on different assets.

15. Timeframe-aware scoring crucial: Signal quality thresholds MUST adjust for 5min vs higher timeframes:

    • 5min charts naturally have lower ADX (12-22 healthy) and ATR (0.2-0.7% healthy) than daily charts
    • Without timeframe awareness, valid 5min breakouts get blocked as "low quality"
    • Anti-chop filter applies -20 points for extreme sideways regardless of timeframe
    • Always pass timeframe parameter from TradingView alerts to scoreSignalQuality()

16. Price position chasing causes flip-flops: Opening longs at 90%+ range or shorts at <10% range reliably loses money:

    • Database analysis showed overnight flip-flop losses all had price position 9-94% (chasing extremes)
    • These trades had valid ADX (16-18) but entered at worst possible time
    • Quality scoring now penalizes -15 to -30 points for range extremes
    • Prevents rapid reversals when price is already overextended

17. TradingView ADX minimum for 5min: Set ADX filter to 15 (not 20+) in TradingView alerts for 5min charts:

    • Higher timeframes can use ADX 20+ for strong trends
    • 5min charts need lower threshold to catch valid breakouts
    • Bot's quality scoring provides second-layer filtering with context-aware metrics
    • Two-stage filtering (TradingView + bot) prevents both overtrading and missing valid signals

File Conventions

• API routes: app/api/[feature]/[action]/route.ts (Next.js 15 App Router)
• Services: lib/[service]/[module].ts (drift, pyth, trading, database)
• Config: Single source in config/trading.ts with env merging
• Types: Define interfaces in same file as implementation (not separate types directory)
• Console logs: Use emojis for visual scanning: 🎯 🚀 ✅ ❌ 💰 📊 🛡️

Per-Symbol Trading Controls

Purpose: Independent enable/disable toggles and position sizing for SOL and ETH to support different trading strategies (e.g., ETH for data collection at minimal size, SOL for profit generation).

Configuration Priority:

1. Per-symbol ENV vars (highest priority)
   • SOLANA_ENABLED, SOLANA_POSITION_SIZE, SOLANA_LEVERAGE
   • ETHEREUM_ENABLED, ETHEREUM_POSITION_SIZE, ETHEREUM_LEVERAGE
2. Market-specific config (from MARKET_CONFIGS in config/trading.ts)
3. Global ENV vars (fallback for BTC and other symbols)
   • MAX_POSITION_SIZE_USD, LEVERAGE
4. Default config (lowest priority)

Settings UI: app/settings/page.tsx has dedicated sections:

• 💎 Solana section: Toggle + position size + leverage + risk calculator
• ⚡ Ethereum section: Toggle + position size + leverage + risk calculator
• 💰 Global fallback: For BTC-PERP and future symbols

Example usage:

// In execute/test endpoints
const { size, leverage, enabled } = getPositionSizeForSymbol(driftSymbol, config)
if (!enabled) {
  return NextResponse.json({
    success: false,
    error: 'Symbol trading disabled'
  }, { status: 400 })
}

Test buttons: Settings UI has symbol-specific test buttons:

• 💎 Test SOL LONG/SHORT (disabled when SOLANA_ENABLED=false)
• ⚡ Test ETH LONG/SHORT (disabled when ETHEREUM_ENABLED=false)

When Making Changes

1. Adding new config: Update DEFAULT_TRADING_CONFIG + getConfigFromEnv() + .env file
2. Adding database fields: Update prisma/schema.prisma → npx prisma migrate dev → npx prisma generate → rebuild Docker
3. Changing order logic: Test with DRY_RUN=true first, use small position sizes ($10)
4. API endpoint changes: Update both endpoint + corresponding n8n workflow JSON (Check Risk and Execute Trade nodes)
5. Docker changes: Rebuild with docker compose build trading-bot then restart container
6. Modifying quality score logic: Update BOTH /api/trading/check-risk and /api/trading/execute endpoints, ensure timeframe-aware thresholds are synchronized
7. Exit strategy changes: Modify Position Manager logic + update on-chain order placement in placeExitOrders()
8. TradingView alert changes: Ensure alerts pass timeframe field (e.g., "timeframe": "5") to enable proper signal quality scoring

Development Roadmap

See POSITION_SCALING_ROADMAP.md for planned optimizations:

• Phase 1 (CURRENT): Collect data with quality scores (20-50 trades needed)
• Phase 2: ATR-based dynamic targets (adapt to volatility)
• Phase 3: Signal quality-based scaling (high quality = larger runners)
• Phase 4: Direction-based optimization (shorts vs longs have different performance)
• Phase 5: Optimize runner size (5% → 10-25%) and trailing stop (0.3% fixed → ATR-based)
• Phase 6: ML-based exit prediction (future)

Data-driven approach: Each phase requires validation through SQL analysis before implementation. No premature optimization.

Integration Points

• n8n: Expects exact response format from /api/trading/execute (see n8n-complete-workflow.json)
• Drift Protocol: Uses SDK v2.75.0 - check docs at docs.drift.trade for API changes
• Pyth Network: WebSocket + HTTP fallback for price feeds (handles reconnection)
• PostgreSQL: Version 16-alpine, must be running before bot starts

---

Key Mental Model: Think of this as two parallel systems (on-chain orders + software monitoring) working together. The Position Manager is the "backup brain" that constantly watches and acts if on-chain orders fail. Both write to the same database for complete trade history.