trading_bot_v4/README.md

# Trading Bot v4 🚀

**Fully Autonomous Trading Bot** with Dual-Layer Redundancy for TradingView → n8n → Drift Protocol (Solana)

## 📊 Live Status

**Capital:** $97.55 USDC | **Target:** $2,500 (Phase 1) | **Leverage:** 15x (SOL)  
**Win Rate:** 60%+ target | **Trades Executed:** 161 | **System:** ✅ Operational

### Development Status

| Phase | Status | Description |
|-------|--------|-------------|
| Phase 1-4 | ✅ **COMPLETE** | Core system: Execution, monitoring, UI, database |
| Phase 5 | 🔄 **IN PROGRESS** | Data collection & optimization (see below) |

## 🎯 Active Optimization Initiatives

Three parallel data-driven improvements targeting **35-40% P&L increase** over 3 months:

### 1. Signal Quality Optimization (🔜 2-3 weeks)
Filter bad trades BEFORE entry by optimizing quality thresholds
```
Progress: [▓░░░░░░░░░] 0/20 blocked signals collected (0%)
Status:   Phase 1 - Data Collection
Impact:   +5% win rate expected (60% → 65%)
```

### 2. Position Scaling & Exit Strategy (🔜 3-4 weeks)
Optimize HOW MUCH to close at each target based on signal quality
```
Progress: [▓▓▓▓▓▓▓▓▓▓] 161/50 trades (322%) ✅ - Need v6 indicator data
Status:   Phase 5 complete (TP2-as-runner), Phase 2-4 pending analysis
Impact:   +15% average win size expected
```

### 3. ATR-Based Take Profit Levels (🔜 6-8 weeks)
Replace fixed % targets with volatility-adaptive ATR multipliers
```
Progress: [▓░░░░░░░░░] 1/50 trades with ATR (2%)
Status:   Phase 1 - Data Collection
Impact:   +10% total P&L expected (better hit rates in all conditions)
```

**📈 Combined Expected Impact:** 35-40% improvement → Phase 1 goal acceleration from 6-7 months to 4-5 months

📋 [**View Master Roadmap**](./OPTIMIZATION_MASTER_ROADMAP.md) for detailed timeline and implementation plan

## What It Does

1. **Receives signals** from TradingView (5-minute OR 15-minute charts)
2. **Executes trades** on Drift Protocol (Solana DEX) with dual stop-loss system
3. **Monitors positions** every 2 seconds via Pyth Network WebSocket + HTTP fallback
4. **Closes positions** automatically at TP1 (partial) / TP2 (80%) / SL (100%)
5. **Adjusts stops** dynamically (breakeven at +0.5%, profit lock at +1.2%)
6. **Tracks everything** in PostgreSQL (trades, prices, P&L, win rate)
7. **Provides web UI** for configuration, monitoring, and analytics

**100% autonomous. Dual-layer safety. No manual intervention required!**

## Architecture: Dual-Layer Redundancy

**Key Design Principle:** Every trade has **TWO independent exit mechanisms**:

1. **On-Chain Orders (Drift Protocol)** - Primary layer
   - TP1/TP2 as LIMIT orders
   - Soft SL as TRIGGER_LIMIT (-1.5%, avoids wicks)
   - Hard SL as TRIGGER_MARKET (-2.5%, guarantees exit)

2. **Software Monitoring (Position Manager)** - Backup layer
   - Checks prices every 2 seconds
   - Closes via MARKET orders if on-chain orders fail
   - Dynamic SL adjustments
   - Emergency stop functionality

**Why?** If Drift orders don't fill (low liquidity, network issues), Position Manager acts as backup. Both write to the same PostgreSQL database for complete trade history.

## 📈 The Money Line v8 Indicator

### How It Works (Simple Explanation)

The **Money Line** is a dynamic support/resistance indicator that adapts to market volatility. Think of it as a "safety zone" - price above = bullish, price below = bearish.

**The Core Concept:**
- **Green line** = Bulls in control, momentum up → Go LONG
- **Red line** = Bears in control, momentum down → Go SHORT  
- **Line flip** = Power shift detected → Enter trade

### What Creates the Line

**1. ATR (Average True Range)**
- Measures how much the market moves on average
- Volatile markets → wider bands, calm markets → tighter bands
- Think: "How much wiggle room does price need?"

**2. The Multiplier**
- ATR × Multiplier = Distance from price to line
- v8 uses 3.8x for 5min charts (stickier than previous versions)
- Higher multiplier = line changes color less often

**3. HalfTrend Algorithm**
- Calculates upper band (resistance) and lower band (support)
- Price crosses above upper band → line turns GREEN (bullish)
- Price crosses below lower band → line turns RED (bearish)
- Line "sticks" to its color until price STRONGLY breaks the opposite band

### v8 Improvements (Anti-Whipsaw Protection)

**Flip Threshold (0.6%)**
- Price must move 0.6% beyond the line before it changes color
- Filters tiny bounces that don't mean anything
- Example: Line at $100 → price must hit $100.60 to flip bullish

**Entry Buffer (0.2 ATR)**
- Confirms the breakout is real, not just a wick
- Waits for price to close beyond line + buffer zone
- Prevents fake signals from sudden spikes

**Stickier Bands**
- Increased multipliers make the line "lazier" to flip
- Follows major trends, ignores minor corrections

### In Simple Terms

*"It's like a traffic light for trading:*
- *Green = Go long (buy)*
- *Red = Go short (sell)*
- *The line doesn't change until there's a REAL trend shift, filtering out fake moves.*

*It adapts to volatility automatically - in crazy markets it needs bigger moves to flip, in calm markets it's more sensitive. This keeps you trading WITH the flow, not against it."*

**Technical Name:** Modified HalfTrend Supertrend with ATR-based adaptive bands and flip threshold confirmation  
**Street Name:** The Money Line 💰

### Indicator Versions
- **v6:** Filtered mode with ADX/volume/RSI confirmations (conservative)
- **v7:** Pure line flips (too noisy, deprecated)
- **v8:** Sticky trend with flip threshold + momentum confirmation (current)

All versions available in `workflows/trading/` directory.

## Quick Start (Docker)

### 1. Prerequisites
- Docker & Docker Compose installed
- Solana wallet with Drift Protocol account
- Helius RPC API key (mainnet)
- TradingView alerts → n8n webhook setup

### 2. Deploy with Docker Compose
```bash
# Clone and setup
cd /home/icke/traderv4

# Configure .env file (copy from .env.example)
nano .env

# Build and start all services
docker compose up -d

# View logs
docker compose logs -f trading-bot

# Stop
docker compose down
```

### 3. Access Web Interface
- **Settings UI:** `http://YOUR_HOST:3001/settings`
- **Analytics:** `http://YOUR_HOST:3001/analytics`
- **API Endpoints:** `http://YOUR_HOST:3001/api/`

### 4. Configure Settings
Open `http://YOUR_HOST:3001/settings` in your browser to:
- Adjust position size ($10-$10,000) and leverage (1x-20x)
- Set stop-loss (-1.5% soft, -2.5% hard) and take-profit levels
- Configure dynamic stop-loss (breakeven +0.5%, profit lock +1.2%)
- Set daily loss limits and max trades per hour
- Toggle DRY_RUN mode for paper trading

After saving, click **"Restart Bot"** to apply changes.

### 5. Setup n8n Workflow
Import `workflows/trading/Money_Machine.json` into your n8n instance:
- Configure TradingView webhook URL
- Set timeframe filters (5min and/or 15min)
- Add API authentication header
- Test with manual execution

## Alternative: Manual Setup

### 1. Install Dependencies
```bash
npm install
```

### 2. Configure Environment
```bash
# Copy and edit .env
cp .env.example .env

# Required variables:
DRIFT_WALLET_PRIVATE_KEY=[your_wallet_array]
SOLANA_RPC_URL=https://mainnet.helius-rpc.com/?api-key=YOUR_KEY
API_SECRET_KEY=your_random_secret_key
DRY_RUN=false
```

### 3. Run
```bash
# Development
npm run dev

# Production
npm run build
npm start
```

---

## Core Features

### Dual Stop-Loss System
- **Soft Stop** (TRIGGER_LIMIT): -1.5% from entry, avoids wick-outs
- **Hard Stop** (TRIGGER_MARKET): -2.5% from entry, guarantees exit
- Both placed on-chain as reduce-only orders
- Position Manager monitors as backup (closes via MARKET if needed)

### Dynamic Stop-Loss Adjustments
- **Breakeven**: Moves SL to +0.01% when price hits +0.5%
- **Profit Lock**: Locks in profit when price hits +1.2%
- **Reduces risk** while letting winners run

### Take-Profit Strategy
- **TP1** (default +0.7%): Closes 50% of position, moves SL to breakeven
- **TP2** (default +1.5%): Closes 80% of remaining position
- **Runner**: 20% remains open if takeProfit2SizePercent < 100%

### Position Manager (Singleton)
- Monitors all positions every 2 seconds
- Singleton pattern: Use `getPositionManager()` - never instantiate directly
- Tracks price updates in database
- Closes positions when targets hit
- Cancels orphaned orders automatically
- Acts as backup if on-chain orders don't fill

### Database Integration (PostgreSQL + Prisma)
**Models:**
- `Trade`: Complete trade history with entry/exit data
- `PriceUpdate`: Price movements every 2 seconds during monitoring
- `SystemEvent`: Errors, restarts, important events
- `DailyStats`: Win rate, profit factor, avg win/loss

**Analytics:**
- Real-time P&L tracking
- Win rate and profit factor
- Best/worst trades
- Drawdown monitoring

### Configuration System (Three-Layer Merge)
1. **Defaults** (`config/trading.ts` - DEFAULT_TRADING_CONFIG)
2. **Environment** (`.env` file via `getConfigFromEnv()`)
3. **Runtime** (API overrides via `getMergedConfig(overrides)`)

Always use `getMergedConfig()` in business logic - never read env vars directly.

### Safety Features
- **Reduce-only orders**: All TP/SL orders can only close, not open positions
- **Account health checks**: Validates margin before every trade
- **Risk validation**: `/api/trading/check-risk` endpoint
- **Daily loss limits**: Stops trading after max loss reached
- **Cooldown periods**: Prevents over-trading
- **DRY_RUN mode**: Paper trading for testing

---

## How It Works: Complete Trade Flow

### 1. Signal Reception (TradingView → n8n)
```
TradingView Alert: "LONG SOLUSDT .P 15"
  ↓
n8n Webhook receives signal
  ↓
Parse Signal node extracts:
  - Symbol: SOLUSDT → SOL-PERP (normalized)
  - Direction: long
  - Timeframe: 15 (from .P 15)
  ↓
Timeframe Filter: Allow 5 or 15 minutes only
  ↓
Check Risk: Validate position limits, daily loss, etc.
```

### 2. Trade Execution (n8n → Next.js API → Drift)
```
POST /api/trading/execute
  ↓
getMergedConfig() - Get current settings
  ↓
initializeDriftService() - Connect to Drift SDK
  ↓
Check account health (margin requirements)
  ↓
openPosition() - Execute MARKET order
  ↓
Calculate dual stop prices (soft -1.5%, hard -2.5%)
  ↓
placeExitOrders() - Place on-chain TP/SL orders
  ├─ TP1: LIMIT reduce-only at +0.7%
  ├─ TP2: LIMIT reduce-only at +1.5%
  ├─ Soft SL: TRIGGER_LIMIT reduce-only at -1.5%
  └─ Hard SL: TRIGGER_MARKET reduce-only at -2.5%
  ↓
createTrade() - Save to PostgreSQL database
  ↓
positionManager.addTrade() - Start monitoring loop
```

### 3. Position Monitoring (Every 2 Seconds)
```
Position Manager Loop:
  ↓
getPythPriceMonitor().getLatestPrice()
  ↓
Calculate current P&L and percentage gain/loss
  ↓
Check TP1 hit? → closePosition(75%) + move SL to breakeven
  ↓
Check TP2 hit? → closePosition(80% of remaining)
  ↓
Check SL hit? → closePosition(100%)
  ↓
Check dynamic adjustments:
  ├─ Price > +0.5%? → Move SL to breakeven (+0.01%)
  └─ Price > +1.2%? → Lock profit (move SL to +X%)
  ↓
addPriceUpdate() - Save price to database
  ↓
Repeat every 2 seconds until position closed
```

### 4. Position Exit (Automatic)
```
Exit Triggered (TP/SL hit):
  ↓
If closed by on-chain order:
  ├─ Position Manager detects position.size === 0
  ├─ Determines exit reason (TP1/TP2/SL from price)
  ├─ updateTradeExit() - Save exit data to database
  └─ removeTrade() - Stop monitoring + cancel orphaned orders
  ↓
If closed by Position Manager:
  ├─ closePosition() - Execute MARKET order
  ├─ cancelAllOrders() - Cancel remaining on-chain orders
  ├─ updateTradeExit() - Save exit data to database
  └─ removeTrade() - Stop monitoring
```

### 5. Order Cleanup (Automatic)
```
When position closes (100%):
  ↓
cancelAllOrders(symbol) - Query all open orders
  ↓
Filter by marketIndex and status === 0 (Open)
  ↓
driftClient.cancelOrders() - Cancel on Drift
  ↓
Logs: "Cancelled X orders" or "Cancelled X orphaned orders"
```

**Result:** Clean exit with no orphaned orders, complete trade history in database, ready for next signal.
---

## Web Interface

### Settings Page (`/settings`)
Beautiful web interface for managing all trading parameters:

**Position Sizing:**
- Adjust position size ($10-$10,000 USD)
- Set leverage (1x-20x)

**Risk Management:**
- Stop-loss percentage (soft -1.5%, hard -2.5%)
- Take-profit 1 & 2 levels
- Emergency stop level

**Dynamic Stop-Loss:**
- Breakeven trigger (+0.5%)
- Profit lock trigger and amount (+1.2%)

**Safety Limits:**
- Max daily loss
- Max trades per hour
- Cooldown between trades

**Execution:**
- Slippage tolerance
- DRY_RUN toggle for testing

**Live Risk Calculator:**
- Shows max loss in USD
- TP1 and TP2 gains
- Risk/Reward ratio

### Analytics Page (`/analytics`)
Real-time trading performance dashboard:
- Current open positions with live P&L
- Trade history with detailed entry/exit data
- Win rate and profit factor
- Total P&L (daily, weekly, monthly)
- Best and worst trades
- Drawdown tracking

### API Endpoints

All endpoints require `Authorization: Bearer YOUR_API_SECRET_KEY` (except `/api/trading/test`)

**Trade Execution:**
```bash
# Execute a trade (production - from n8n)
POST /api/trading/execute
{
  "symbol": "SOL-PERP",
  "direction": "long",
  "timeframe": "15",
  "signalStrength": "strong"
}

# Test trade (no auth required - from settings UI)
POST /api/trading/test
{
  "symbol": "SOL-PERP",
  "direction": "long",
  "timeframe": "15"
}

# Close a position (partial or full)
POST /api/trading/close
{
  "symbol": "SOL-PERP",
  "percentToClose": 100  // or 50, 75, etc.
}

# Get active positions
GET /api/trading/positions
# Returns: { positions: [...], monitoring: [...] }

# Validate trade (risk check)
POST /api/trading/check-risk
{
  "symbol": "SOL-PERP",
  "direction": "long"
}
```

**Settings Management:**
```bash
# Get current settings
GET /api/settings

# Update settings (writes to .env file)
POST /api/settings
{
  "MAX_POSITION_SIZE_USD": 100,
  "LEVERAGE": 10,
  "STOP_LOSS_PERCENT": -1.5,
  "SOFT_STOP_LOSS_PERCENT": -1.5,
  "HARD_STOP_LOSS_PERCENT": -2.5,
  "TAKE_PROFIT_1_PERCENT": 0.7,
  "TAKE_PROFIT_2_PERCENT": 1.5,
  "TAKE_PROFIT_2_SIZE_PERCENT": 80,
  "BREAKEVEN_TRIGGER_PERCENT": 0.5,
  "PROFIT_LOCK_TRIGGER_PERCENT": 1.2,
  "DRY_RUN": false
}

# Restart bot container (apply settings)
POST /api/restart
# Creates /tmp/trading-bot-restart.flag
# watch-restart.sh detects flag and runs: docker restart trading-bot-v4
```

**Analytics:**
```bash
# Get trade statistics
GET /api/analytics/stats
# Returns: { winRate, profitFactor, totalTrades, totalPnL, ... }

# Get recent trades
GET /api/analytics/positions
# Returns: { openPositions: [...], recentTrades: [...] }
```

**Symbol Normalization:**
- TradingView sends: `SOLUSDT`, `BTCUSDT`, `ETHUSDT`
- Bot converts to: `SOL-PERP`, `BTC-PERP`, `ETH-PERP`
- Always use Drift format in API calls

---

---

## Docker Deployment

### Architecture
- **Multi-stage build**: deps → builder → runner (Node 20 Alpine)
- **Next.js standalone** output for production (~400MB image)
- **PostgreSQL 16-alpine** for trade history
- **Isolated network** (172.28.0.0/16)
- **Health monitoring** and logging

### Container Details
- **trading-bot-v4**: Main application
  - Port: 3001 (external) → 3000 (internal)
  - Restart: unless-stopped
  - Volumes: .env file mounted

- **trading-bot-postgres**: Database
  - Port: 5432 (internal only)
  - Persistent volume: trading-bot-postgres-data
  - Auto-backup recommended

### Critical Build Steps
1. Install deps: `npm install --production`
2. Copy source and generate Prisma client: `npx prisma generate`
3. Build Next.js: `npm run build` (standalone mode)
4. Runner stage: Copy standalone + static + node_modules + Prisma client

**Why Prisma generate before build?** The Trade type from Prisma must exist before Next.js compiles TypeScript.

### Commands
```bash
# Build and deploy
docker compose build trading-bot
docker compose up -d

# View logs (real-time)
docker compose logs -f trading-bot

# View logs (last 100 lines)
docker compose logs --tail=100 trading-bot

# Restart after config changes
docker compose restart trading-bot

# Rebuild and restart (force recreate)
docker compose up -d --force-recreate trading-bot

# Stop everything
docker compose down

# Stop and remove volumes (WARNING: deletes database)
docker compose down -v
```

### Database Operations
```bash
# Connect to database
docker exec -it trading-bot-postgres psql -U postgres -d trading_bot_v4

# Run migrations from host
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/trading_bot_v4" npx prisma migrate dev

# Generate Prisma client
npx prisma generate

# View tables
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\dt"
```

**DATABASE_URL caveat:** Use `trading-bot-postgres` (container name) in .env for runtime, but `localhost:5432` for Prisma CLI migrations from host.

### Restart Watcher (Required for Web UI Restart Button)
The restart watcher monitors for restart requests from the web UI:

```bash
# Start watcher manually
cd /home/icke/traderv4
nohup ./watch-restart.sh > logs/restart-watcher.log 2>&1 &

# OR install as systemd service (recommended)
sudo cp trading-bot-restart-watcher.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable trading-bot-restart-watcher
sudo systemctl start trading-bot-restart-watcher

# Check watcher status
sudo systemctl status trading-bot-restart-watcher
```

The watcher enables the "Restart Bot" button in the web UI to automatically restart the container when settings are changed.

### Environment Variables
All settings configured via `.env` file:

**Required:**
- `DRIFT_WALLET_PRIVATE_KEY`: Solana wallet (JSON array or base58 string)
- `SOLANA_RPC_URL`: Helius RPC endpoint (mainnet recommended)
- `API_SECRET_KEY`: Random secret for API authentication
- `DATABASE_URL`: PostgreSQL connection string

**Trading Parameters:**
- `MAX_POSITION_SIZE_USD`: Position size in USD
- `LEVERAGE`: Leverage multiplier (1-20x)
- `STOP_LOSS_PERCENT`: Soft stop percentage (e.g., -1.5)
- `HARD_STOP_LOSS_PERCENT`: Hard stop percentage (e.g., -2.5)
- `TAKE_PROFIT_1_PERCENT`: TP1 target (e.g., 0.7)
- `TAKE_PROFIT_2_PERCENT`: TP2 target (e.g., 1.5)
- `TAKE_PROFIT_2_SIZE_PERCENT`: How much to close at TP2 (e.g., 80)

**Dynamic Stop-Loss:**
- `BREAKEVEN_TRIGGER_PERCENT`: Move to breakeven at (e.g., 0.5)
- `PROFIT_LOCK_TRIGGER_PERCENT`: Lock profit at (e.g., 1.2)
- `PROFIT_LOCK_AMOUNT_PERCENT`: Profit to lock (e.g., 0.5)

**Safety:**
- `MAX_DAILY_LOSS`: Max loss per day in USD
- `MAX_TRADES_PER_HOUR`: Rate limiting
- `TRADE_COOLDOWN_MINUTES`: Cooldown between trades
- `DRY_RUN`: Enable paper trading (true/false)
- `USE_DUAL_STOPS`: Enable dual stop system (true/false)

Changes to `.env` require container restart to take effect.

### Singleton Services (Critical Pattern)
**Never create multiple instances** - always use getter functions:

```typescript
// Drift Client
const driftService = await initializeDriftService()  // NOT: new DriftService()
const driftService = getDriftService()               // After init

// Position Manager
const positionManager = getPositionManager()         // NOT: new PositionManager()

// Database
const prisma = getPrismaClient()                     // NOT: new PrismaClient()
```

Creating multiple instances causes connection issues and state inconsistencies.

---

## File Structure

```
traderv4/
├── README.md                           ← You are here
├── docker-compose.yml                  ← Docker orchestration
├── Dockerfile                          ← Multi-stage build
├── .env                                ← Configuration (create from .env.example)
├── package.json                        ← Dependencies (Next.js 15, Drift SDK, Prisma)
├── next.config.js                      ← Next.js config (standalone output)
├── tsconfig.json                       ← TypeScript config
│
├── app/                                ← Next.js 15 App Router
│   ├── layout.tsx                      ← Root layout with Tailwind
│   ├── page.tsx                        ← Home page
│   ├── globals.css                     ← Global styles
│   ├── settings/
│   │   └── page.tsx                    ← Settings UI
│   ├── analytics/
│   │   └── page.tsx                    ← Analytics dashboard
│   └── api/
│       ├── settings/
│       │   └── route.ts                ← GET/POST settings, writes to .env
│       ├── restart/
│       │   └── route.ts                ← Creates restart flag file
│       ├── analytics/
│       │   ├── stats/route.ts          ← Trade statistics
│       │   └── positions/route.ts      ← Open/recent positions
│       └── trading/
│           ├── execute/route.ts        ← Main execution (production)
│           ├── test/route.ts           ← Test execution (UI)
│           ├── close/route.ts          ← Close positions
│           ├── positions/route.ts      ← Query positions
│           ├── check-risk/route.ts     ← Risk validation
│           └── remove-position/route.ts ← Remove from monitoring
│
├── lib/                                ← Business logic
│   ├── drift/
│   │   ├── client.ts                   ← Drift SDK wrapper (singleton)
│   │   └── orders.ts                   ← Order execution & cancellation
│   ├── pyth/
│   │   └── price-monitor.ts            ← WebSocket + HTTP fallback
│   ├── trading/
│   │   └── position-manager.ts         ← Monitoring loop (singleton)
│   ├── database/
│   │   ├── trades.ts                   ← Trade CRUD operations
│   │   └── views.ts                    ← Analytics queries
│   └── notifications/
│       └── telegram.ts                 ← Telegram alerts (optional)
│
├── config/
│   └── trading.ts                      ← Market configs & defaults
│
├── prisma/
│   ├── schema.prisma                   ← Database models
│   └── migrations/                     ← Migration history
│       ├── 20251026200052_init/
│       └── 20251027080947_add_test_trade_flag/
│
├── workflows/                          ← n8n workflow JSON files
│   ├── trading/
│   │   └── Money_Machine.json          ← Main trading workflow
│   ├── analytics/
│   │   ├── n8n-daily-report.json
│   │   ├── n8n-database-analytics.json
│   │   └── n8n-stop-loss-analysis.json
│   └── telegram/
│       └── telegram-webhook-FINAL.json
│
├── scripts/                            ← Utility scripts
│   ├── docker-build.sh
│   ├── docker-start.sh
│   ├── docker-stop.sh
│   ├── docker-logs.sh
│   ├── watch-restart.sh                ← Restart watcher daemon
│   ├── send_trade.sh                   ← Test trade execution
│   └── test-exit-orders.sh             ← Test exit order placement
│
├── tests/                              ← Test files
│   ├── test-drift-v4.ts
│   ├── test-full-flow.ts
│   ├── test-position-manager.ts
│   └── test-price-monitor.ts
│
├── docs/                               ← Documentation
│   ├── SETUP.md                        ← Detailed setup guide
│   ├── DOCKER.md                       ← Docker deployment
│   ├── TESTING.md                      ← Testing guide
│   ├── TELEGRAM_BOT_README.md          ← Telegram setup
│   ├── N8N_WORKFLOW_SETUP.md           ← n8n configuration
│   ├── PHASE_2_COMPLETE.md             ← Phase 2 features
│   └── QUICKREF_PHASE2.md              ← Quick reference
│
└── logs/                               ← Log files (created at runtime)
```

---

## Documentation

| Document | Purpose |
|----------|---------|
| `README.md` | This overview |
| `docs/setup/SETUP.md` | Detailed setup instructions |
| `docs/setup/DOCKER.md` | Docker deployment guide |
| `docs/setup/TELEGRAM_BOT_README.md` | Telegram bot setup |
| `docs/guides/TESTING.md` | Comprehensive testing guide |
| `docs/history/PHASE_2_COMPLETE.md` | Phase 2 feature overview |
| `workflows/trading/` | n8n workflow files |
- `../N8N_SETUP_GUIDE.md` - n8n configuration

---

## Trade Example (Real-World Flow)

### Entry Signal from TradingView
```
Alert Message: "LONG SOLUSDT .P 15"
  ↓
n8n receives webhook
  ↓
Parse: symbol=SOL-PERP, direction=long, timeframe=15
  ↓
Timeframe check: 15 minutes ✅ (allowed)
  ↓
Risk check: Daily loss OK, no existing position ✅
  ↓
Execute trade via API
```

### Position Opened
```
Symbol: SOL-PERP
Direction: LONG
Entry: $200.00
Position Size: $100 (10x leverage = $1,000 notional)

On-Chain Orders Placed:
├─ TP1: LIMIT at $201.40 (+0.7%) - Close 50%
├─ TP2: LIMIT at $203.00 (+1.5%) - Close 80% of remaining
├─ Soft SL: TRIGGER_LIMIT at $197.00 (-1.5%)
└─ Hard SL: TRIGGER_MARKET at $195.00 (-2.5%)

Position Manager: ✅ Monitoring started (every 2s)
Database: ✅ Trade #601 saved
```

### TP1 Hit (First Target)
```
Price reaches $201.40
  ↓
On-chain TP1 order fills → 50% closed
  ↓
Position Manager detects partial close:
  ├─ Profit: +$7.00 (+7% account)
  ├─ Remaining: 50% ($500 notional)
  ├─ Move SL to breakeven: $200.02 (+0.01%)
  └─ Trade is now RISK-FREE ✅
  ↓
Database updated: exitReason=TP1_PARTIAL
```

### Price Continues Higher
```
Price reaches $202.44 (+1.22%)
  ↓
Position Manager dynamic adjustment:
  ├─ Trigger: +1.2% profit lock activated
  ├─ Move SL to: $201.00 (+0.5% profit locked)
  └─ Letting winner run with locked profit ✅
```

### TP2 Hit (Second Target)
```
Price reaches $203.00
  ↓
On-chain TP2 order fills → 80% of remaining closed (40% of original)
  ↓
Position Manager detects:
  ├─ Profit from TP2: +$6.00
  ├─ Remaining: 10% runner ($100 notional)
  └─ Runner continues with locked profit SL
  ↓
Database updated: exitReason=TP2_PARTIAL
```

### Final Exit (Runner)
```
Option 1: Runner hits new high → Manual/trailing stop
Option 2: Price pulls back → Locked profit SL hits at $201.00
Option 3: Emergency stop or manual close

Total P&L:
├─ TP1: +$7.00 (50% at +0.7%)
├─ TP2: +$6.00 (40% at +1.5%)
└─ Runner: +$3.00 (10% at +3.0%, closed at pullback)
═══════════════════
Total: +$16.00 (+16% account growth)

Database: ✅ Trade #601 complete
```

### If Stop-Loss Hit Instead
```
Price drops to $197.00
  ↓
Soft SL (TRIGGER_LIMIT) activates:
  ├─ Triggers at $197.00
  ├─ Limit order at $196.95 (avoid wick)
  └─ If fills → Position closed ✅
  ↓
If soft SL doesn't fill (low liquidity):
  ├─ Price drops to $195.00
  ├─ Hard SL (TRIGGER_MARKET) activates
  └─ Market order guarantees exit ✅
  ↓
Position Manager backup:
  ├─ Detects position still open at -2.5%
  └─ Closes via MARKET order if needed
  ↓
Loss: -$25.00 (-2.5% account)
Database: ✅ exitReason=STOP_LOSS
```

**Key Points:**
- Dual stop system ensures exit even in volatile markets
- Position Manager acts as backup to on-chain orders
- Dynamic SL makes trades risk-free after +0.5%
- Profit lock captures gains before reversals
- Complete audit trail in database

---

## Common Issues & Solutions

### "Drift service not initialized"
**Cause:** Drift client not connected before trade execution
**Solution:** Ensure `initializeDriftService()` called before operations

### "Position not found in monitoring"
**Cause:** Race condition - orders placed after Position Manager started
**Solution:** ✅ FIXED - Orders now placed before monitoring starts

### "Orphaned orders after exit"
**Cause:** Old bug - Position Manager detected closure before orders existed
**Solution:** ✅ FIXED - Order placement sequencing corrected
**Cleanup:** Cancel manually on Drift UI or wait for next trade's auto-cleanup

### "TP2 closes entire position instead of 80%"
**Cause:** Old bug - TP2 calculated from original position instead of remaining
**Solution:** ✅ FIXED - TP2 now calculates from remaining after TP1

### "Database save failed but trade executed"
**Cause:** PostgreSQL connection issue or schema mismatch
**Solution:** Trade still executes successfully, check database logs and fix connection

### "Prisma Client not generated" (Docker build)
**Cause:** `npx prisma generate` not run before `npm run build`
**Solution:** Ensure Dockerfile runs Prisma generate in builder stage

### "Wrong DATABASE_URL" (localhost vs container)
**Container runtime:** Use `trading-bot-postgres:5432` in .env
**Prisma CLI (host):** Use `localhost:5432` for migrations
**Solution:** Maintain two DATABASE_URL values for different contexts

### Container won't restart after settings change
**Cause:** Restart watcher not running
**Solution:** 
```bash
sudo systemctl status trading-bot-restart-watcher
sudo systemctl start trading-bot-restart-watcher
```

---

## Safety Guidelines

1. **Start Small**: Use $10-50 positions for first 20 trades
2. **Test Thoroughly**: 
   - Run with DRY_RUN=true first
   - Execute test trades from settings UI
   - Verify all exits work correctly
3. **Monitor Closely**: Watch first 10 auto-exits in real-time
4. **Verify Database**: Check that all trades are being saved correctly
5. **Check Drift UI**: Confirm positions and orders match expectations
6. **Scale Gradually**: Increase position size 2x per week maximum
7. **Daily Review**: Check analytics page every day
8. **Backup Database**: Export PostgreSQL data weekly

**Risk Warning:** Cryptocurrency trading involves substantial risk. This bot executes trades automatically. Start small and never risk more than you can afford to lose.

---

## Testing Commands

```bash
# Local development
npm run dev

# Build production
npm run build && npm start

# Docker build and restart
docker compose build trading-bot
docker compose up -d --force-recreate trading-bot
docker logs -f trading-bot-v4

# Test trade from UI
# Go to http://localhost:3001/settings
# Click "Test LONG" or "Test SHORT"

# Test trade from terminal
./send_trade.sh LONG SOL-PERP 15

# Check current positions
curl -s -X GET http://localhost:3001/api/trading/positions \
  -H "Authorization: Bearer YOUR_API_SECRET_KEY" | jq

# Test exit order placement
./test-exit-orders.sh

# Database operations
npx prisma generate
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/trading_bot_v4" npx prisma migrate dev
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "SELECT * FROM \"Trade\" ORDER BY \"createdAt\" DESC LIMIT 5;"

# Check logs
docker compose logs --tail=100 trading-bot
docker compose logs -f trading-bot | grep "Position Manager"
```

---

## 📚 Documentation

**Complete documentation available in `/docs/` directory - organized by category:**

### **📂 Quick Navigation**
- **[docs/](docs/README.md)** - Main documentation hub with navigation
- **[docs/setup/](docs/setup/)** - Setup guides & configuration
- **[docs/architecture/](docs/architecture/)** - System design & technical details
- **[docs/roadmaps/](docs/roadmaps/)** - Strategic planning & feature roadmaps
- **[docs/analysis/](docs/analysis/)** - Performance analyses & optimization
- **[docs/bugs/](docs/bugs/)** - Critical incidents & fixes
- **[docs/deployments/](docs/deployments/)** - Deployment verification
- **[docs/cluster/](docs/cluster/)** - Distributed computing & EPYC cluster
- **[.github/copilot-instructions.md](.github/copilot-instructions.md)** - AI agent instructions (6,400+ lines)

### **🎯 Essential Reading**
| Document | Purpose |
|----------|---------|
| [docs/setup/SIGNAL_QUALITY_SETUP_GUIDE.md](docs/setup/SIGNAL_QUALITY_SETUP_GUIDE.md) | TradingView alert configuration |
| [docs/architecture/ADAPTIVE_LEVERAGE_SYSTEM.md](docs/architecture/ADAPTIVE_LEVERAGE_SYSTEM.md) | Risk management system |
| [docs/roadmaps/OPTIMIZATION_MASTER_ROADMAP.md](docs/roadmaps/OPTIMIZATION_MASTER_ROADMAP.md) | Strategic vision |
| [TRADING_GOALS.md](TRADING_GOALS.md) | Financial objectives ($106 → $100k+) |
| [.github/copilot-instructions.md](.github/copilot-instructions.md) | Complete system reference |

### **🔍 Finding Documentation**
**By Topic:**
- Performance data → `docs/analysis/`
- System design → `docs/architecture/`  
- Bug history → `docs/bugs/`
- Infrastructure → `docs/cluster/`
- Feature status → `docs/deployments/`
- Future plans → `docs/roadmaps/`
- Setup guides → `docs/setup/`

**By Status:**
- ✅ Completed → Check `docs/deployments/*_COMPLETE.md`
- 🔄 In Progress → Check `docs/roadmaps/` for phase status
- 📊 Analysis → Check `docs/analysis/` for data insights

---

## Resources

- **Drift Protocol**: https://drift.trade
- **Drift Docs**: https://docs.drift.trade
- **Drift SDK**: https://github.com/drift-labs/protocol-v2
- **Pyth Network**: https://pyth.network
- **Solana RPC**: https://helius.dev (recommended)
- **Next.js**: https://nextjs.org
- **Prisma ORM**: https://prisma.io

---

## Development Notes

### Key Patterns to Follow
1. **Singleton Services**: Always use getter functions, never instantiate directly
2. **Configuration**: Always use `getMergedConfig()`, never read env vars directly
3. **Database Errors**: Wrap in try/catch, don't fail trades on DB errors
4. **Price Calculations**: Direction matters - long vs short use opposite math
5. **Reduce-Only Orders**: All TP/SL orders MUST have `reduceOnly: true`
6. **Symbol Normalization**: Always use `normalizeTradingViewSymbol()`

### Adding New Features
1. **New Config**: Update DEFAULT_TRADING_CONFIG + getConfigFromEnv() + .env
2. **New Database Fields**: Update schema.prisma → migrate → regenerate → rebuild Docker
3. **New API Endpoint**: Follow auth pattern, use getMergedConfig(), init services
4. **Order Logic Changes**: Test with DRY_RUN=true first, use small positions

### Recent Bug Fixes (Oct 2024)
- ✅ TP2 runner calculation: Now uses remaining position after TP1
- ✅ Race condition: Exit orders now placed BEFORE monitoring starts
- ✅ Order cancellation: removeTrade() properly cancels orphaned orders
- ✅ Dynamic SL: Breakeven at +0.5%, profit lock at +1.2%

---

**Ready to trade autonomously? Start with `docs/SETUP.md` and test with DRY_RUN=true! 🚀**

*Dual-layer safety. Complete audit trail. Built for reliability.*