From 715fa8bd11ecfb4a3741b6cb8abf1c106e653906 Mon Sep 17 00:00:00 2001 From: mindesbunister Date: Tue, 28 Oct 2025 12:08:18 +0100 Subject: [PATCH] Update README with comprehensive architecture, dual-layer redundancy, and current features MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add dual-layer redundancy explanation (on-chain + Position Manager) - Document dual stop-loss system (soft TRIGGER_LIMIT + hard TRIGGER_MARKET) - Add complete trade flow documentation (signal → execution → monitoring → exit) - Update supported timeframes (5min and 15min) - Document database integration (PostgreSQL + Prisma) - Add configuration system details (three-layer merge) - Document recent bug fixes (TP2 calculation, race condition, order cancellation) - Add comprehensive file structure - Update API endpoints with all current routes - Add real-world trade examples with P&L calculations - Document singleton patterns and critical coding practices - Add troubleshooting section with common issues - Update testing commands and safety guidelines --- README.md | 821 ++++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 672 insertions(+), 149 deletions(-) diff --git a/README.md b/README.md index 1966ad2..f128c29 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Trading Bot v4 🚀 -**Fully Autonomous Trading Bot** for TradingView → n8n → Drift Protocol (Solana) +**Fully Autonomous Trading Bot** with Dual-Layer Redundancy for TradingView → n8n → Drift Protocol (Solana) ## Status @@ -9,23 +9,54 @@ | Phase 1 | ✅ **COMPLETE** | Trade execution from TradingView signals | | Phase 2 | ✅ **COMPLETE** | Real-time monitoring & automatic exits | | Phase 3 | ✅ **COMPLETE** | Web UI, settings management, Docker deployment | +| Phase 4 | ✅ **COMPLETE** | Database integration, analytics, race condition fixes | ## What It Does -1. **Receives signals** from TradingView (5-minute chart) -2. **Executes trades** on Drift Protocol (Solana DEX) -3. **Monitors prices** in real-time via Pyth Network -4. **Closes positions** automatically at TP1/TP2/SL -5. **Adjusts stops** dynamically (breakeven, profit lock) -6. **Provides web UI** for configuration and monitoring +1. **Receives signals** from TradingView (5-minute OR 15-minute charts) +2. **Executes trades** on Drift Protocol (Solana DEX) with dual stop-loss system +3. **Monitors positions** every 2 seconds via Pyth Network WebSocket + HTTP fallback +4. **Closes positions** automatically at TP1 (partial) / TP2 (80%) / SL (100%) +5. **Adjusts stops** dynamically (breakeven at +0.5%, profit lock at +1.2%) +6. **Tracks everything** in PostgreSQL (trades, prices, P&L, win rate) +7. **Provides web UI** for configuration, monitoring, and analytics -**100% autonomous. No manual intervention required!** +**100% autonomous. Dual-layer safety. No manual intervention required!** + +## Architecture: Dual-Layer Redundancy + +**Key Design Principle:** Every trade has **TWO independent exit mechanisms**: + +1. **On-Chain Orders (Drift Protocol)** - Primary layer + - TP1/TP2 as LIMIT orders + - Soft SL as TRIGGER_LIMIT (-1.5%, avoids wicks) + - Hard SL as TRIGGER_MARKET (-2.5%, guarantees exit) + +2. **Software Monitoring (Position Manager)** - Backup layer + - Checks prices every 2 seconds + - Closes via MARKET orders if on-chain orders fail + - Dynamic SL adjustments + - Emergency stop functionality + +**Why?** If Drift orders don't fill (low liquidity, network issues), Position Manager acts as backup. Both write to the same PostgreSQL database for complete trade history. ## Quick Start (Docker) -### 1. Deploy with Docker Compose +### 1. Prerequisites +- Docker & Docker Compose installed +- Solana wallet with Drift Protocol account +- Helius RPC API key (mainnet) +- TradingView alerts → n8n webhook setup + +### 2. Deploy with Docker Compose ```bash -# Build and start +# Clone and setup +cd /home/icke/traderv4 + +# Configure .env file (copy from .env.example) +nano .env + +# Build and start all services docker compose up -d # View logs @@ -35,20 +66,27 @@ docker compose logs -f trading-bot docker compose down ``` -### 2. Access Web Interface +### 3. Access Web Interface - **Settings UI:** `http://YOUR_HOST:3001/settings` +- **Analytics:** `http://YOUR_HOST:3001/analytics` - **API Endpoints:** `http://YOUR_HOST:3001/api/` -### 3. Configure Settings +### 4. Configure Settings Open `http://YOUR_HOST:3001/settings` in your browser to: -- Adjust position size and leverage -- Set stop-loss and take-profit levels -- Configure dynamic stop-loss triggers -- Set daily loss limits -- Toggle DRY_RUN mode +- Adjust position size ($10-$10,000) and leverage (1x-20x) +- Set stop-loss (-1.5% soft, -2.5% hard) and take-profit levels +- Configure dynamic stop-loss (breakeven +0.5%, profit lock +1.2%) +- Set daily loss limits and max trades per hour +- Toggle DRY_RUN mode for paper trading -### 4. Setup n8n Workflow -Import `n8n-complete-workflow.json` into your n8n instance and configure TradingView alerts. +After saving, click **"Restart Bot"** to apply changes. + +### 5. Setup n8n Workflow +Import `workflows/trading/Money_Machine.json` into your n8n instance: +- Configure TradingView webhook URL +- Set timeframe filters (5min and/or 15min) +- Add API authentication header +- Test with manual execution ## Alternative: Manual Setup @@ -81,31 +119,159 @@ npm start --- -## Features +## Core Features -### Phase 1: Trade Execution ✅ -- Drift Protocol integration -- Market order execution -- TradingView signal normalization -- n8n webhook endpoint -- Risk validation API +### Dual Stop-Loss System +- **Soft Stop** (TRIGGER_LIMIT): -1.5% from entry, avoids wick-outs +- **Hard Stop** (TRIGGER_MARKET): -2.5% from entry, guarantees exit +- Both placed on-chain as reduce-only orders +- Position Manager monitors as backup (closes via MARKET if needed) -### Phase 2: Autonomous Trading ✅ -- **Pyth price monitoring** (WebSocket + polling) -- **Position manager** (tracks all trades) -- **Automatic exits** (TP1/TP2/SL/Emergency) -- **Dynamic SL** (breakeven + profit lock) -- **Multi-position** support -- **Real-time P&L** tracking +### Dynamic Stop-Loss Adjustments +- **Breakeven**: Moves SL to +0.01% when price hits +0.5% +- **Profit Lock**: Locks in profit when price hits +1.2% +- **Reduces risk** while letting winners run -### Phase 3: Production Ready ✅ -- **Web UI** for settings management -- **Docker deployment** with multi-stage builds -- **REST API** for all operations -- **Risk calculator** with live preview -- **Settings persistence** to .env file -- **PostgreSQL** integration ready +### Take-Profit Strategy +- **TP1** (default +0.7%): Closes 50% of position, moves SL to breakeven +- **TP2** (default +1.5%): Closes 80% of remaining position +- **Runner**: 20% remains open if takeProfit2SizePercent < 100% +### Position Manager (Singleton) +- Monitors all positions every 2 seconds +- Singleton pattern: Use `getPositionManager()` - never instantiate directly +- Tracks price updates in database +- Closes positions when targets hit +- Cancels orphaned orders automatically +- Acts as backup if on-chain orders don't fill + +### Database Integration (PostgreSQL + Prisma) +**Models:** +- `Trade`: Complete trade history with entry/exit data +- `PriceUpdate`: Price movements every 2 seconds during monitoring +- `SystemEvent`: Errors, restarts, important events +- `DailyStats`: Win rate, profit factor, avg win/loss + +**Analytics:** +- Real-time P&L tracking +- Win rate and profit factor +- Best/worst trades +- Drawdown monitoring + +### Configuration System (Three-Layer Merge) +1. **Defaults** (`config/trading.ts` - DEFAULT_TRADING_CONFIG) +2. **Environment** (`.env` file via `getConfigFromEnv()`) +3. **Runtime** (API overrides via `getMergedConfig(overrides)`) + +Always use `getMergedConfig()` in business logic - never read env vars directly. + +### Safety Features +- **Reduce-only orders**: All TP/SL orders can only close, not open positions +- **Account health checks**: Validates margin before every trade +- **Risk validation**: `/api/trading/check-risk` endpoint +- **Daily loss limits**: Stops trading after max loss reached +- **Cooldown periods**: Prevents over-trading +- **DRY_RUN mode**: Paper trading for testing + +--- + +## How It Works: Complete Trade Flow + +### 1. Signal Reception (TradingView → n8n) +``` +TradingView Alert: "LONG SOLUSDT .P 15" + ↓ +n8n Webhook receives signal + ↓ +Parse Signal node extracts: + - Symbol: SOLUSDT → SOL-PERP (normalized) + - Direction: long + - Timeframe: 15 (from .P 15) + ↓ +Timeframe Filter: Allow 5 or 15 minutes only + ↓ +Check Risk: Validate position limits, daily loss, etc. +``` + +### 2. Trade Execution (n8n → Next.js API → Drift) +``` +POST /api/trading/execute + ↓ +getMergedConfig() - Get current settings + ↓ +initializeDriftService() - Connect to Drift SDK + ↓ +Check account health (margin requirements) + ↓ +openPosition() - Execute MARKET order + ↓ +Calculate dual stop prices (soft -1.5%, hard -2.5%) + ↓ +placeExitOrders() - Place on-chain TP/SL orders + ├─ TP1: LIMIT reduce-only at +0.7% + ├─ TP2: LIMIT reduce-only at +1.5% + ├─ Soft SL: TRIGGER_LIMIT reduce-only at -1.5% + └─ Hard SL: TRIGGER_MARKET reduce-only at -2.5% + ↓ +createTrade() - Save to PostgreSQL database + ↓ +positionManager.addTrade() - Start monitoring loop +``` + +### 3. Position Monitoring (Every 2 Seconds) +``` +Position Manager Loop: + ↓ +getPythPriceMonitor().getLatestPrice() + ↓ +Calculate current P&L and percentage gain/loss + ↓ +Check TP1 hit? → closePosition(75%) + move SL to breakeven + ↓ +Check TP2 hit? → closePosition(80% of remaining) + ↓ +Check SL hit? → closePosition(100%) + ↓ +Check dynamic adjustments: + ├─ Price > +0.5%? → Move SL to breakeven (+0.01%) + └─ Price > +1.2%? → Lock profit (move SL to +X%) + ↓ +addPriceUpdate() - Save price to database + ↓ +Repeat every 2 seconds until position closed +``` + +### 4. Position Exit (Automatic) +``` +Exit Triggered (TP/SL hit): + ↓ +If closed by on-chain order: + ├─ Position Manager detects position.size === 0 + ├─ Determines exit reason (TP1/TP2/SL from price) + ├─ updateTradeExit() - Save exit data to database + └─ removeTrade() - Stop monitoring + cancel orphaned orders + ↓ +If closed by Position Manager: + ├─ closePosition() - Execute MARKET order + ├─ cancelAllOrders() - Cancel remaining on-chain orders + ├─ updateTradeExit() - Save exit data to database + └─ removeTrade() - Stop monitoring +``` + +### 5. Order Cleanup (Automatic) +``` +When position closes (100%): + ↓ +cancelAllOrders(symbol) - Query all open orders + ↓ +Filter by marketIndex and status === 0 (Open) + ↓ +driftClient.cancelOrders() - Cancel on Drift + ↓ +Logs: "Cancelled X orders" or "Cancelled X orphaned orders" +``` + +**Result:** Clean exit with no orphaned orders, complete trade history in database, ready for next signal. --- ## Web Interface @@ -118,13 +284,13 @@ Beautiful web interface for managing all trading parameters: - Set leverage (1x-20x) **Risk Management:** -- Stop-loss percentage +- Stop-loss percentage (soft -1.5%, hard -2.5%) - Take-profit 1 & 2 levels - Emergency stop level **Dynamic Stop-Loss:** -- Breakeven trigger -- Profit lock trigger and amount +- Breakeven trigger (+0.5%) +- Profit lock trigger and amount (+1.2%) **Safety Limits:** - Max daily loss @@ -140,30 +306,48 @@ Beautiful web interface for managing all trading parameters: - TP1 and TP2 gains - Risk/Reward ratio +### Analytics Page (`/analytics`) +Real-time trading performance dashboard: +- Current open positions with live P&L +- Trade history with detailed entry/exit data +- Win rate and profit factor +- Total P&L (daily, weekly, monthly) +- Best and worst trades +- Drawdown tracking + ### API Endpoints -All endpoints require `Authorization: Bearer YOUR_API_SECRET_KEY` +All endpoints require `Authorization: Bearer YOUR_API_SECRET_KEY` (except `/api/trading/test`) **Trade Execution:** ```bash -# Execute a trade +# Execute a trade (production - from n8n) POST /api/trading/execute { "symbol": "SOL-PERP", "direction": "long", - "timeframe": "5", + "timeframe": "15", "signalStrength": "strong" } -# Close a position +# Test trade (no auth required - from settings UI) +POST /api/trading/test +{ + "symbol": "SOL-PERP", + "direction": "long", + "timeframe": "15" +} + +# Close a position (partial or full) POST /api/trading/close { "symbol": "SOL-PERP", - "percentToClose": 100 + "percentToClose": 100 // or 50, 75, etc. } # Get active positions GET /api/trading/positions +# Returns: { positions: [...], monitoring: [...] } # Validate trade (risk check) POST /api/trading/check-risk @@ -178,69 +362,118 @@ POST /api/trading/check-risk # Get current settings GET /api/settings -# Update settings +# Update settings (writes to .env file) POST /api/settings { "MAX_POSITION_SIZE_USD": 100, "LEVERAGE": 10, "STOP_LOSS_PERCENT": -1.5, - ... + "SOFT_STOP_LOSS_PERCENT": -1.5, + "HARD_STOP_LOSS_PERCENT": -2.5, + "TAKE_PROFIT_1_PERCENT": 0.7, + "TAKE_PROFIT_2_PERCENT": 1.5, + "TAKE_PROFIT_2_SIZE_PERCENT": 80, + "BREAKEVEN_TRIGGER_PERCENT": 0.5, + "PROFIT_LOCK_TRIGGER_PERCENT": 1.2, + "DRY_RUN": false } # Restart bot container (apply settings) POST /api/restart +# Creates /tmp/trading-bot-restart.flag +# watch-restart.sh detects flag and runs: docker restart trading-bot-v4 ``` -**Notes:** -- Settings changes require container restart to take effect -- Use the web UI's "Restart Bot" button or call `/api/restart` -- Restart watcher must be running (see setup below) +**Analytics:** +```bash +# Get trade statistics +GET /api/analytics/stats +# Returns: { winRate, profitFactor, totalTrades, totalPnL, ... } -# Update settings -POST /api/settings -{ - "MAX_POSITION_SIZE_USD": 100, - "LEVERAGE": 5, - "STOP_LOSS_PERCENT": -1.5, - ... -} +# Get recent trades +GET /api/analytics/positions +# Returns: { openPositions: [...], recentTrades: [...] } ``` +**Symbol Normalization:** +- TradingView sends: `SOLUSDT`, `BTCUSDT`, `ETHUSDT` +- Bot converts to: `SOL-PERP`, `BTC-PERP`, `ETH-PERP` +- Always use Drift format in API calls + +--- + --- ## Docker Deployment ### Architecture -- **Multi-stage build** for optimized image size -- **Next.js standalone** output for production -- **PostgreSQL** for trade history +- **Multi-stage build**: deps → builder → runner (Node 20 Alpine) +- **Next.js standalone** output for production (~400MB image) +- **PostgreSQL 16-alpine** for trade history - **Isolated network** (172.28.0.0/16) - **Health monitoring** and logging ### Container Details -- **Port:** 3001 (external) → 3000 (internal) -- **Image:** Node 20 Alpine -- **Size:** ~400MB (optimized) -- **Restart:** unless-stopped +- **trading-bot-v4**: Main application + - Port: 3001 (external) → 3000 (internal) + - Restart: unless-stopped + - Volumes: .env file mounted + +- **trading-bot-postgres**: Database + - Port: 5432 (internal only) + - Persistent volume: trading-bot-postgres-data + - Auto-backup recommended + +### Critical Build Steps +1. Install deps: `npm install --production` +2. Copy source and generate Prisma client: `npx prisma generate` +3. Build Next.js: `npm run build` (standalone mode) +4. Runner stage: Copy standalone + static + node_modules + Prisma client + +**Why Prisma generate before build?** The Trade type from Prisma must exist before Next.js compiles TypeScript. ### Commands ```bash -# Build image +# Build and deploy docker compose build trading-bot - -# Start services docker compose up -d -# View logs +# View logs (real-time) docker compose logs -f trading-bot +# View logs (last 100 lines) +docker compose logs --tail=100 trading-bot + # Restart after config changes docker compose restart trading-bot +# Rebuild and restart (force recreate) +docker compose up -d --force-recreate trading-bot + # Stop everything docker compose down + +# Stop and remove volumes (WARNING: deletes database) +docker compose down -v ``` +### Database Operations +```bash +# Connect to database +docker exec -it trading-bot-postgres psql -U postgres -d trading_bot_v4 + +# Run migrations from host +DATABASE_URL="postgresql://postgres:postgres@localhost:5432/trading_bot_v4" npx prisma migrate dev + +# Generate Prisma client +npx prisma generate + +# View tables +docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\dt" +``` + +**DATABASE_URL caveat:** Use `trading-bot-postgres` (container name) in .env for runtime, but `localhost:5432` for Prisma CLI migrations from host. + ### Restart Watcher (Required for Web UI Restart Button) The restart watcher monitors for restart requests from the web UI: @@ -262,15 +495,54 @@ sudo systemctl status trading-bot-restart-watcher The watcher enables the "Restart Bot" button in the web UI to automatically restart the container when settings are changed. ### Environment Variables -All settings are configured via `.env` file: -- Drift wallet credentials -- Solana RPC endpoint (Helius recommended) -- Trading parameters (size, leverage, SL, TP) -- Risk limits and safety controls -- API authentication key +All settings configured via `.env` file: + +**Required:** +- `DRIFT_WALLET_PRIVATE_KEY`: Solana wallet (JSON array or base58 string) +- `SOLANA_RPC_URL`: Helius RPC endpoint (mainnet recommended) +- `API_SECRET_KEY`: Random secret for API authentication +- `DATABASE_URL`: PostgreSQL connection string + +**Trading Parameters:** +- `MAX_POSITION_SIZE_USD`: Position size in USD +- `LEVERAGE`: Leverage multiplier (1-20x) +- `STOP_LOSS_PERCENT`: Soft stop percentage (e.g., -1.5) +- `HARD_STOP_LOSS_PERCENT`: Hard stop percentage (e.g., -2.5) +- `TAKE_PROFIT_1_PERCENT`: TP1 target (e.g., 0.7) +- `TAKE_PROFIT_2_PERCENT`: TP2 target (e.g., 1.5) +- `TAKE_PROFIT_2_SIZE_PERCENT`: How much to close at TP2 (e.g., 80) + +**Dynamic Stop-Loss:** +- `BREAKEVEN_TRIGGER_PERCENT`: Move to breakeven at (e.g., 0.5) +- `PROFIT_LOCK_TRIGGER_PERCENT`: Lock profit at (e.g., 1.2) +- `PROFIT_LOCK_AMOUNT_PERCENT`: Profit to lock (e.g., 0.5) + +**Safety:** +- `MAX_DAILY_LOSS`: Max loss per day in USD +- `MAX_TRADES_PER_HOUR`: Rate limiting +- `TRADE_COOLDOWN_MINUTES`: Cooldown between trades +- `DRY_RUN`: Enable paper trading (true/false) +- `USE_DUAL_STOPS`: Enable dual stop system (true/false) Changes to `.env` require container restart to take effect. +### Singleton Services (Critical Pattern) +**Never create multiple instances** - always use getter functions: + +```typescript +// Drift Client +const driftService = await initializeDriftService() // NOT: new DriftService() +const driftService = getDriftService() // After init + +// Position Manager +const positionManager = getPositionManager() // NOT: new PositionManager() + +// Database +const prisma = getPrismaClient() // NOT: new PrismaClient() +``` + +Creating multiple instances causes connection issues and state inconsistencies. + --- ## File Structure @@ -280,66 +552,93 @@ traderv4/ ├── README.md ← You are here ├── docker-compose.yml ← Docker orchestration ├── Dockerfile ← Multi-stage build -├── .env ← Configuration (template) -├── package.json ← Dependencies +├── .env ← Configuration (create from .env.example) +├── package.json ← Dependencies (Next.js 15, Drift SDK, Prisma) +├── next.config.js ← Next.js config (standalone output) +├── tsconfig.json ← TypeScript config │ -├── docs/ ← Documentation -│ ├── setup/ -│ │ ├── DOCKER.md ← Docker deployment guide -│ │ ├── SETUP.md ← Setup instructions -│ │ ├── SETTINGS_SETUP.md ← Settings guide -│ │ ├── TELEGRAM_BOT_README.md ← Telegram bot guide -│ │ ├── N8N_WORKFLOW_SETUP.md ← n8n workflow setup -│ │ └── N8N_DATABASE_SETUP.md ← n8n database setup -│ ├── guides/ -│ │ ├── TESTING.md ← Testing guide -│ │ ├── N8N_WORKFLOW_GUIDE.md ← n8n usage guide -│ │ └── WORKFLOW_VERIFICATION.md ← Workflow testing -│ └── history/ -│ ├── PHASE_1_COMPLETE.md ← Phase 1 completion -│ ├── PHASE_2_COMPLETE.md ← Phase 2 completion -│ └── ... ← Development history -│ -├── workflows/ ← n8n workflow files -│ ├── trading/ ← Main trading workflows -│ ├── analytics/ ← Analytics workflows -│ └── telegram/ ← Telegram bot workflows -│ -├── scripts/ ← Utility scripts -│ ├── docker/ ← Docker helpers -│ ├── setup/ ← Setup scripts -│ └── testing/ ← Test scripts -│ -├── tests/ ← Test files -│ -├── app/ -│ ├── layout.tsx ← Root layout -│ ├── globals.css ← Tailwind styles +├── app/ ← Next.js 15 App Router +│ ├── layout.tsx ← Root layout with Tailwind +│ ├── page.tsx ← Home page +│ ├── globals.css ← Global styles │ ├── settings/ │ │ └── page.tsx ← Settings UI +│ ├── analytics/ +│ │ └── page.tsx ← Analytics dashboard │ └── api/ │ ├── settings/ -│ │ └── route.ts ← Settings API +│ │ └── route.ts ← GET/POST settings, writes to .env +│ ├── restart/ +│ │ └── route.ts ← Creates restart flag file +│ ├── analytics/ +│ │ ├── stats/route.ts ← Trade statistics +│ │ └── positions/route.ts ← Open/recent positions │ └── trading/ -│ ├── execute/route.ts ← Execute trades +│ ├── execute/route.ts ← Main execution (production) +│ ├── test/route.ts ← Test execution (UI) │ ├── close/route.ts ← Close positions │ ├── positions/route.ts ← Query positions -│ └── check-risk/route.ts ← Risk validation +│ ├── check-risk/route.ts ← Risk validation +│ └── remove-position/route.ts ← Remove from monitoring │ -├── lib/ +├── lib/ ← Business logic │ ├── drift/ -│ │ ├── client.ts ← Drift SDK wrapper -│ │ └── orders.ts ← Order execution +│ │ ├── client.ts ← Drift SDK wrapper (singleton) +│ │ └── orders.ts ← Order execution & cancellation │ ├── pyth/ -│ │ └── price-monitor.ts ← Real-time prices -│ └── trading/ -│ └── position-manager.ts ← Auto-exit logic +│ │ └── price-monitor.ts ← WebSocket + HTTP fallback +│ ├── trading/ +│ │ └── position-manager.ts ← Monitoring loop (singleton) +│ ├── database/ +│ │ ├── trades.ts ← Trade CRUD operations +│ │ └── views.ts ← Analytics queries +│ └── notifications/ +│ └── telegram.ts ← Telegram alerts (optional) │ ├── config/ -│ └── trading.ts ← Market configurations +│ └── trading.ts ← Market configs & defaults │ ├── prisma/ -│ └── schema.prisma ← Database schema +│ ├── schema.prisma ← Database models +│ └── migrations/ ← Migration history +│ ├── 20251026200052_init/ +│ └── 20251027080947_add_test_trade_flag/ +│ +├── workflows/ ← n8n workflow JSON files +│ ├── trading/ +│ │ └── Money_Machine.json ← Main trading workflow +│ ├── analytics/ +│ │ ├── n8n-daily-report.json +│ │ ├── n8n-database-analytics.json +│ │ └── n8n-stop-loss-analysis.json +│ └── telegram/ +│ └── telegram-webhook-FINAL.json +│ +├── scripts/ ← Utility scripts +│ ├── docker-build.sh +│ ├── docker-start.sh +│ ├── docker-stop.sh +│ ├── docker-logs.sh +│ ├── watch-restart.sh ← Restart watcher daemon +│ ├── send_trade.sh ← Test trade execution +│ └── test-exit-orders.sh ← Test exit order placement +│ +├── tests/ ← Test files +│ ├── test-drift-v4.ts +│ ├── test-full-flow.ts +│ ├── test-position-manager.ts +│ └── test-price-monitor.ts +│ +├── docs/ ← Documentation +│ ├── SETUP.md ← Detailed setup guide +│ ├── DOCKER.md ← Docker deployment +│ ├── TESTING.md ← Testing guide +│ ├── TELEGRAM_BOT_README.md ← Telegram setup +│ ├── N8N_WORKFLOW_SETUP.md ← n8n configuration +│ ├── PHASE_2_COMPLETE.md ← Phase 2 features +│ └── QUICKREF_PHASE2.md ← Quick reference +│ +└── logs/ ← Log files (created at runtime) ``` --- @@ -359,42 +658,239 @@ traderv4/ --- -## Trade Example +## Trade Example (Real-World Flow) -### Entry Signal +### Entry Signal from TradingView ``` -TradingView: LONG SOL @ $140.00 -Position: $1,000 (10x = $10,000) -SL: $137.90 (-1.5%) -TP1: $140.98 (+0.7%) -TP2: $142.10 (+1.5%) +Alert Message: "LONG SOLUSDT .P 15" + ↓ +n8n receives webhook + ↓ +Parse: symbol=SOL-PERP, direction=long, timeframe=15 + ↓ +Timeframe check: 15 minutes ✅ (allowed) + ↓ +Risk check: Daily loss OK, no existing position ✅ + ↓ +Execute trade via API ``` -### TP1 Hit +### Position Opened ``` -✅ Price reaches $140.98 -→ Auto-close 50% (+$70) -→ Move SL to $140.21 (breakeven) -→ Trade is now RISK-FREE +Symbol: SOL-PERP +Direction: LONG +Entry: $200.00 +Position Size: $100 (10x leverage = $1,000 notional) + +On-Chain Orders Placed: +├─ TP1: LIMIT at $201.40 (+0.7%) - Close 50% +├─ TP2: LIMIT at $203.00 (+1.5%) - Close 80% of remaining +├─ Soft SL: TRIGGER_LIMIT at $197.00 (-1.5%) +└─ Hard SL: TRIGGER_MARKET at $195.00 (-2.5%) + +Position Manager: ✅ Monitoring started (every 2s) +Database: ✅ Trade #601 saved ``` -### TP2 Hit +### TP1 Hit (First Target) ``` -✅ Price reaches $142.10 -→ Auto-close remaining 50% (+$150) -→ Total P&L: +$220 (+22% account) -→ Trade complete! +Price reaches $201.40 + ↓ +On-chain TP1 order fills → 50% closed + ↓ +Position Manager detects partial close: + ├─ Profit: +$7.00 (+7% account) + ├─ Remaining: 50% ($500 notional) + ├─ Move SL to breakeven: $200.02 (+0.01%) + └─ Trade is now RISK-FREE ✅ + ↓ +Database updated: exitReason=TP1_PARTIAL +``` + +### Price Continues Higher +``` +Price reaches $202.44 (+1.22%) + ↓ +Position Manager dynamic adjustment: + ├─ Trigger: +1.2% profit lock activated + ├─ Move SL to: $201.00 (+0.5% profit locked) + └─ Letting winner run with locked profit ✅ +``` + +### TP2 Hit (Second Target) +``` +Price reaches $203.00 + ↓ +On-chain TP2 order fills → 80% of remaining closed (40% of original) + ↓ +Position Manager detects: + ├─ Profit from TP2: +$6.00 + ├─ Remaining: 10% runner ($100 notional) + └─ Runner continues with locked profit SL + ↓ +Database updated: exitReason=TP2_PARTIAL +``` + +### Final Exit (Runner) +``` +Option 1: Runner hits new high → Manual/trailing stop +Option 2: Price pulls back → Locked profit SL hits at $201.00 +Option 3: Emergency stop or manual close + +Total P&L: +├─ TP1: +$7.00 (50% at +0.7%) +├─ TP2: +$6.00 (40% at +1.5%) +└─ Runner: +$3.00 (10% at +3.0%, closed at pullback) +═══════════════════ +Total: +$16.00 (+16% account growth) + +Database: ✅ Trade #601 complete +``` + +### If Stop-Loss Hit Instead +``` +Price drops to $197.00 + ↓ +Soft SL (TRIGGER_LIMIT) activates: + ├─ Triggers at $197.00 + ├─ Limit order at $196.95 (avoid wick) + └─ If fills → Position closed ✅ + ↓ +If soft SL doesn't fill (low liquidity): + ├─ Price drops to $195.00 + ├─ Hard SL (TRIGGER_MARKET) activates + └─ Market order guarantees exit ✅ + ↓ +Position Manager backup: + ├─ Detects position still open at -2.5% + └─ Closes via MARKET order if needed + ↓ +Loss: -$25.00 (-2.5% account) +Database: ✅ exitReason=STOP_LOSS +``` + +**Key Points:** +- Dual stop system ensures exit even in volatile markets +- Position Manager acts as backup to on-chain orders +- Dynamic SL makes trades risk-free after +0.5% +- Profit lock captures gains before reversals +- Complete audit trail in database + +--- + +## Common Issues & Solutions + +### "Drift service not initialized" +**Cause:** Drift client not connected before trade execution +**Solution:** Ensure `initializeDriftService()` called before operations + +### "Position not found in monitoring" +**Cause:** Race condition - orders placed after Position Manager started +**Solution:** ✅ FIXED - Orders now placed before monitoring starts + +### "Orphaned orders after exit" +**Cause:** Old bug - Position Manager detected closure before orders existed +**Solution:** ✅ FIXED - Order placement sequencing corrected +**Cleanup:** Cancel manually on Drift UI or wait for next trade's auto-cleanup + +### "TP2 closes entire position instead of 80%" +**Cause:** Old bug - TP2 calculated from original position instead of remaining +**Solution:** ✅ FIXED - TP2 now calculates from remaining after TP1 + +### "Database save failed but trade executed" +**Cause:** PostgreSQL connection issue or schema mismatch +**Solution:** Trade still executes successfully, check database logs and fix connection + +### "Prisma Client not generated" (Docker build) +**Cause:** `npx prisma generate` not run before `npm run build` +**Solution:** Ensure Dockerfile runs Prisma generate in builder stage + +### "Wrong DATABASE_URL" (localhost vs container) +**Container runtime:** Use `trading-bot-postgres:5432` in .env +**Prisma CLI (host):** Use `localhost:5432` for migrations +**Solution:** Maintain two DATABASE_URL values for different contexts + +### Container won't restart after settings change +**Cause:** Restart watcher not running +**Solution:** +```bash +sudo systemctl status trading-bot-restart-watcher +sudo systemctl start trading-bot-restart-watcher ``` --- ## Safety Guidelines -1. **Start Small**: Use $10-50 positions first -2. **Test Thoroughly**: Run all test scripts -3. **Monitor Closely**: Watch first 10 auto-exits -4. **Verify Fills**: Check Drift UI after exits -5. **Scale Gradually**: Increase size weekly +1. **Start Small**: Use $10-50 positions for first 20 trades +2. **Test Thoroughly**: + - Run with DRY_RUN=true first + - Execute test trades from settings UI + - Verify all exits work correctly +3. **Monitor Closely**: Watch first 10 auto-exits in real-time +4. **Verify Database**: Check that all trades are being saved correctly +5. **Check Drift UI**: Confirm positions and orders match expectations +6. **Scale Gradually**: Increase position size 2x per week maximum +7. **Daily Review**: Check analytics page every day +8. **Backup Database**: Export PostgreSQL data weekly + +**Risk Warning:** Cryptocurrency trading involves substantial risk. This bot executes trades automatically. Start small and never risk more than you can afford to lose. + +--- + +## Testing Commands + +```bash +# Local development +npm run dev + +# Build production +npm run build && npm start + +# Docker build and restart +docker compose build trading-bot +docker compose up -d --force-recreate trading-bot +docker logs -f trading-bot-v4 + +# Test trade from UI +# Go to http://localhost:3001/settings +# Click "Test LONG" or "Test SHORT" + +# Test trade from terminal +./send_trade.sh LONG SOL-PERP 15 + +# Check current positions +curl -s -X GET http://localhost:3001/api/trading/positions \ + -H "Authorization: Bearer YOUR_API_SECRET_KEY" | jq + +# Test exit order placement +./test-exit-orders.sh + +# Database operations +npx prisma generate +DATABASE_URL="postgresql://postgres:postgres@localhost:5432/trading_bot_v4" npx prisma migrate dev +docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "SELECT * FROM \"Trade\" ORDER BY \"createdAt\" DESC LIMIT 5;" + +# Check logs +docker compose logs --tail=100 trading-bot +docker compose logs -f trading-bot | grep "Position Manager" +``` + +--- + +## Documentation + +| Document | Purpose | +|----------|---------| +| `README.md` | Complete system overview (this file) | +| `docs/SETUP.md` | Detailed setup instructions | +| `docs/DOCKER.md` | Docker deployment guide | +| `docs/TESTING.md` | Comprehensive testing guide | +| `docs/TELEGRAM_BOT_README.md` | Telegram bot setup | +| `docs/N8N_WORKFLOW_SETUP.md` | n8n workflow configuration | +| `docs/PHASE_2_COMPLETE.md` | Phase 2 features and architecture | +| `docs/QUICKREF_PHASE2.md` | Quick reference guide | +| `.github/copilot-instructions.md` | AI agent instructions (architecture) | --- @@ -402,11 +898,38 @@ TP2: $142.10 (+1.5%) - **Drift Protocol**: https://drift.trade - **Drift Docs**: https://docs.drift.trade +- **Drift SDK**: https://github.com/drift-labs/protocol-v2 - **Pyth Network**: https://pyth.network -- **Solana RPC**: https://helius.dev +- **Solana RPC**: https://helius.dev (recommended) +- **Next.js**: https://nextjs.org +- **Prisma ORM**: https://prisma.io --- -**Ready to trade autonomously? Read `QUICKREF_PHASE2.md` to get started! 🚀** +## Development Notes -*Start small, monitor closely, scale gradually!* +### Key Patterns to Follow +1. **Singleton Services**: Always use getter functions, never instantiate directly +2. **Configuration**: Always use `getMergedConfig()`, never read env vars directly +3. **Database Errors**: Wrap in try/catch, don't fail trades on DB errors +4. **Price Calculations**: Direction matters - long vs short use opposite math +5. **Reduce-Only Orders**: All TP/SL orders MUST have `reduceOnly: true` +6. **Symbol Normalization**: Always use `normalizeTradingViewSymbol()` + +### Adding New Features +1. **New Config**: Update DEFAULT_TRADING_CONFIG + getConfigFromEnv() + .env +2. **New Database Fields**: Update schema.prisma → migrate → regenerate → rebuild Docker +3. **New API Endpoint**: Follow auth pattern, use getMergedConfig(), init services +4. **Order Logic Changes**: Test with DRY_RUN=true first, use small positions + +### Recent Bug Fixes (Oct 2024) +- ✅ TP2 runner calculation: Now uses remaining position after TP1 +- ✅ Race condition: Exit orders now placed BEFORE monitoring starts +- ✅ Order cancellation: removeTrade() properly cancels orphaned orders +- ✅ Dynamic SL: Breakeven at +0.5%, profit lock at +1.2% + +--- + +**Ready to trade autonomously? Start with `docs/SETUP.md` and test with DRY_RUN=true! 🚀** + +*Dual-layer safety. Complete audit trail. Built for reliability.*