chore: Organize workspace structure - move docs, workflows, scripts to subdirectories
Organization: - Created docs/ with setup/, guides/, history/ subdirectories - Created workflows/ with trading/, analytics/, telegram/, archive/ subdirectories - Created scripts/ with docker/, setup/, testing/ subdirectories - Created tests/ for TypeScript test files - Created archive/ for unused reference files Moved files: - 17 documentation files → docs/ - 16 workflow JSON files → workflows/ - 10 shell scripts → scripts/ - 4 test files → tests/ - 5 unused files → archive/ Updated: - README.md with new file structure and documentation paths Deleted: - data/ (empty directory) - screenshots/ (empty directory) Critical files remain in root: - telegram_command_bot.py (active bot - used by Dockerfile) - watch-restart.sh (systemd service dependency) - All Dockerfiles and docker-compose files - All environment files Validation: Containers running (trading-bot-v4, telegram-trade-bot, postgres) API responding (positions endpoint tested) Telegram bot functional (/status command tested) All critical files present in root No code changes - purely organizational. System continues running without interruption. Recovery: git revert HEAD or git reset --hard cleanup-before
This commit is contained in:
mindesbunister
557
docs/setup/DOCKER.md
Normal file
557
docs/setup/DOCKER.md
Normal file
@@ -0,0 +1,557 @@
|
||||
# Trading Bot v4 - Docker Deployment Guide
|
||||
|
||||
Complete guide for containerized deployment with Docker and Docker Compose.
|
||||
|
||||
---
|
||||
|
||||
## 📋 Table of Contents
|
||||
|
||||
1. [Quick Start](#quick-start)
|
||||
2. [Production Deployment](#production-deployment)
|
||||
3. [Development Setup](#development-setup)
|
||||
4. [Configuration](#configuration)
|
||||
5. [Docker Commands](#docker-commands)
|
||||
6. [Troubleshooting](#troubleshooting)
|
||||
7. [Best Practices](#best-practices)
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
```bash
|
||||
# Install Docker
|
||||
curl -fsSL https://get.docker.com -o get-docker.sh
|
||||
sudo sh get-docker.sh
|
||||
|
||||
# Install Docker Compose
|
||||
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
|
||||
sudo chmod +x /usr/local/bin/docker-compose
|
||||
|
||||
# Verify installation
|
||||
docker --version
|
||||
docker-compose --version
|
||||
```
|
||||
|
||||
### Minimal Setup (Production)
|
||||
|
||||
```bash
|
||||
# 1. Navigate to v4 directory
|
||||
cd v4
|
||||
|
||||
# 2. Create .env file from template
|
||||
cp .env.example .env
|
||||
|
||||
# 3. Edit .env with your credentials
|
||||
nano .env # or vim, code, etc.
|
||||
|
||||
# 4. Build and start
|
||||
docker-compose up -d
|
||||
|
||||
# 5. View logs
|
||||
docker-compose logs -f trading-bot
|
||||
|
||||
# 6. Check status
|
||||
docker-compose ps
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🏭 Production Deployment
|
||||
|
||||
### Step 1: Prepare Environment
|
||||
|
||||
```bash
|
||||
cd v4
|
||||
|
||||
# Create production .env file
|
||||
cp .env.example .env
|
||||
|
||||
# Edit required fields (minimum required)
|
||||
DRIFT_WALLET_PRIVATE_KEY=your_base58_private_key
|
||||
SOLANA_RPC_URL=https://mainnet.helius-rpc.com/?api-key=YOUR_KEY
|
||||
API_SECRET_KEY=$(openssl rand -hex 32)
|
||||
PYTH_HERMES_URL=https://hermes.pyth.network
|
||||
|
||||
# Trading config (safe defaults)
|
||||
MAX_POSITION_SIZE_USD=50
|
||||
LEVERAGE=10
|
||||
DRY_RUN=false
|
||||
|
||||
# Database password (if using PostgreSQL)
|
||||
POSTGRES_PASSWORD=$(openssl rand -hex 16)
|
||||
```
|
||||
|
||||
### Step 2: Build Image
|
||||
|
||||
```bash
|
||||
# Build with cache
|
||||
docker-compose build
|
||||
|
||||
# Build without cache (clean build)
|
||||
docker-compose build --no-cache
|
||||
|
||||
# Build with progress output
|
||||
docker-compose build --progress=plain
|
||||
```
|
||||
|
||||
### Step 3: Start Services
|
||||
|
||||
```bash
|
||||
# Start all services in background
|
||||
docker-compose up -d
|
||||
|
||||
# Start specific service
|
||||
docker-compose up -d trading-bot
|
||||
|
||||
# Start with recreation (force restart)
|
||||
docker-compose up -d --force-recreate
|
||||
```
|
||||
|
||||
### Step 4: Verify Deployment
|
||||
|
||||
```bash
|
||||
# Check running containers
|
||||
docker-compose ps
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f trading-bot
|
||||
|
||||
# Check health
|
||||
docker-compose exec trading-bot wget -qO- http://localhost:3000/api/health
|
||||
|
||||
# Test API
|
||||
curl -H "Authorization: Bearer YOUR_API_KEY" \
|
||||
http://localhost:3000/api/trading/positions
|
||||
```
|
||||
|
||||
### Step 5: Monitor
|
||||
|
||||
```bash
|
||||
# Follow logs in real-time
|
||||
docker-compose logs -f
|
||||
|
||||
# View resource usage
|
||||
docker stats
|
||||
|
||||
# Check container details
|
||||
docker inspect trading-bot-v4
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Development Setup
|
||||
|
||||
### Hot Reload Development
|
||||
|
||||
```bash
|
||||
cd v4
|
||||
|
||||
# Create dev .env
|
||||
cp .env.example .env
|
||||
|
||||
# Set to devnet for safety
|
||||
echo "DRIFT_ENV=devnet" >> .env
|
||||
echo "DRY_RUN=true" >> .env
|
||||
echo "MAX_POSITION_SIZE_USD=10" >> .env
|
||||
|
||||
# Start development container
|
||||
docker-compose -f docker-compose.dev.yml up
|
||||
|
||||
# Rebuild on code changes
|
||||
docker-compose -f docker-compose.dev.yml up --build
|
||||
```
|
||||
|
||||
### Debug Mode
|
||||
|
||||
```bash
|
||||
# Start with Node.js debugger
|
||||
docker-compose -f docker-compose.dev.yml up
|
||||
|
||||
# Attach debugger in VS Code:
|
||||
# 1. Open Debug panel (Ctrl+Shift+D)
|
||||
# 2. Select "Attach to Docker"
|
||||
# 3. Set breakpoints
|
||||
# 4. Start debugging
|
||||
|
||||
# Or use Chrome DevTools:
|
||||
# Open: chrome://inspect
|
||||
# Click: "Configure" → Add localhost:9229
|
||||
```
|
||||
|
||||
### Run Tests in Container
|
||||
|
||||
```bash
|
||||
# Execute tests
|
||||
docker-compose exec trading-bot npm test
|
||||
|
||||
# Run specific test
|
||||
docker-compose exec trading-bot npx tsx v4/test-price-monitor.ts
|
||||
|
||||
# Shell access for manual testing
|
||||
docker-compose exec trading-bot sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Create `.env` file in `v4/` directory:
|
||||
|
||||
```env
|
||||
# Required
|
||||
DRIFT_WALLET_PRIVATE_KEY=your_key
|
||||
SOLANA_RPC_URL=your_rpc
|
||||
API_SECRET_KEY=your_secret
|
||||
|
||||
# Optional overrides
|
||||
MAX_POSITION_SIZE_USD=50
|
||||
LEVERAGE=10
|
||||
LOG_LEVEL=info
|
||||
```
|
||||
|
||||
See `.env.example` for complete list.
|
||||
|
||||
### Docker Compose Override
|
||||
|
||||
Create `docker-compose.override.yml` for local customizations:
|
||||
|
||||
```yaml
|
||||
version: '3.9'
|
||||
|
||||
services:
|
||||
trading-bot:
|
||||
ports:
|
||||
- "3001:3000" # Use different port
|
||||
environment:
|
||||
LOG_LEVEL: debug # More verbose logging
|
||||
volumes:
|
||||
- ./custom-config:/app/config # Custom config directory
|
||||
```
|
||||
|
||||
### Resource Limits
|
||||
|
||||
Edit `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2' # Max 2 CPU cores
|
||||
memory: 2G # Max 2GB RAM
|
||||
reservations:
|
||||
cpus: '1' # Reserve 1 core
|
||||
memory: 1G # Reserve 1GB
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎮 Docker Commands Reference
|
||||
|
||||
### Container Management
|
||||
|
||||
```bash
|
||||
# Start services
|
||||
docker-compose up -d
|
||||
|
||||
# Stop services
|
||||
docker-compose stop
|
||||
|
||||
# Stop and remove containers
|
||||
docker-compose down
|
||||
|
||||
# Restart specific service
|
||||
docker-compose restart trading-bot
|
||||
|
||||
# View container status
|
||||
docker-compose ps
|
||||
|
||||
# View resource usage
|
||||
docker stats trading-bot-v4
|
||||
```
|
||||
|
||||
### Logs & Debugging
|
||||
|
||||
```bash
|
||||
# View all logs
|
||||
docker-compose logs
|
||||
|
||||
# Follow logs in real-time
|
||||
docker-compose logs -f trading-bot
|
||||
|
||||
# Last 100 lines
|
||||
docker-compose logs --tail=100 trading-bot
|
||||
|
||||
# Logs since timestamp
|
||||
docker-compose logs --since 2024-10-23T10:00:00
|
||||
|
||||
# Shell access
|
||||
docker-compose exec trading-bot sh
|
||||
|
||||
# Run command in container
|
||||
docker-compose exec trading-bot node -v
|
||||
```
|
||||
|
||||
### Database Operations
|
||||
|
||||
```bash
|
||||
# Access PostgreSQL CLI
|
||||
docker-compose exec postgres psql -U postgres -d trading_bot_v4
|
||||
|
||||
# Backup database
|
||||
docker-compose exec postgres pg_dump -U postgres trading_bot_v4 > backup.sql
|
||||
|
||||
# Restore database
|
||||
docker-compose exec -T postgres psql -U postgres trading_bot_v4 < backup.sql
|
||||
|
||||
# View database logs
|
||||
docker-compose logs postgres
|
||||
```
|
||||
|
||||
### Cleanup
|
||||
|
||||
```bash
|
||||
# Stop and remove containers
|
||||
docker-compose down
|
||||
|
||||
# Remove containers and volumes
|
||||
docker-compose down -v
|
||||
|
||||
# Remove everything including images
|
||||
docker-compose down --rmi all -v
|
||||
|
||||
# Clean up unused Docker resources
|
||||
docker system prune -a
|
||||
```
|
||||
|
||||
### Image Management
|
||||
|
||||
```bash
|
||||
# Build image
|
||||
docker-compose build
|
||||
|
||||
# Rebuild without cache
|
||||
docker-compose build --no-cache
|
||||
|
||||
# Pull latest base images
|
||||
docker-compose pull
|
||||
|
||||
# View images
|
||||
docker images | grep trading-bot
|
||||
|
||||
# Remove old images
|
||||
docker rmi trading-bot-v4:old
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### Container Won't Start
|
||||
|
||||
```bash
|
||||
# Check logs for errors
|
||||
docker-compose logs trading-bot
|
||||
|
||||
# Verify environment variables
|
||||
docker-compose config
|
||||
|
||||
# Check if port is already in use
|
||||
sudo lsof -i :3000
|
||||
|
||||
# Rebuild from scratch
|
||||
docker-compose down -v
|
||||
docker-compose build --no-cache
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### Connection Issues
|
||||
|
||||
```bash
|
||||
# Test internal network
|
||||
docker-compose exec trading-bot ping postgres
|
||||
|
||||
# Check exposed ports
|
||||
docker-compose port trading-bot 3000
|
||||
|
||||
# Verify RPC connection
|
||||
docker-compose exec trading-bot wget -qO- $SOLANA_RPC_URL
|
||||
|
||||
# Test Pyth connection
|
||||
docker-compose exec trading-bot wget -qO- https://hermes.pyth.network
|
||||
```
|
||||
|
||||
### Performance Issues
|
||||
|
||||
```bash
|
||||
# Check resource usage
|
||||
docker stats
|
||||
|
||||
# View container processes
|
||||
docker top trading-bot-v4
|
||||
|
||||
# Increase resources in docker-compose.yml
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2'
|
||||
memory: 2G
|
||||
|
||||
# Restart with new limits
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### Database Issues
|
||||
|
||||
```bash
|
||||
# Check database health
|
||||
docker-compose exec postgres pg_isready
|
||||
|
||||
# View database connections
|
||||
docker-compose exec postgres psql -U postgres -c "SELECT * FROM pg_stat_activity"
|
||||
|
||||
# Reset database
|
||||
docker-compose down -v postgres
|
||||
docker-compose up -d postgres
|
||||
```
|
||||
|
||||
### Permission Issues
|
||||
|
||||
```bash
|
||||
# Fix volume permissions
|
||||
sudo chown -R 1001:1001 ./logs
|
||||
sudo chmod -R 755 ./logs
|
||||
|
||||
# Run as root (not recommended)
|
||||
docker-compose run --user root trading-bot sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Best Practices
|
||||
|
||||
### Security
|
||||
|
||||
1. **Never commit .env files**
|
||||
```bash
|
||||
echo ".env" >> .gitignore
|
||||
echo ".env.*" >> .gitignore
|
||||
```
|
||||
|
||||
2. **Use secrets for sensitive data**
|
||||
```yaml
|
||||
services:
|
||||
trading-bot:
|
||||
secrets:
|
||||
- drift_private_key
|
||||
|
||||
secrets:
|
||||
drift_private_key:
|
||||
file: ./secrets/drift_key.txt
|
||||
```
|
||||
|
||||
3. **Run as non-root user** (already configured in Dockerfile)
|
||||
|
||||
4. **Limit container capabilities**
|
||||
```yaml
|
||||
cap_drop:
|
||||
- ALL
|
||||
cap_add:
|
||||
- NET_BIND_SERVICE
|
||||
```
|
||||
|
||||
### Performance
|
||||
|
||||
1. **Use multi-stage builds** (already configured)
|
||||
|
||||
2. **Mount volumes for persistence**
|
||||
```yaml
|
||||
volumes:
|
||||
- ./logs:/app/logs
|
||||
- postgres-data:/var/lib/postgresql/data
|
||||
```
|
||||
|
||||
3. **Set resource limits**
|
||||
```yaml
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 1G
|
||||
```
|
||||
|
||||
4. **Use BuildKit for faster builds**
|
||||
```bash
|
||||
DOCKER_BUILDKIT=1 docker-compose build
|
||||
```
|
||||
|
||||
### Monitoring
|
||||
|
||||
1. **Health checks** (already configured)
|
||||
|
||||
2. **Log rotation**
|
||||
```yaml
|
||||
logging:
|
||||
driver: "json-file"
|
||||
options:
|
||||
max-size: "10m"
|
||||
max-file: "3"
|
||||
```
|
||||
|
||||
3. **Metrics collection**
|
||||
```bash
|
||||
# Export container metrics
|
||||
docker stats --format "table {{.Container}}\t{{.CPUPerc}}\t{{.MemUsage}}"
|
||||
```
|
||||
|
||||
### Deployment
|
||||
|
||||
1. **Use tagged images**
|
||||
```bash
|
||||
docker tag trading-bot-v4:latest trading-bot-v4:1.0.0
|
||||
```
|
||||
|
||||
2. **Automated backups**
|
||||
```bash
|
||||
# Backup script
|
||||
docker-compose exec postgres pg_dump -U postgres trading_bot_v4 | \
|
||||
gzip > backup-$(date +%Y%m%d).sql.gz
|
||||
```
|
||||
|
||||
3. **Blue-green deployment**
|
||||
```bash
|
||||
# Start new version on different port
|
||||
docker-compose -f docker-compose.blue.yml up -d
|
||||
|
||||
# Test new version
|
||||
curl http://localhost:3001/api/health
|
||||
|
||||
# Switch traffic (nginx/traefik)
|
||||
# Stop old version
|
||||
docker-compose -f docker-compose.green.yml down
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔗 Additional Resources
|
||||
|
||||
- **Docker Docs**: https://docs.docker.com
|
||||
- **Docker Compose**: https://docs.docker.com/compose
|
||||
- **Node.js Docker**: https://nodejs.org/en/docs/guides/nodejs-docker-webapp
|
||||
- **Next.js Docker**: https://nextjs.org/docs/deployment#docker-image
|
||||
|
||||
---
|
||||
|
||||
## 📞 Support
|
||||
|
||||
For issues specific to:
|
||||
- **Docker setup**: Check this guide first
|
||||
- **Trading bot**: See `../TRADING_BOT_V4_MANUAL.md`
|
||||
- **Phase 2 features**: See `PHASE_2_COMPLETE.md`
|
||||
- **Testing**: See `TESTING.md`
|
||||
|
||||
---
|
||||
|
||||
**Ready to deploy! 🚀**
|
||||
Reference in New Issue
Block a user