diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 811ff2c..2b70e6e 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -614,6 +614,129 @@ docker system df - Active containers - Tagged images currently in use +### Docker Optimization & Build Cache Management (Nov 26, 2025) + +**Purpose:** Prevent Docker cache accumulation (40+ GB) through automated cleanup and BuildKit optimizations + +**Three-Layer Optimization Strategy:** + +**1. Multi-Stage Builds (ALREADY IMPLEMENTED)** +```dockerfile +# Dockerfile already uses multi-stage pattern: +FROM node:20-alpine AS deps # Install dependencies +FROM node:20-alpine AS builder # Build application +FROM node:20-alpine AS runner # Final minimal image + +# Benefits: +# - Smaller final images (only runtime dependencies) +# - Faster builds (caches each stage independently) +# - Better layer reuse +``` + +**2. BuildKit Auto-Cleanup (Nov 26, 2025)** +```bash +# /etc/docker/daemon.json configuration: +{ + "features": { + "buildkit": true + }, + "builder": { + "gc": { + "enabled": true, + "defaultKeepStorage": "20GB" + } + } +} + +# Restart Docker to apply: +sudo systemctl restart docker + +# Verify BuildKit active: +docker buildx version # Should show v0.14.1+ +``` + +**Auto-Cleanup Behavior:** +- **Threshold:** 20GB build cache limit +- **Action:** Automatically garbage collects when exceeded +- **Safety:** Keeps recent layers for build speed +- **Monitoring:** Check current usage: `docker system df` + +**Current Disk Usage Baseline (Nov 26, 2025):** +- Build Cache: 11.13GB (healthy, under 20GB threshold) +- Images: 59.2GB (33.3GB reclaimable) +- Volumes: 8.5GB (7.9GB reclaimable) +- Containers: 232.9MB + +**3. Automated Cleanup Script (READY TO USE)** +```bash +# Script: /home/icke/traderv4/cleanup_trading_bot.sh (94 lines) +# Executable: -rwxr-xr-x (already set) + +# Features: +# - Step 1: Keeps last 2 trading-bot images (rollback safety) +# - Step 2: Removes dangling images (untagged layers) +# - Step 3: Prunes build cache (biggest space saver) +# - Step 4: Safe volume handling (protects postgres) +# - Reporting: Shows disk space before/after + +# Manual usage (recommended after builds): +cd /home/icke/traderv4 +docker compose build trading-bot && ./cleanup_trading_bot.sh + +# Automated usage (daily cleanup at 2 AM): +# Add to crontab: crontab -e +0 2 * * * /home/icke/traderv4/cleanup_trading_bot.sh + +# Check current disk usage: +docker system df +``` + +**Script Safety Measures:** +- **Never removes:** Named volumes (trading-bot-postgres, etc.) +- **Never removes:** Running containers +- **Never removes:** Tagged images currently in use +- **Keeps:** Last 2 trading-bot images for quick rollback +- **Reports:** Space freed after cleanup (typical: 40-50 GB) + +**When to Run Cleanup:** +1. **After builds:** Most effective, immediate cleanup +2. **Weekly:** If building frequently during development +3. **On demand:** When disk space warnings appear +4. **Before deployments:** Clean slate for major updates + +**Typical Space Savings:** +- Manual script run: 40-50 GB (build cache + dangling images) +- BuildKit auto-cleanup: Maintains 20GB cap automatically +- Combined approach: Prevents accumulation entirely + +**Monitoring Commands:** +```bash +# Check current disk usage +docker system df + +# Detailed breakdown +docker system df -v + +# Check BuildKit cache +docker buildx du + +# Verify auto-cleanup threshold +grep -A10 "builder" /etc/docker/daemon.json +``` + +**Why This Matters:** +- **Problem:** User previously hit 40GB cache accumulation +- **Solution:** BuildKit auto-cleanup (20GB cap) + manual script (on-demand) +- **Result:** System self-maintains, prevents disk full scenarios +- **Team benefit:** Documented process for all developers + +**Implementation Status:** +- ✅ Multi-stage builds: Already present in Dockerfile (builder → runner) +- ✅ BuildKit auto-cleanup: Configured in daemon.json (20GB threshold) +- ✅ Cleanup script: Exists and ready (/home/icke/traderv4/cleanup_trading_bot.sh) +- ✅ Docker daemon: Restarted with new config (BuildKit v0.14.1 active) +- ✅ Current state: Healthy (11.13GB cache, under threshold) + --- ## Multi-Timeframe Price Tracking System (Nov 19, 2025)