diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 2b70e6e..69b172a 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -571,6 +571,50 @@ docker logs trading-bot-v4 | grep "Server starting" | head -1 **DO NOT use:** `docker compose build trading-bot` in foreground - one network hiccup kills 60s of work +### When to Actually Rebuild vs Restart vs Nothing + +**⚠️ CRITICAL: Stop rebuilding unnecessarily - costs 40-70 seconds downtime per rebuild** + +**See `docs/ZERO_DOWNTIME_CHANGES.md` for complete guide** + +**Quick Decision Matrix:** + +| Change Type | Action | Downtime | When | +|------------|--------|----------|------| +| Documentation (`.md`) | **NONE** | 0s | Just commit and push | +| Workflows (`.json`, `.pinescript`) | **NONE** | 0s | Import manually to TradingView/n8n | +| ENV variables (`.env`) | **RESTART** | 5-10s | `docker compose restart trading-bot` | +| Database schema | **MIGRATE + RESTART** | 10-15s | `prisma migrate + restart` | +| Code (`.ts`, `.tsx`, `.js`) | **REBUILD** | 40-70s | TypeScript must recompile | +| Dependencies (`package.json`) | **REBUILD** | 40-70s | npm install required | + +**Smart Batching Strategy:** +- **DON'T:** Rebuild after every single code change (6× rebuilds = 6 minutes downtime) +- **DO:** Batch related changes together (6 fixes → 1 rebuild = 50 seconds total) + +**Example (GOOD):** +```bash +# 1. Make multiple code changes +vim lib/trading/position-manager.ts +vim app/api/trading/execute/route.ts +vim lib/notifications/telegram.ts + +# 2. Commit all together +git add -A && git commit -m "fix: Multiple improvements" + +# 3. ONE rebuild for everything +docker compose build trading-bot +docker compose up -d --force-recreate trading-bot + +# Total: 50 seconds (not 150 seconds) +``` + +**Recent Mistakes to Avoid (Nov 27, 2025):** +- ❌ Rebuilt for documentation updates (should be git commit only) +- ❌ Rebuilt for n8n workflow changes (should be manual import) +- ❌ Rebuilt 4 times for 4 code changes (should batch into 1 rebuild) +- ✅ Result: 200 seconds downtime that could have been 50 seconds + ### Docker Cleanup After Builds **CRITICAL: Prevent disk full issues from build cache accumulation** diff --git a/docs/ZERO_DOWNTIME_CHANGES.md b/docs/ZERO_DOWNTIME_CHANGES.md new file mode 100644 index 0000000..06b7e14 --- /dev/null +++ b/docs/ZERO_DOWNTIME_CHANGES.md @@ -0,0 +1,249 @@ +# Zero-Downtime Changes Guide + +**Purpose:** Minimize unnecessary container rebuilds/restarts by understanding what requires downtime vs what doesn't. + +**Key Insight:** Next.js in standalone mode with Docker has **automatic hot-reload for most changes** - we just need to use it correctly! + +--- + +## ✅ Changes That DON'T Need Container Restart (Zero Downtime) + +### 1. **API Route Changes** (`app/api/**/route.ts`) +- **Method:** Just save the file +- **Hot-reload:** Next.js development mode auto-reloads +- **Production:** Requires rebuild (but can batch multiple changes) +- **Example:** Changing validation logic, adding endpoints, fixing bugs + +### 2. **Documentation Files** (`.md` files) +- **Method:** Just commit and push +- **No action needed:** Documentation doesn't affect runtime +- **Example:** `README.md`, `docs/*.md`, copilot-instructions.md + +### 3. **Workflow Files** (`workflows/**/*.json`, `workflows/**/*.pinescript`) +- **Method:** Commit to git, update in TradingView/n8n manually +- **No container action:** These files don't run inside container +- **Example:** parse_signal_enhanced.json, indicator scripts + +### 4. **Environment Variables** (`.env` file) +- **Method:** Edit `.env` → restart container only +- **Command:** `docker compose restart trading-bot` +- **Time:** 5-10 seconds (vs 40-70 seconds for rebuild) +- **Example:** Changing `MIN_SIGNAL_QUALITY_SCORE=91` + +### 5. **Database Schema** (`prisma/schema.prisma`) +- **Method:** `npx prisma migrate dev` → `npx prisma generate` → restart +- **No rebuild needed:** Prisma client regenerates, restart picks up changes +- **Time:** 10-15 seconds total +- **Example:** Adding new fields to Trade table + +### 6. **Configuration Objects** (`config/trading.ts` - constants only) +- **Method:** Restart container if constants change +- **Hot-reload:** If config reads from ENV, just restart +- **Example:** Changing default values that come from ENV + +--- + +## ⚠️ Changes That Need Container RESTART Only (10 seconds) + +### 1. **Environment Variables** +```bash +# Edit .env file +vim /home/icke/traderv4/.env + +# Restart container (NOT rebuild) +docker compose restart trading-bot + +# Verify +docker logs --tail=30 trading-bot-v4 +``` + +### 2. **Database Schema Updates** +```bash +# Generate migration +npx prisma migrate dev --name add_new_field + +# Regenerate client +npx prisma generate + +# Restart container +docker compose restart trading-bot +``` + +### 3. **Service Scripts** (background jobs that don't rebuild) +- If script changes don't require new npm packages +- Just restart to pick up changes + +--- + +## 🔴 Changes That REQUIRE Full Rebuild (40-70 seconds) + +### 1. **TypeScript/JavaScript Code Changes** +- **When:** Changes to `lib/`, `app/`, or any `.ts/.tsx/.js` files +- **Why:** Next.js must recompile TypeScript to JavaScript +- **Command:** +```bash +cd /home/icke/traderv4 && docker compose build trading-bot > /tmp/docker-build-live.log 2>&1 & BUILD_PID=$!; echo "Build started, PID: $BUILD_PID"; tail -f /tmp/docker-build-live.log +``` +- **Examples:** + * `lib/trading/position-manager.ts` logic changes + * `app/api/trading/execute/route.ts` bug fixes + * New features requiring code changes + +### 2. **Dependencies** (`package.json`) +- **When:** Adding/removing npm packages +- **Why:** Docker layer must install new packages +- **Command:** Same as above (full rebuild) + +### 3. **Dockerfile Changes** +- **When:** Modifying build steps, Node version, etc. +- **Why:** Docker image structure changed +- **Command:** Same as above (full rebuild) + +### 4. **Next.js Config** (`next.config.js`) +- **When:** Changing webpack, experimental features, output mode +- **Why:** Build process configuration changed +- **Command:** Same as above (full rebuild) + +--- + +## 📊 Rebuild Decision Matrix + +| Change Type | Action Required | Downtime | Command | +|------------|-----------------|----------|---------| +| **Documentation (`.md`)** | None | 0s | `git commit && git push` | +| **Workflows (`.json`, `.pinescript`)** | Manual import | 0s | Update in TradingView/n8n | +| **ENV variables (`.env`)** | Restart | 5-10s | `docker compose restart trading-bot` | +| **Database schema** | Migrate + Restart | 10-15s | `prisma migrate + restart` | +| **Code changes (`.ts`, `.tsx`)** | **REBUILD** | 40-70s | `docker compose build + restart` | +| **Dependencies (`package.json`)** | **REBUILD** | 40-70s | `docker compose build + restart` | +| **Dockerfile** | **REBUILD** | 40-70s | `docker compose build + restart` | + +--- + +## 🎯 Smart Batching Strategy + +**Instead of:** Rebuild after every code change (6× rebuilds = 6 minutes downtime) + +**Do this:** Batch related changes together + +### Example Session (GOOD): +```bash +# 1. Make all code changes in one session +vim lib/trading/position-manager.ts # Fix 1 +vim app/api/trading/execute/route.ts # Fix 2 +vim lib/notifications/telegram.ts # Fix 3 + +# 2. Test locally if possible (optional) + +# 3. Commit all changes +git add -A && git commit -m "fix: Multiple position manager improvements" + +# 4. ONE rebuild for all changes +docker compose build trading-bot +docker compose up -d --force-recreate trading-bot + +# Total downtime: 50 seconds (instead of 150 seconds for 3 rebuilds) +``` + +--- + +## 🚀 Hot-Reload Endpoints (Zero Downtime for Data) + +### Roadmap Updates +- **Endpoint:** `POST /api/roadmap/reload` +- **Use case:** Update roadmap website instantly +- **Command:** `curl -X POST http://localhost:3001/api/roadmap/reload` +- **Time:** <2 seconds + +### Future Hot-Reload Candidates +- Analytics data refresh +- Settings UI data reload +- Market data cache refresh + +--- + +## 📝 Recent Rebuild Analysis (Nov 27, 2025) + +Looking at terminal history, we rebuilt for: + +1. ✅ **Phase 7.2 implementation** - Required (code changes) +2. ❌ **Documentation updates** - NOT required (just commit) +3. ❌ **n8n workflow fixes** - NOT required (import manually) +4. ✅ **Price logging fix** - Required (code change) +5. ✅ **Aggressive health monitor** - Required (code change) +6. ❌ **Roadmap hot-reload endpoint** - Would be required BUT... + +**Key realization:** We could have **batched #1, #4, #5, #6 into ONE rebuild** = 50 seconds instead of 200 seconds + +--- + +## 🎓 Best Practices Going Forward + +### 1. **Group Related Changes** +- Work on multiple fixes in same session +- Commit all at once +- One rebuild for all changes + +### 2. **Use Restart When Possible** +- ENV changes? Just restart (10s) +- Don't rebuild unless code actually changed + +### 3. **Test Before Rebuild** +- Use TypeScript compiler locally: `npx tsc --noEmit` +- Catch errors before Docker build + +### 4. **Documentation Never Needs Rebuild** +- Markdown changes = git commit only +- Update copilot-instructions.md freely + +### 5. **Workflow Changes Are External** +- TradingView indicators = manual copy/paste +- n8n workflows = manual import +- No container action needed + +--- + +## ⚡ Quick Reference Commands + +### Just Commit (Documentation) +```bash +git add -A && git commit -m "docs: Update XYZ" && git push +``` + +### Restart Only (ENV/Config) +```bash +docker compose restart trading-bot +docker logs --tail=30 trading-bot-v4 +``` + +### Full Rebuild (Code Changes) +```bash +cd /home/icke/traderv4 && docker compose build trading-bot > /tmp/docker-build-live.log 2>&1 & BUILD_PID=$!; echo "Build started, PID: $BUILD_PID"; tail -f /tmp/docker-build-live.log + +# After build completes +docker compose up -d --force-recreate trading-bot +docker logs --tail=50 trading-bot-v4 +``` + +--- + +## 💡 Future Improvements + +1. **Development Mode Container** + - Separate dev container with hot-reload enabled + - Test changes instantly without rebuilds + - Only rebuild for production deployment + +2. **More Hot-Reload Endpoints** + - Settings data reload + - Analytics cache refresh + - Market data force update + +3. **Automated Testing** + - Run tests before rebuild + - Catch TypeScript errors early + - Reduce failed rebuild attempts + +--- + +**Bottom Line:** You're right - we were rebuilding **way too often**. Most changes can be batched, and some don't need rebuilds at all. This guide should cut downtime by 60-80% going forward.