root/trading_bot_v4
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Add zero-downtime changes guide - stop unnecessary rebuilds

Browse Source 
PROBLEM: Rebuilding container 4-6 times per session when most changes don't need it
- Every rebuild: 40-70 seconds downtime
- Recent session: 200 seconds downtime that could've been 50 seconds
- Rebuilding for documentation (should be git only)
- Rebuilding for n8n workflows (should be manual import)
- Rebuilding for ENV changes (should be restart only)

SOLUTION: Created comprehensive guide on what actually needs rebuilds

ZERO DOWNTIME (just commit):
- Documentation (.md files)
- Workflows (.json, .pinescript)
- Hot-reload endpoints (roadmap reload)

RESTART ONLY (5-10 seconds):
- ENV variable changes (.env)
- Database schema (prisma migrate + generate)

REBUILD REQUIRED (40-70 seconds):
- Code changes (.ts, .tsx, .js)
- Dependencies (package.json)
- Dockerfile changes

SMART BATCHING:
- Group multiple code changes into ONE rebuild
- Example: 6 fixes → 1 rebuild = 50s total (not 6× rebuilds = 300s)

CREATED FILES:
- docs/ZERO_DOWNTIME_CHANGES.md (comprehensive guide with examples)
- Updated copilot-instructions.md (quick decision matrix)

EXPECTED IMPACT:
- 60-80% reduction in rebuild frequency
- 60-80% reduction in downtime per session
- Better workflow: batch changes, test together, deploy once

User was right: We were rebuilding WAY too often unnecessarily ✅
This commit is contained in:
mindesbunister
2025-11-27 14:08:42 +01:00
parent 49f19b1a8c
commit a676eb4753
2 changed files with 293 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
.github/copilot-instructions.md vendored
View File

@@ -571,6 +571,50 @@ docker logs trading-bot-v4 | grep "Server starting" | head -1
**DO NOT use:** `docker compose build trading-bot` in foreground - one network hiccup kills 60s of work **DO NOT use:** `docker compose build trading-bot` in foreground - one network hiccup kills 60s of work
### When to Actually Rebuild vs Restart vs Nothing
**⚠️ CRITICAL: Stop rebuilding unnecessarily - costs 40-70 seconds downtime per rebuild**
**See `docs/ZERO_DOWNTIME_CHANGES.md` for complete guide**
**Quick Decision Matrix:**
| Change Type | Action | Downtime | When |
|------------|--------|----------|------|
| Documentation (`.md`) | **NONE** | 0s | Just commit and push |
| Workflows (`.json`, `.pinescript`) | **NONE** | 0s | Import manually to TradingView/n8n |
| ENV variables (`.env`) | **RESTART** | 5-10s | `docker compose restart trading-bot` |
| Database schema | **MIGRATE + RESTART** | 10-15s | `prisma migrate + restart` |
| Code (`.ts`, `.tsx`, `.js`) | **REBUILD** | 40-70s | TypeScript must recompile |
| Dependencies (`package.json`) | **REBUILD** | 40-70s | npm install required |
**Smart Batching Strategy:**
- **DON'T:** Rebuild after every single code change (6× rebuilds = 6 minutes downtime)
- **DO:** Batch related changes together (6 fixes → 1 rebuild = 50 seconds total)
**Example (GOOD):**
```bash
# 1. Make multiple code changes
vim lib/trading/position-manager.ts
vim app/api/trading/execute/route.ts
vim lib/notifications/telegram.ts
# 2. Commit all together
git add -A && git commit -m "fix: Multiple improvements"
# 3. ONE rebuild for everything
docker compose build trading-bot
docker compose up -d --force-recreate trading-bot
# Total: 50 seconds (not 150 seconds)
```
**Recent Mistakes to Avoid (Nov 27, 2025):**
- ❌ Rebuilt for documentation updates (should be git commit only)
- ❌ Rebuilt for n8n workflow changes (should be manual import)
- ❌ Rebuilt 4 times for 4 code changes (should batch into 1 rebuild)
- ✅ Result: 200 seconds downtime that could have been 50 seconds
### Docker Cleanup After Builds ### Docker Cleanup After Builds
**CRITICAL: Prevent disk full issues from build cache accumulation** **CRITICAL: Prevent disk full issues from build cache accumulation**

249
docs/ZERO_DOWNTIME_CHANGES.md Normal file
View File

@@ -0,0 +1,249 @@
# Zero-Downtime Changes Guide
**Purpose:** Minimize unnecessary container rebuilds/restarts by understanding what requires downtime vs what doesn't.
**Key Insight:** Next.js in standalone mode with Docker has **automatic hot-reload for most changes** - we just need to use it correctly!
---
## ✅ Changes That DON'T Need Container Restart (Zero Downtime)
### 1. **API Route Changes** (`app/api/**/route.ts`)
- **Method:** Just save the file
- **Hot-reload:** Next.js development mode auto-reloads
- **Production:** Requires rebuild (but can batch multiple changes)
- **Example:** Changing validation logic, adding endpoints, fixing bugs
### 2. **Documentation Files** (`.md` files)
- **Method:** Just commit and push
- **No action needed:** Documentation doesn't affect runtime
- **Example:** `README.md`, `docs/*.md`, copilot-instructions.md
### 3. **Workflow Files** (`workflows/**/*.json`, `workflows/**/*.pinescript`)
- **Method:** Commit to git, update in TradingView/n8n manually
- **No container action:** These files don't run inside container
- **Example:** parse_signal_enhanced.json, indicator scripts
### 4. **Environment Variables** (`.env` file)
- **Method:** Edit `.env` → restart container only
- **Command:** `docker compose restart trading-bot`
- **Time:** 5-10 seconds (vs 40-70 seconds for rebuild)
- **Example:** Changing `MIN_SIGNAL_QUALITY_SCORE=91`
### 5. **Database Schema** (`prisma/schema.prisma`)
- **Method:** `npx prisma migrate dev` → `npx prisma generate` → restart
- **No rebuild needed:** Prisma client regenerates, restart picks up changes
- **Time:** 10-15 seconds total
- **Example:** Adding new fields to Trade table
### 6. **Configuration Objects** (`config/trading.ts` - constants only)
- **Method:** Restart container if constants change
- **Hot-reload:** If config reads from ENV, just restart
- **Example:** Changing default values that come from ENV
---
## ⚠️ Changes That Need Container RESTART Only (10 seconds)
### 1. **Environment Variables**
```bash
# Edit .env file
vim /home/icke/traderv4/.env
# Restart container (NOT rebuild)
docker compose restart trading-bot
# Verify
docker logs --tail=30 trading-bot-v4
```
### 2. **Database Schema Updates**
```bash
# Generate migration
npx prisma migrate dev --name add_new_field
# Regenerate client
npx prisma generate
# Restart container
docker compose restart trading-bot
```
### 3. **Service Scripts** (background jobs that don't rebuild)
- If script changes don't require new npm packages
- Just restart to pick up changes
---
## 🔴 Changes That REQUIRE Full Rebuild (40-70 seconds)
### 1. **TypeScript/JavaScript Code Changes**
- **When:** Changes to `lib/`, `app/`, or any `.ts/.tsx/.js` files
- **Why:** Next.js must recompile TypeScript to JavaScript
- **Command:**
```bash
cd /home/icke/traderv4 && docker compose build trading-bot > /tmp/docker-build-live.log 2>&1 & BUILD_PID=$!; echo "Build started, PID: $BUILD_PID"; tail -f /tmp/docker-build-live.log
```
- **Examples:**
* `lib/trading/position-manager.ts` logic changes
* `app/api/trading/execute/route.ts` bug fixes
* New features requiring code changes
### 2. **Dependencies** (`package.json`)
- **When:** Adding/removing npm packages
- **Why:** Docker layer must install new packages
- **Command:** Same as above (full rebuild)
### 3. **Dockerfile Changes**
- **When:** Modifying build steps, Node version, etc.
- **Why:** Docker image structure changed
- **Command:** Same as above (full rebuild)
### 4. **Next.js Config** (`next.config.js`)
- **When:** Changing webpack, experimental features, output mode
- **Why:** Build process configuration changed
- **Command:** Same as above (full rebuild)
---
## 📊 Rebuild Decision Matrix
| Change Type | Action Required | Downtime | Command |
|------------|-----------------|----------|---------|
| **Documentation (`.md`)** | None | 0s | `git commit && git push` |
| **Workflows (`.json`, `.pinescript`)** | Manual import | 0s | Update in TradingView/n8n |
| **ENV variables (`.env`)** | Restart | 5-10s | `docker compose restart trading-bot` |
| **Database schema** | Migrate + Restart | 10-15s | `prisma migrate + restart` |
| **Code changes (`.ts`, `.tsx`)** | **REBUILD** | 40-70s | `docker compose build + restart` |
| **Dependencies (`package.json`)** | **REBUILD** | 40-70s | `docker compose build + restart` |
| **Dockerfile** | **REBUILD** | 40-70s | `docker compose build + restart` |
---
## 🎯 Smart Batching Strategy
**Instead of:** Rebuild after every code change (6× rebuilds = 6 minutes downtime)
**Do this:** Batch related changes together
### Example Session (GOOD):
```bash
# 1. Make all code changes in one session
vim lib/trading/position-manager.ts # Fix 1
vim app/api/trading/execute/route.ts # Fix 2
vim lib/notifications/telegram.ts # Fix 3
# 2. Test locally if possible (optional)
# 3. Commit all changes
git add -A && git commit -m "fix: Multiple position manager improvements"
# 4. ONE rebuild for all changes
docker compose build trading-bot
docker compose up -d --force-recreate trading-bot
# Total downtime: 50 seconds (instead of 150 seconds for 3 rebuilds)
```
---
## 🚀 Hot-Reload Endpoints (Zero Downtime for Data)
### Roadmap Updates
- **Endpoint:** `POST /api/roadmap/reload`
- **Use case:** Update roadmap website instantly
- **Command:** `curl -X POST http://localhost:3001/api/roadmap/reload`
- **Time:** <2 seconds
### Future Hot-Reload Candidates
- Analytics data refresh
- Settings UI data reload
- Market data cache refresh
---
## 📝 Recent Rebuild Analysis (Nov 27, 2025)
Looking at terminal history, we rebuilt for:
1. ✅ **Phase 7.2 implementation** - Required (code changes)
2. ❌ **Documentation updates** - NOT required (just commit)
3. ❌ **n8n workflow fixes** - NOT required (import manually)
4. ✅ **Price logging fix** - Required (code change)
5. ✅ **Aggressive health monitor** - Required (code change)
6. ❌ **Roadmap hot-reload endpoint** - Would be required BUT...
**Key realization:** We could have **batched #1, #4, #5, #6 into ONE rebuild** = 50 seconds instead of 200 seconds
---
## 🎓 Best Practices Going Forward
### 1. **Group Related Changes**
- Work on multiple fixes in same session
- Commit all at once
- One rebuild for all changes
### 2. **Use Restart When Possible**
- ENV changes? Just restart (10s)
- Don't rebuild unless code actually changed
### 3. **Test Before Rebuild**
- Use TypeScript compiler locally: `npx tsc --noEmit`
- Catch errors before Docker build
### 4. **Documentation Never Needs Rebuild**
- Markdown changes = git commit only
- Update copilot-instructions.md freely
### 5. **Workflow Changes Are External**
- TradingView indicators = manual copy/paste
- n8n workflows = manual import
- No container action needed
---
## ⚡ Quick Reference Commands
### Just Commit (Documentation)
```bash
git add -A && git commit -m "docs: Update XYZ" && git push
```
### Restart Only (ENV/Config)
```bash
docker compose restart trading-bot
docker logs --tail=30 trading-bot-v4
```
### Full Rebuild (Code Changes)
```bash
cd /home/icke/traderv4 && docker compose build trading-bot > /tmp/docker-build-live.log 2>&1 & BUILD_PID=$!; echo "Build started, PID: $BUILD_PID"; tail -f /tmp/docker-build-live.log
# After build completes
docker compose up -d --force-recreate trading-bot
docker logs --tail=50 trading-bot-v4
```
---
## 💡 Future Improvements
1. **Development Mode Container**
- Separate dev container with hot-reload enabled
- Test changes instantly without rebuilds
- Only rebuild for production deployment
2. **More Hot-Reload Endpoints**
- Settings data reload
- Analytics cache refresh
- Market data force update
3. **Automated Testing**
- Run tests before rebuild
- Catch TypeScript errors early
- Reduce failed rebuild attempts
---
**Bottom Line:** You're right - we were rebuilding **way too often**. Most changes can be batched, and some don't need rebuilds at all. This guide should cut downtime by 60-80% going forward.