a676eb4753
PROBLEM: Rebuilding container 4-6 times per session when most changes don't need it
- Every rebuild: 40-70 seconds downtime
- Recent session: 200 seconds downtime that could've been 50 seconds
- Rebuilding for documentation (should be git only)
- Rebuilding for n8n workflows (should be manual import)
- Rebuilding for ENV changes (should be restart only)
SOLUTION: Created comprehensive guide on what actually needs rebuilds
ZERO DOWNTIME (just commit):
- Documentation (.md files)
- Workflows (.json, .pinescript)
- Hot-reload endpoints (roadmap reload)
RESTART ONLY (5-10 seconds):
- ENV variable changes (.env)
- Database schema (prisma migrate + generate)
REBUILD REQUIRED (40-70 seconds):
- Code changes (.ts, .tsx, .js)
- Dependencies (package.json)
- Dockerfile changes
SMART BATCHING:
- Group multiple code changes into ONE rebuild
- Example: 6 fixes → 1 rebuild = 50s total (not 6× rebuilds = 300s)
CREATED FILES:
- docs/ZERO_DOWNTIME_CHANGES.md (comprehensive guide with examples)
- Updated copilot-instructions.md (quick decision matrix)
EXPECTED IMPACT:
- 60-80% reduction in rebuild frequency
- 60-80% reduction in downtime per session
- Better workflow: batch changes, test together, deploy once
User was right: We were rebuilding WAY too often unnecessarily ✅
250 lines
7.9 KiB
Markdown
250 lines
7.9 KiB
Markdown
# Zero-Downtime Changes Guide
|
||
|
||
**Purpose:** Minimize unnecessary container rebuilds/restarts by understanding what requires downtime vs what doesn't.
|
||
|
||
**Key Insight:** Next.js in standalone mode with Docker has **automatic hot-reload for most changes** - we just need to use it correctly!
|
||
|
||
---
|
||
|
||
## ✅ Changes That DON'T Need Container Restart (Zero Downtime)
|
||
|
||
### 1. **API Route Changes** (`app/api/**/route.ts`)
|
||
- **Method:** Just save the file
|
||
- **Hot-reload:** Next.js development mode auto-reloads
|
||
- **Production:** Requires rebuild (but can batch multiple changes)
|
||
- **Example:** Changing validation logic, adding endpoints, fixing bugs
|
||
|
||
### 2. **Documentation Files** (`.md` files)
|
||
- **Method:** Just commit and push
|
||
- **No action needed:** Documentation doesn't affect runtime
|
||
- **Example:** `README.md`, `docs/*.md`, copilot-instructions.md
|
||
|
||
### 3. **Workflow Files** (`workflows/**/*.json`, `workflows/**/*.pinescript`)
|
||
- **Method:** Commit to git, update in TradingView/n8n manually
|
||
- **No container action:** These files don't run inside container
|
||
- **Example:** parse_signal_enhanced.json, indicator scripts
|
||
|
||
### 4. **Environment Variables** (`.env` file)
|
||
- **Method:** Edit `.env` → restart container only
|
||
- **Command:** `docker compose restart trading-bot`
|
||
- **Time:** 5-10 seconds (vs 40-70 seconds for rebuild)
|
||
- **Example:** Changing `MIN_SIGNAL_QUALITY_SCORE=91`
|
||
|
||
### 5. **Database Schema** (`prisma/schema.prisma`)
|
||
- **Method:** `npx prisma migrate dev` → `npx prisma generate` → restart
|
||
- **No rebuild needed:** Prisma client regenerates, restart picks up changes
|
||
- **Time:** 10-15 seconds total
|
||
- **Example:** Adding new fields to Trade table
|
||
|
||
### 6. **Configuration Objects** (`config/trading.ts` - constants only)
|
||
- **Method:** Restart container if constants change
|
||
- **Hot-reload:** If config reads from ENV, just restart
|
||
- **Example:** Changing default values that come from ENV
|
||
|
||
---
|
||
|
||
## ⚠️ Changes That Need Container RESTART Only (10 seconds)
|
||
|
||
### 1. **Environment Variables**
|
||
```bash
|
||
# Edit .env file
|
||
vim /home/icke/traderv4/.env
|
||
|
||
# Restart container (NOT rebuild)
|
||
docker compose restart trading-bot
|
||
|
||
# Verify
|
||
docker logs --tail=30 trading-bot-v4
|
||
```
|
||
|
||
### 2. **Database Schema Updates**
|
||
```bash
|
||
# Generate migration
|
||
npx prisma migrate dev --name add_new_field
|
||
|
||
# Regenerate client
|
||
npx prisma generate
|
||
|
||
# Restart container
|
||
docker compose restart trading-bot
|
||
```
|
||
|
||
### 3. **Service Scripts** (background jobs that don't rebuild)
|
||
- If script changes don't require new npm packages
|
||
- Just restart to pick up changes
|
||
|
||
---
|
||
|
||
## 🔴 Changes That REQUIRE Full Rebuild (40-70 seconds)
|
||
|
||
### 1. **TypeScript/JavaScript Code Changes**
|
||
- **When:** Changes to `lib/`, `app/`, or any `.ts/.tsx/.js` files
|
||
- **Why:** Next.js must recompile TypeScript to JavaScript
|
||
- **Command:**
|
||
```bash
|
||
cd /home/icke/traderv4 && docker compose build trading-bot > /tmp/docker-build-live.log 2>&1 & BUILD_PID=$!; echo "Build started, PID: $BUILD_PID"; tail -f /tmp/docker-build-live.log
|
||
```
|
||
- **Examples:**
|
||
* `lib/trading/position-manager.ts` logic changes
|
||
* `app/api/trading/execute/route.ts` bug fixes
|
||
* New features requiring code changes
|
||
|
||
### 2. **Dependencies** (`package.json`)
|
||
- **When:** Adding/removing npm packages
|
||
- **Why:** Docker layer must install new packages
|
||
- **Command:** Same as above (full rebuild)
|
||
|
||
### 3. **Dockerfile Changes**
|
||
- **When:** Modifying build steps, Node version, etc.
|
||
- **Why:** Docker image structure changed
|
||
- **Command:** Same as above (full rebuild)
|
||
|
||
### 4. **Next.js Config** (`next.config.js`)
|
||
- **When:** Changing webpack, experimental features, output mode
|
||
- **Why:** Build process configuration changed
|
||
- **Command:** Same as above (full rebuild)
|
||
|
||
---
|
||
|
||
## 📊 Rebuild Decision Matrix
|
||
|
||
| Change Type | Action Required | Downtime | Command |
|
||
|------------|-----------------|----------|---------|
|
||
| **Documentation (`.md`)** | None | 0s | `git commit && git push` |
|
||
| **Workflows (`.json`, `.pinescript`)** | Manual import | 0s | Update in TradingView/n8n |
|
||
| **ENV variables (`.env`)** | Restart | 5-10s | `docker compose restart trading-bot` |
|
||
| **Database schema** | Migrate + Restart | 10-15s | `prisma migrate + restart` |
|
||
| **Code changes (`.ts`, `.tsx`)** | **REBUILD** | 40-70s | `docker compose build + restart` |
|
||
| **Dependencies (`package.json`)** | **REBUILD** | 40-70s | `docker compose build + restart` |
|
||
| **Dockerfile** | **REBUILD** | 40-70s | `docker compose build + restart` |
|
||
|
||
---
|
||
|
||
## 🎯 Smart Batching Strategy
|
||
|
||
**Instead of:** Rebuild after every code change (6× rebuilds = 6 minutes downtime)
|
||
|
||
**Do this:** Batch related changes together
|
||
|
||
### Example Session (GOOD):
|
||
```bash
|
||
# 1. Make all code changes in one session
|
||
vim lib/trading/position-manager.ts # Fix 1
|
||
vim app/api/trading/execute/route.ts # Fix 2
|
||
vim lib/notifications/telegram.ts # Fix 3
|
||
|
||
# 2. Test locally if possible (optional)
|
||
|
||
# 3. Commit all changes
|
||
git add -A && git commit -m "fix: Multiple position manager improvements"
|
||
|
||
# 4. ONE rebuild for all changes
|
||
docker compose build trading-bot
|
||
docker compose up -d --force-recreate trading-bot
|
||
|
||
# Total downtime: 50 seconds (instead of 150 seconds for 3 rebuilds)
|
||
```
|
||
|
||
---
|
||
|
||
## 🚀 Hot-Reload Endpoints (Zero Downtime for Data)
|
||
|
||
### Roadmap Updates
|
||
- **Endpoint:** `POST /api/roadmap/reload`
|
||
- **Use case:** Update roadmap website instantly
|
||
- **Command:** `curl -X POST http://localhost:3001/api/roadmap/reload`
|
||
- **Time:** <2 seconds
|
||
|
||
### Future Hot-Reload Candidates
|
||
- Analytics data refresh
|
||
- Settings UI data reload
|
||
- Market data cache refresh
|
||
|
||
---
|
||
|
||
## 📝 Recent Rebuild Analysis (Nov 27, 2025)
|
||
|
||
Looking at terminal history, we rebuilt for:
|
||
|
||
1. ✅ **Phase 7.2 implementation** - Required (code changes)
|
||
2. ❌ **Documentation updates** - NOT required (just commit)
|
||
3. ❌ **n8n workflow fixes** - NOT required (import manually)
|
||
4. ✅ **Price logging fix** - Required (code change)
|
||
5. ✅ **Aggressive health monitor** - Required (code change)
|
||
6. ❌ **Roadmap hot-reload endpoint** - Would be required BUT...
|
||
|
||
**Key realization:** We could have **batched #1, #4, #5, #6 into ONE rebuild** = 50 seconds instead of 200 seconds
|
||
|
||
---
|
||
|
||
## 🎓 Best Practices Going Forward
|
||
|
||
### 1. **Group Related Changes**
|
||
- Work on multiple fixes in same session
|
||
- Commit all at once
|
||
- One rebuild for all changes
|
||
|
||
### 2. **Use Restart When Possible**
|
||
- ENV changes? Just restart (10s)
|
||
- Don't rebuild unless code actually changed
|
||
|
||
### 3. **Test Before Rebuild**
|
||
- Use TypeScript compiler locally: `npx tsc --noEmit`
|
||
- Catch errors before Docker build
|
||
|
||
### 4. **Documentation Never Needs Rebuild**
|
||
- Markdown changes = git commit only
|
||
- Update copilot-instructions.md freely
|
||
|
||
### 5. **Workflow Changes Are External**
|
||
- TradingView indicators = manual copy/paste
|
||
- n8n workflows = manual import
|
||
- No container action needed
|
||
|
||
---
|
||
|
||
## ⚡ Quick Reference Commands
|
||
|
||
### Just Commit (Documentation)
|
||
```bash
|
||
git add -A && git commit -m "docs: Update XYZ" && git push
|
||
```
|
||
|
||
### Restart Only (ENV/Config)
|
||
```bash
|
||
docker compose restart trading-bot
|
||
docker logs --tail=30 trading-bot-v4
|
||
```
|
||
|
||
### Full Rebuild (Code Changes)
|
||
```bash
|
||
cd /home/icke/traderv4 && docker compose build trading-bot > /tmp/docker-build-live.log 2>&1 & BUILD_PID=$!; echo "Build started, PID: $BUILD_PID"; tail -f /tmp/docker-build-live.log
|
||
|
||
# After build completes
|
||
docker compose up -d --force-recreate trading-bot
|
||
docker logs --tail=50 trading-bot-v4
|
||
```
|
||
|
||
---
|
||
|
||
## 💡 Future Improvements
|
||
|
||
1. **Development Mode Container**
|
||
- Separate dev container with hot-reload enabled
|
||
- Test changes instantly without rebuilds
|
||
- Only rebuild for production deployment
|
||
|
||
2. **More Hot-Reload Endpoints**
|
||
- Settings data reload
|
||
- Analytics cache refresh
|
||
- Market data force update
|
||
|
||
3. **Automated Testing**
|
||
- Run tests before rebuild
|
||
- Catch TypeScript errors early
|
||
- Reduce failed rebuild attempts
|
||
|
||
---
|
||
|
||
**Bottom Line:** You're right - we were rebuilding **way too often**. Most changes can be batched, and some don't need rebuilds at all. This guide should cut downtime by 60-80% going forward.
|