docs: Major documentation reorganization + ENV variable reference
**Documentation Structure:** - Created docs/ subdirectory organization (analysis/, architecture/, bugs/, cluster/, deployments/, roadmaps/, setup/, archived/) - Moved 68 root markdown files to appropriate categories - Root directory now clean (only README.md remains) - Total: 83 markdown files now organized by purpose **New Content:** - Added comprehensive Environment Variable Reference to copilot-instructions.md - 100+ ENV variables documented with types, defaults, purpose, notes - Organized by category: Required (Drift/RPC/Pyth), Trading Config (quality/ leverage/sizing), ATR System, Runner System, Risk Limits, Notifications, etc. - Includes usage examples (correct vs wrong patterns) **File Distribution:** - docs/analysis/ - Performance analyses, blocked signals, profit projections - docs/architecture/ - Adaptive leverage, ATR trailing, indicator tracking - docs/bugs/ - CRITICAL_*.md, FIXES_*.md bug reports (7 files) - docs/cluster/ - EPYC setup, distributed computing docs (3 files) - docs/deployments/ - *_COMPLETE.md, DEPLOYMENT_*.md status (12 files) - docs/roadmaps/ - All *ROADMAP*.md strategic planning files (7 files) - docs/setup/ - TradingView guides, signal quality, n8n setup (8 files) - docs/archived/2025_pre_nov/ - Obsolete verification checklist (1 file) **Key Improvements:** - ENV variable reference: Single source of truth for all configuration - Common Pitfalls #68-71: Already complete, verified during audit - Better findability: Category-based navigation vs 68 files in root - Preserves history: All files git mv (rename), not copy/delete - Zero broken functionality: Only documentation moved, no code changes **Verification:** - 83 markdown files now in docs/ subdirectories - Root directory cleaned: 68 files → 0 files (except README.md) - Git history preserved for all moved files - Container running: trading-bot-v4 (no restart needed) **Next Steps:** - Create README.md files in each docs subdirectory - Add navigation index - Update main README.md with new structure - Consolidate duplicate deployment docs - Archive truly obsolete files (old SQL backups) See: docs/analysis/CLEANUP_PLAN.md for complete reorganization strategy
This commit is contained in:
mindesbunister
232
docs/archived/2025_pre_nov/VERIFICATION_CHECKLIST_NOV12.md
Normal file
232
docs/archived/2025_pre_nov/VERIFICATION_CHECKLIST_NOV12.md
Normal file
@@ -0,0 +1,232 @@
|
||||
# Verification Checklist - External Closure Duplicate Bug Fix
|
||||
|
||||
**Date:** November 12, 2025
|
||||
**Issue:** Position Manager was recording external closures 5-8 times, compounding P&L
|
||||
**Fix Applied:** Remove trade from `activeTrades` Map BEFORE database update
|
||||
**Status:** 🔴 **UNVERIFIED** - Code deployed but not tested with real trade
|
||||
|
||||
---
|
||||
|
||||
## What Was Fixed
|
||||
|
||||
**File:** `lib/trading/position-manager.ts` (lines 493-530)
|
||||
|
||||
**The Bug:**
|
||||
```typescript
|
||||
// OLD (BROKEN):
|
||||
await updateTradeExit({ realizedPnL: -$7.98 }) // Update DB
|
||||
await this.removeTrade(trade.id) // Remove from monitoring
|
||||
// Problem: Monitoring loop runs again before removal, processes AGAIN
|
||||
```
|
||||
|
||||
**The Fix:**
|
||||
```typescript
|
||||
// NEW (SHOULD WORK):
|
||||
this.activeTrades.delete(trade.id) // Remove FIRST
|
||||
await updateTradeExit({ realizedPnL: -$7.98 }) // Then update DB
|
||||
// If loop runs again, trade already gone, skips duplicate update
|
||||
```
|
||||
|
||||
**Evidence of Bug:** Last trade (cmhvwlmcr0000r007g0sizd7s) showed:
|
||||
- Dashboard: -$58.43 loss (WRONG - 8× actual)
|
||||
- Drift Protocol: -$7.98 loss (CORRECT)
|
||||
- Logs: 8 consecutive "External closure recorded" messages
|
||||
|
||||
---
|
||||
|
||||
## Manual Corrections Applied
|
||||
|
||||
✅ **Database corrected for last trade:**
|
||||
```sql
|
||||
-- Entry price: $160.62259 → $160.512 (matches Drift)
|
||||
-- Exit price: $160.09242 → $159.967 (matches Drift)
|
||||
-- P&L: -$58.43 → -$7.98 (matches Drift -0.34%)
|
||||
```
|
||||
|
||||
✅ **Verification query:**
|
||||
```bash
|
||||
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c \
|
||||
"SELECT \"entryPrice\", \"exitPrice\", \"realizedPnL\" FROM \"Trade\"
|
||||
WHERE id = 'cmhvwlmcr0000r007g0sizd7s';"
|
||||
```
|
||||
Result: `160.512 | 159.967 | -7.98` ✅ CORRECT
|
||||
|
||||
---
|
||||
|
||||
## Verification Requirements (Next External Closure)
|
||||
|
||||
### 1. Watch Docker Logs in Real-Time
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
docker logs -f trading-bot-v4 | grep -E "(External closure|DUPLICATE|Removed trade)"
|
||||
```
|
||||
|
||||
**What to Look For:**
|
||||
|
||||
✅ **CORRECT behavior (fix working):**
|
||||
```
|
||||
📊 SOL-PERP | Price: 160.0246 | P&L: -0.37%
|
||||
🗑️ Removed trade cmh... from monitoring (BEFORE DB update to prevent duplicates)
|
||||
Active trades remaining: 0
|
||||
💾 External closure recorded: SL at $160.0246 | P&L: $-7.96
|
||||
```
|
||||
**Only ONE "External closure recorded" line!**
|
||||
|
||||
❌ **BROKEN behavior (fix failed):**
|
||||
```
|
||||
💾 External closure recorded: SL at $160.0246 | P&L: $-7.96
|
||||
💾 External closure recorded: SL at $160.0359 | P&L: $-15.92 ← DUPLICATE!
|
||||
💾 External closure recorded: SL at $160.0472 | P&L: $-23.88 ← DUPLICATE!
|
||||
```
|
||||
**Multiple "External closure recorded" = BUG STILL EXISTS**
|
||||
|
||||
🎯 **BEST outcome (proves fix caught duplicate attempt):**
|
||||
```
|
||||
🗑️ Removed trade cmh... from monitoring (BEFORE DB update to prevent duplicates)
|
||||
💾 External closure recorded: SL at $160.0246 | P&L: $-7.96
|
||||
⚠️ DUPLICATE PROCESSING PREVENTED: Trade cmh... already removed from monitoring
|
||||
This is the bug fix working - without it, we'd update DB again with compounded P&L
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Compare Database P&L to Drift Protocol
|
||||
|
||||
**After next external closure, run:**
|
||||
|
||||
```bash
|
||||
# Get last trade from database
|
||||
docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c \
|
||||
"SELECT id, symbol, direction, \"entryPrice\", \"exitPrice\",
|
||||
\"positionSizeUSD\", \"realizedPnL\", \"exitReason\",
|
||||
TO_CHAR(\"exitTime\", 'HH24:MI:SS') as exit_time
|
||||
FROM \"Trade\"
|
||||
ORDER BY \"createdAt\" DESC LIMIT 1;"
|
||||
```
|
||||
|
||||
**Then check Drift Protocol UI:**
|
||||
- Go to Position History
|
||||
- Find matching trade by timestamp
|
||||
- Compare: Entry Price, Exit Price, P&L %
|
||||
|
||||
**Manual P&L calculation:**
|
||||
```
|
||||
For LONG: (exitPrice - entryPrice) × positionSize_in_tokens
|
||||
For SHORT: (entryPrice - exitPrice) × positionSize_in_tokens
|
||||
|
||||
Example from last trade:
|
||||
(159.967 - 160.512) × 13.3 SOL = -$7.25
|
||||
Database showed: -$7.98 ✅ Close enough (slippage)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Count Database Updates
|
||||
|
||||
**Check how many times the trade was updated:**
|
||||
|
||||
```sql
|
||||
-- Look for the trade ID in SystemEvent logs
|
||||
SELECT "eventType", "message", "createdAt"
|
||||
FROM "SystemEvent"
|
||||
WHERE "message" LIKE '%cmh%' -- Replace with actual trade ID
|
||||
ORDER BY "createdAt" DESC;
|
||||
```
|
||||
|
||||
**Expected:** 1 update event per external closure
|
||||
**Bug present:** 5-8 update events for same closure
|
||||
|
||||
---
|
||||
|
||||
### 4. Check for Price Mismatch
|
||||
|
||||
**Entry/exit prices should match Drift within 0.5%:**
|
||||
|
||||
```bash
|
||||
# From Drift UI: Note actual fill prices
|
||||
# From database: Compare with query above
|
||||
|
||||
Acceptable difference: < 0.5% (slippage)
|
||||
Large difference (>1%): Indicates oracle price mismatch
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
**Next trade that closes externally (TP1/TP2/SL on-chain order):**
|
||||
|
||||
- [ ] **Logs:** Only ONE "External closure recorded" message
|
||||
- [ ] **Logs:** "Removed trade ... BEFORE DB update" appears once
|
||||
- [ ] **Logs:** No multiple consecutive closure messages
|
||||
- [ ] **Database P&L:** Matches Drift Protocol actual P&L (within 1%)
|
||||
- [ ] **Entry/Exit Prices:** Match Drift order fills (within 0.5%)
|
||||
- [ ] **No duplicate updates:** Trade ID appears once in recent database queries
|
||||
|
||||
**If all checks pass:** ✅ Fix is VERIFIED - update Common Pitfall #26 as "VERIFIED"
|
||||
|
||||
**If any check fails:** 🔴 Bug still exists - investigate further
|
||||
|
||||
---
|
||||
|
||||
## Additional Monitoring (Next 5 External Closures)
|
||||
|
||||
Track these trades to ensure fix is stable:
|
||||
|
||||
| Trade # | Symbol | Exit Time | DB P&L | Drift P&L | Match? | Logs OK? |
|
||||
|---------|--------|-----------|--------|-----------|--------|----------|
|
||||
| 1 | | | | | | |
|
||||
| 2 | | | | | | |
|
||||
| 3 | | | | | | |
|
||||
| 4 | | | | | | |
|
||||
| 5 | | | | | | |
|
||||
|
||||
**Once 5 consecutive trades verify correctly:** Update status to ✅ **VERIFIED & STABLE**
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan (If Fix Fails)
|
||||
|
||||
If verification fails and bug persists:
|
||||
|
||||
1. **Immediate:** Disable automated trading via settings UI
|
||||
2. **Check:** Review Position Manager logs for error patterns
|
||||
3. **Revert:** Restore previous version from git
|
||||
4. **Debug:** Add more extensive logging before retry
|
||||
5. **Document:** Update Common Pitfall #26 with findings
|
||||
|
||||
---
|
||||
|
||||
## Related Issues to Watch For
|
||||
|
||||
While monitoring, also check for:
|
||||
|
||||
- [ ] TP1 detection working correctly (75% close, 25% runner)
|
||||
- [ ] SL moves to breakeven after TP1 as expected
|
||||
- [ ] Trailing stop activates at TP2 trigger
|
||||
- [ ] MAE/MFE tracking updates correctly
|
||||
- [ ] No phantom trades (position size mismatches)
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
**Fix is considered successful when:**
|
||||
1. Next 5 external closures show NO duplicate "External closure recorded" messages
|
||||
2. Database P&L matches Drift Protocol within 1% for all 5 trades
|
||||
3. No compounding losses (each trade recorded exactly once)
|
||||
4. User confirms dashboard P&L matches expectations
|
||||
|
||||
**Then:** Mark VERIFICATION_CHECKLIST_NOV12.md as ✅ COMPLETE and update copilot-instructions.md
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- **Capital at risk:** $97.55 USDC (monitor closely during verification)
|
||||
- **Current leverage:** 15x SOL, 1x ETH
|
||||
- **Trade frequency:** ~2-5 trades/day expected
|
||||
- **Verification timeline:** Should complete within 1-3 days
|
||||
|
||||
**Last Updated:** November 12, 2025 - Fix deployed, awaiting first test trade
|
||||
Reference in New Issue
Block a user