- Added STEP 1: Run tests BEFORE deployment (113 tests in ~30s)
- Added STEP 2: Validate with test trade AFTER deployment
- Why mandatory: Catch bugs (tokens vs USD, false TP1, wrong SL) before real money loss
- Tests prevent Common Pitfalls #24, #43, #45, #52, #54, #67 recurrence
- DO NOT deploy if tests fail - fix issue or update tests first
Added comprehensive feature discovery section to copilot-instructions.md:
- Quick Reference Table: 9 common scenarios with existing features
- Quick Search Commands: bash commands for feature discovery
- Feature Discovery by Category: 6 categories with 30+ features
- Decision Flowchart: 5-step verification process
- Historical Examples: why each feature was built
This helps users/AI agents discover existing features before rebuilding.
Co-authored-by: mindesbunister <32161838+mindesbunister@users.noreply.github.com>
- Only 31 records from multi-timeframe alerts (not 11,429)
- 11,398 records are 1-minute data (kept as DATA_COLLECTION_ONLY)
- Total marked as OLD_V9_VERSION: 31 (15min/1H/4H/Daily only)
- Discovery: All TradingView alerts (5min/15min/1H/4H/Daily) attached to OLD v9 version
- Impact: 11,429 records from wrong indicator settings (confirmBars=0 vs current)
- Solution: Marked as DATA_COLLECTION_OLD_V9_VERSION to prevent analysis contamination
- Exception: 1-minute data (11,398) kept as DATA_COLLECTION_ONLY (unaffected)
- Fresh data from corrected alerts will use DATA_COLLECTION_ONLY going forward
- Old data preserved for historical reference, clearly marked
CRITICAL LESSON LEARNED (Dec 5, 2025):
Document the data analysis disaster caused by MFE/MAE stored in mixed units.
What Happened:
- Analyzed blocked vs executed signals to improve win rate
- SQL showed executed signals: 20.15% avg MFE (appeared excellent)
- Implemented "optimizations" based on this data:
* Tighter ATR multipliers (2.0→1.5, 4.0→3.0)
* Higher TP1 close (60%→75%)
* Increased leverage (1×→5×)
- User questioned: Why doesn't TP1 hit if it's 20% MFE?
- Investigation: Only 2/11 trades reached TP1 price target
- Root cause: Old records stored MFE in DOLLARS, new in PERCENTAGES
- TRUE MFE: 0.76% (long), 1.20% (short) - NOT 20%!
- 26× inflation due to unit mismatch
Why This Matters:
- This is a REAL MONEY system - wrong analysis = wrong trades = losses
- MFE/MAE used for critical decisions (exit timing, quality validation)
- Agent made "data-driven" optimizations on 26× inflated data
- All changes had to be reverted (commits a67a338, f65aae5)
MANDATORY SQL Pattern:
- ALWAYS filter by createdAt >= '2025-11-23' for MFE/MAE queries
- OR recalculate from prices (maxFavorablePrice - entryPrice)
- NEVER trust raw AVG(maxFavorableExcursion) without date filter
Prevention:
- Verify stored vs calculated values before ANY MFE/MAE analysis
- Check sample of recent vs old records to detect unit changes
- Document data format changes in Common Pitfalls immediately
Related:
- Common Pitfall #54: Original MFE/MAE units bug (Nov 23, 2025)
- Revert commit: a15f17f
- Incorrect optimization: a67a338, f65aae5
CRITICAL DATA BUG DISCOVERED (Dec 5, 2025):
Previous commits a67a338 and f65aae5 implemented optimizations based on
INCORRECT analysis of maxFavorableExcursion (MFE) data.
Problem: Old Trade records stored MFE in DOLLARS, not PERCENTAGES
- Appeared to show 20%+ average favorable movement
- Actually only 0.76% (long) and 1.20% (short) average movement
- 26× inflation of perceived performance due to unit mismatch
Incorrect Changes Reverted:
- ATR_MULTIPLIER_TP1: 1.5 → back to 2.0
- ATR_MULTIPLIER_TP2: 3.0 → back to 4.0
- ATR_MULTIPLIER_SL: 2.5 → back to 3.0
- TAKE_PROFIT_1_SIZE_PERCENT: 75 → back to 60
- LEVERAGE: 5 → back to 1
- Safety bounds restored to original values
- TRAILING_STOP_ATR_MULTIPLIER: back to 2.5
REAL FINDINGS (after data correction):
- TP1 orders ARE being placed (tp1OrderTx populated)
- TP1 prices NOT being reached (only 2/11 trades in sample)
- Recent trades (6 total): avg MFE 0.74%, only 2/6 reached TP1
- Problem is ENTRY QUALITY, not exit timing
- Quality 90+ signals barely move favorably before reversing
See Common Pitfall #54 - MFE data stored in mixed units
Need to filter by createdAt >= '2025-11-23' for accurate analysis
Updated copilot-instructions.md with:
- New ATR multipliers (1.5×/3.0×/2.5× vs 2.0×/4.0×/3.0×)
- Rationale: 0% TP hit rate despite 17-24% avg MFE
- Problem: Targets hit then reversed before monitoring loop detected
- Solution: Tighter targets catch moves before reversal
- 75% close at TP1 (vs 60%) to bank profit immediately
- 25% runner (vs 40%) for extended trends with tighter trail
- Leverage 5× during testing phase
This is MANDATORY documentation update per #1 priority rule.
- Added DUAL REMOTE SETUP section with origin (Gitea) and github (GitHub) configuration
- Documented post-commit hook location and purpose (.git/hooks/post-commit)
- Explained auto-sync to github vs manual push to origin workflow
- Added verification commands for sync status checking
- Included automation setup details and testing confirmation
- Updated commit workflow to reflect hook behavior
- Added recent example (de77cfe test commit) to demonstrate automation
- Created docs/COMMON_PITFALLS.md with all 72 pitfalls
- Organized by severity and category for better navigation
- Added quick reference table and cross-reference index
- Reduced copilot-instructions.md from 6,575 to 3,608 lines (45%)
- Kept Top 10 critical pitfalls in main instructions
- Preserved all git commits, dates, code examples
- Updated docs/README.md with references to new doc
Benefits:
- Faster AI agent context loading
- Easier maintenance and updates
- Better searchability by category
- Clear pattern recognition for similar issues
- Maintains comprehensive knowledge base
Co-authored-by: mindesbunister <32161838+mindesbunister@users.noreply.github.com>
User mandate: Manual Telegram trades bypass quality scoring entirely.
Documentation updates:
- Added 'Manual Trade Quality Bypass' section
- Explains user requirement for instant execution
- Documents implementation details (timeframe='manual' detection)
- Clarifies that analytics check is now advisory only
- Notes --force flag no longer needed for manual trades
Context: This is part of the mandatory documentation workflow -
every code change requires corresponding documentation update.
Related commit: 0982578 (quality bypass implementation)
Date: Dec 4, 2025
- New IRON-CLAD RULE: Always search docs before making suggestions or asking questions
- Purpose: Prevent wasting user time with already-answered questions
- Examples: TradingView rate limits, roadmap features, known bugs, configuration
- Workflow: Read request → Search docs → Check if answered → THEN respond
- Applies to: Features, bugs, config, architecture, deployment, troubleshooting
- Red flags: User says 'we already documented this' or 'check docs first'
- Why: User spent months documenting comprehensively, 'NOTHING gets lost' principle
- Impact: Respect user's documentation effort, save time = save money in financial system
Files modified:
- .github/copilot-instructions.md (line ~103-150, added Rule #5 with examples and workflow)
- Added explicit onboarding workflow at top of file
- 4-step sequence: copilot-instructions → docs/README → README → explore
- Lists all 8 documentation subdirectories with descriptions
- Emphasizes 'NOTHING gets lost' principle
- Ensures new agents have clear entry point without manual explanation
- Pitfall #68: Smart Entry using webhook percentage as signal price
* Root cause: TradingView webhook price field contained percentage (70.80) instead of market price (42.50)
* Impact: 97% pullback calculations made Smart Entry impossible to trigger
* Fix: Use Pyth current price instead of webhook price
* Commit: 7d0d38a
- Pitfall #69: Direction-specific leverage thresholds not explicit
* Made LONG/SHORT leverage assignment explicit even though values same
* Improves code clarity and maintainability
* Commit: 58f812f
Both fixes deployed Dec 3, 2025, 09:02:45 CET (timestamp verified)
Bug: 23 duplicate Telegram notifications with P&L compounding (-7.96 to -,129.24)
Cause: Multiple monitoring loops passed has() check before any deleted from Map
Fix: Use Map.delete() atomic return value as deduplication lock
Result: First caller deletes and proceeds, subsequent callers return immediately
Related: #48-49 (TP1 P&L compound), #59-61 (external closure duplicates)
Deployed: Dec 2, 2025 17:32:52 UTC (commit 93dd950)
Bug: Multiple monitoring loops detect ghost simultaneously
- Loop 1: has(tradeId) → true → proceeds
- Loop 2: has(tradeId) → true → ALSO proceeds (race condition)
- Both send Telegram notifications with compounding P&L
Real incident (Dec 2, 2025):
- Manual SHORT at $138.84
- 23 duplicate notifications
- P&L compounded: -$47.96 → -$1,129.24 (23× accumulation)
- Database shows single trade with final compounded value
Fix: Map.delete() returns true if key existed, false if already removed
- Call delete() FIRST
- Check return value
proceeds
- All other loops get false → skip immediately
- Atomic operation prevents race condition
Pattern: This is variant of Common Pitfalls #48, #49, #59, #60, #61
- All had "check then delete" pattern
- All vulnerable to async timing issues
- Solution: "delete then check" pattern
- Map.delete() is synchronous and atomic
Files changed:
- lib/trading/position-manager.ts lines 390-410
Related: DUPLICATE PREVENTED message was working but too late
CRITICAL UPDATES to AI assistant instructions:
1. MANDATORY GIT WORKFLOW (DO NOT SKIP):
- Added explicit requirement: implement → test → verify → document → commit → push
- Made git commits NON-OPTIONAL for all significant changes
- Added to both general prompt and copilot-instructions.md
- Rationale: Agent has pattern of skipping documentation/commits
2. CHALLENGE USER IDEAS:
- Added requirement to think critically about user requests
- Instruction: "Think freely and don't hold back"
- Goal: Find BEST solution, not just A solution
- Push back on ideas that don't make sense
- Ask "is there a simpler/faster/safer way?"
3. COMPREHENSIVE DOCUMENTATION SECTION:
- Replaced brief documentation note with full workflow guide
- Added 80+ lines of detailed documentation requirements
- Includes examples, red flags, mindset principles
- Emphasizes: "Git commit + Documentation = Complete work"
Files modified:
- .github/prompts/general prompt.prompt.md (added sections 5a, 6, updated 7-8)
- .github/copilot-instructions.md (comprehensive documentation workflow)
User mandate: "I am sick and tired of reminding you" - this makes it automatic.
Impact: Future implementations will ALWAYS include documentation and git commits as part of standard workflow, not as afterthoughts.
- ProxyJump (-J) doesn't work from Docker container
- Changed to nested SSH: hop -> target
- Proper command escaping for nested SSH
- Worker2 (srv-bd-host01) only accessible via worker1 (pve-nu-monitor01)
- Created /api/cluster/logs endpoint to read coordinator.log
- Added real-time log display in cluster UI (updates every 3s)
- Shows last 100 lines of coordinator.log in terminal-style display
- Includes manual refresh button
- Improves debugging experience - no need to SSH for logs
User feedback: 'why dont we add the output of the log at the bottom of the page so i know whats going on'
This addresses poor visibility into coordinator errors and failures.
Next step: Fix SSH timeout issue blocking worker execution.
USER MANDATE (Dec 1, 2025): Documentation MUST go hand-in-hand with EVERY git commit.
This is NOT optional. This is NOT a suggestion. This is MANDATORY.
Changes:
- Elevated documentation section to #1 PRIORITY status
- Added user's direct quote: 'this HAS to go hand in hand'
- Expanded from 15 lines to 100+ lines with comprehensive guidelines
- Added 'Why This is #1 Priority' section with user's frustration quote
- Added explicit 'When Documentation is MANDATORY' checklist
- Added 'The Correct Mindset' section emphasizing it's part of the work
- Added 4 scenario examples showing what MUST be documented
- Added 'Red Flags' section to catch missing documentation
- Added 'Integration with Existing Sections' guide
- Made it crystal clear: Code without documentation = INCOMPLETE WORK
This addresses user's repeated reminders about documentation being mandatory.
Future AI agents will now see this as the #1 priority it is.
NO MORE PUSHING CODE WITHOUT DOCUMENTATION UPDATES.
- Document database-first architecture pattern
- Include problem, root cause, and solution details
- Add verification methodology with before/after examples
- Document cluster control system (Start/Stop buttons)
- Include database schema and operational state
- Add lessons learned about infrastructure vs business logic
- Reference STATUS_DETECTION_FIX_COMPLETE.md for full details
- Current state: 2 workers active, processing 4000 combinations
- Removed v10 TradingView indicator (moneyline_v10_momentum_dots.pinescript)
- Removed v10 penalty system from signal-quality.ts (-30/-25 point penalties)
- Removed backtest result files (sweep_*.csv)
- Updated copilot-instructions.md to remove v10 references
- Simplified direction-specific quality thresholds (LONG 90+, SHORT 80+)
Rationale:
- 1,944 parameter combinations tested in backtest
- All top results IDENTICAL (568 trades, $498 P&L, 61.09% WR)
- Momentum parameters had ZERO impact on trade selection
- Profit factor 1.027 too low (barely profitable after fees)
- Max drawdown -$1,270 vs +$498 profit = terrible risk-reward
- v10 penalties were blocking good trades (bug: applied to wrong positions)
Keeping v9 as production system - simpler, proven, effective.
- Bug: Execute endpoint calculated quality but never validated it
- Three trades executed at quality 30/50/50 (threshold: 90/95)
- All three stopped out, confirming low quality = losing trades
- Root cause: TradingView sent incomplete data (metrics=0, old v5) + missing validation after timeframe check
- Fix: Added validation block lines 193-213 in execute/route.ts
- Returns HTTP 400 if quality < minQualityScore
- Deployed: Nov 27, 2025 23:16 UTC (commit cefa3e6)
- Lesson: Calculate ≠ Validate - minQualityScore must be enforced at ALL execution pathways
This documents the CRITICAL FIX from commit cefa3e6.
Per Nov 27 mandatory documentation rules, work is INCOMPLETE without copilot-instructions.md updates.
CRITICAL: Added iron-clad rule that copilot-instructions.md MUST be updated
for every significant change. User is 'sick and tired' of reminding.
New mandatory section explains:
- When to update this file (8 specific scenarios)
- Why it's the primary knowledge base for future developers
- Automatic workflow: Change → Code → Test → Update Docs → Commit
1-Minute Data Collection documented:
- Direction field is meaningless (TradingView artifact)
- Analysis should ignore direction for timeframe='1'
- Focus on ADX/ATR/RSI/volume/price position metrics
- Example correct vs wrong SQL queries
This is NON-NEGOTIABLE going forward.
- Position Manager section: Complete Phase 7.3 documentation with real-time ADX queries
- Documented adaptive multiplier logic: acceleration bonus, deceleration penalty, combined 3.16× max
- Added example calculation showing 2.15× wider trail vs old static system
- When Making Changes section: Added Phase 7.3 verification steps and log monitoring
- Trailing stop changes: Updated with new adaptive system details and testing procedures
- References: PHASE_7.3_ADAPTIVE_TRAILING_DEPLOYED.md and 1MIN_DATA_ENHANCEMENTS_ROADMAP.md
PROBLEM: Rebuilding container 4-6 times per session when most changes don't need it
- Every rebuild: 40-70 seconds downtime
- Recent session: 200 seconds downtime that could've been 50 seconds
- Rebuilding for documentation (should be git only)
- Rebuilding for n8n workflows (should be manual import)
- Rebuilding for ENV changes (should be restart only)
SOLUTION: Created comprehensive guide on what actually needs rebuilds
ZERO DOWNTIME (just commit):
- Documentation (.md files)
- Workflows (.json, .pinescript)
- Hot-reload endpoints (roadmap reload)
RESTART ONLY (5-10 seconds):
- ENV variable changes (.env)
- Database schema (prisma migrate + generate)
REBUILD REQUIRED (40-70 seconds):
- Code changes (.ts, .tsx, .js)
- Dependencies (package.json)
- Dockerfile changes
SMART BATCHING:
- Group multiple code changes into ONE rebuild
- Example: 6 fixes → 1 rebuild = 50s total (not 6× rebuilds = 300s)
CREATED FILES:
- docs/ZERO_DOWNTIME_CHANGES.md (comprehensive guide with examples)
- Updated copilot-instructions.md (quick decision matrix)
EXPECTED IMPACT:
- 60-80% reduction in rebuild frequency
- 60-80% reduction in downtime per session
- Better workflow: batch changes, test together, deploy once
User was right: We were rebuilding WAY too often unnecessarily ✅
Updated Indicator Version Tracking section:
- Changed v8 from PRODUCTION to ARCHIVED (Nov 18-26)
- Added v9 as new PRODUCTION SYSTEM (Nov 26+)
- Documented v9 momentum-based SHORT filter (ADX + Price Position)
- Removed RSI filter rationale (RSI 50+ has best 68.2% WR)
- Added data evidence from 95 SHORT trade analysis
- Documented first day results: 2 losses, both blocked by momentum filter
Updated Stop Hunt Revenge System section:
- Added 'Revenge Timing Enhancement - 90s Confirmation' subsection
- Documented Nov 26 retest problem (would stop at $137.50 before $144.50 move)
- Explained Option 2 approach (90s = 1.5 minutes confirmation)
- Added implementation code snippets from stop-hunt-tracker.ts
- User insight: ATR not suitable (measures volatility, not S/R)
- Status: DEPLOYED Nov 26, 20:52:55 CET, VERIFIED
Related commits:
- 2017cba: v9 SHORT quality improvements - momentum-based filtering
- 40ddac5: Revenge timing Option 2 - 90s confirmation (DEPLOYED)
- Multi-Timeframe section: Added Nov 26 implementation note
- Quality scoring now calculated for ALL timeframes (not just 5min)
- Data collection signals get real quality scores (not hardcoded 0)
- BlockedSignal records include full quality metadata
- Enables SQL: WHERE signalQualityScore >= minScoreRequired
- Execute Trade workflow: Added timeframe routing logic
- When Making Changes: Added item #19 for multi-timeframe updates
- Reflects implementation in commit dbada47
