Implementation of 1-minute data enhancements Phase 2:
- Queue signals when price not at favorable pullback level
- Monitor every 15s for 0.15-0.5% pullback (LONG=dip, SHORT=bounce)
- Validate ADX hasn't dropped >2 points (trend still strong)
- Timeout at 2 minutes → execute at current price
- Expected improvement: 0.2-0.5% per trade = ,600-4,000 over 100 trades
Files:
- lib/trading/smart-entry-timer.ts (616 lines, zero TS errors)
- app/api/trading/execute/route.ts (integrated smart entry check)
- .env (SMART_ENTRY_* configuration, disabled by default)
Next steps:
- Test with SMART_ENTRY_ENABLED=true in development
- Monitor first 5-10 trades for improvement verification
- Enable in production after successful testing
DOCUMENTATION:
- Created 1MIN_DATA_ENHANCEMENTS_ROADMAP.md (comprehensive 7-phase plan)
- Copied to docs/ folder for permanent documentation
- Updated website roadmap API with Phase 7 items
PHASE 7 FOUNDATION ✅ COMPLETE (Nov 27, 2025):
- 1-minute data collection working (verified)
- Revenge system ADX validation deployed
- Market data cache updates every 60 seconds
- Foundation for 6 future enhancements
PLANNED ENHANCEMENTS:
1. Smart Entry Timing (0.2-0.5% better entries)
2. Signal Quality Real-Time Validation (block degraded signals)
3. Stop-Hunt Early Warning System (predictive revenge)
4. Dynamic Position Sizing (ADX momentum-based leverage)
5. Re-Entry Analytics Momentum Filters (trend strength)
6. Dynamic Trailing Stop Optimization (adaptive trail width)
EXPECTED IMPACT:
- Entry improvement: $1,600-4,000 over 100 trades
- Block 5-10% degraded signals
- Revenge success rate: +10-15%
- Runner profitability: +10-20%
- Better risk-adjusted returns across all systems
User requested: "put that on every documentation. it has to go on the websites roadmap as well"
All locations updated ✅
- Updated description: Hostinger hot standby operational since Nov 25
- Clarified impact: App-level HA working (99.9%), DB HA in progress
- Item breakdown now emphasizes OPERATIONAL vs PLANNED:
* ✅ OPERATIONAL: Hostinger hot standby with PostgreSQL replica
* ✅ OPERATIONAL: DNS failover (INWX API, 90s automatic switching)
* ✅ OPERATIONAL: Health monitoring (systemd service)
* ✅ VALIDATED: Live test Nov 25 (0s downtime, auto failback)
* ✅ OPERATIONAL: PostgreSQL streaming replication
* ⏳ WAITING: Oracle Cloud free tier (Patroni upgrade)
* ⏳ PLANNED: 3-node Patroni cluster for true DB HA
- What we HAVE: Hot standby, automatic app failover, PostgreSQL replica
- What we NEED: Patroni for automatic DB leader election
- Changed status from 'complete' to 'in-progress'
- Removed premature 'completed' date (Nov 25 was DNS failover only)
- Updated description: Waiting for Oracle Cloud free tier approval
- Item breakdown:
* ✅ DNS failover working (app-level HA)
* ✅ Health monitoring operational
* ✅ Live test validated (0s downtime)
* ⏳ Oracle Cloud approval pending (database-level HA)
* ⏳ Patroni 3-node cluster planned (true PostgreSQL HA)
* ⏳ Automatic DB failover with Patroni
* ⏳ Distributed consensus with etcd
- Current: App HA working, Database HA in progress
- Updated Phase 6: High Availability Setup status from 'planned' to 'complete'
- Added completed date: November 25, 2025
- Updated description with specific implementation details:
* Primary srvdocker02 + Secondary Hostinger servers
* PostgreSQL streaming replication (<1s lag)
* DNS failover with INWX API
* Health monitoring with 30-second checks
* Live test validated: 0s downtime, automatic failback
* Cost: ~$20-30/month for 99.9% uptime
- Roadmap page will now show HA as completed achievement
- Aligns with homepage achievements banner and master roadmap docs
- Added maGap field to RiskCheckRequest interface
- Added maGap field to ExecuteTradeRequest interface
- Health check already enhanced with database connectivity check
- Fixes TypeScript build errors blocking deployment
Integrated MA gap analysis into signal quality evaluation pipeline:
BACKEND SCORING (lib/trading/signal-quality.ts):
- Added maGap?: number parameter to scoreSignalQuality interface
- Implemented convergence/divergence scoring logic:
* LONG: +15pts tight bullish (0-2%), +12pts converging (-2-0%), +8pts early momentum (-5--2%)
* SHORT: +15pts tight bearish (-2-0%), +12pts converging (0-2%), +8pts early momentum (2-5%)
* Penalties: -5pts for misaligned MA structure (>5% wrong direction)
N8N PARSER (workflows/trading/parse_signal_enhanced.json):
- Added MAGAP:([-\d.]+) regex pattern for negative number support
- Extracts maGap from TradingView v9 alert messages
- Returns maGap in parsed output (backward compatible with v8)
- Updated comment to show v9 format
API ENDPOINTS:
- app/api/trading/check-risk/route.ts: Pass maGap to scoreSignalQuality (2 calls)
- app/api/trading/execute/route.ts: Pass maGap to scoreSignalQuality (2 calls)
FULL PIPELINE NOW COMPLETE:
1. TradingView v9 → Generates signal with MAGAP field
2. n8n webhook → Extracts maGap from alert message
3. Backend scoring → Evaluates MA gap convergence (+8 to +15 pts)
4. Quality threshold → Borderline signals (75-85) can reach 91+
5. Execute decision → Only signals scoring ≥91 are executed
MOTIVATION:
Helps borderline quality signals reach execution threshold without overriding
safety rules. Addresses Nov 25 missed opportunity where good signal had MA
convergence but borderline quality score.
TESTING REQUIRED:
- Verify n8n parses MAGAP correctly from v9 alerts
- Confirm backend receives maGap parameter
- Validate MA gap scoring applied to quality calculation
- Monitor first 10-20 v9 signals for scoring accuracy
- Created LONG_ADAPTIVE_LEVERAGE_VERIFICATION.md with complete verification
- Logic testing confirms Q95+ = 15x, Q90-94 = 10x (100% correct)
- Updated test endpoint to pass direction parameter (best practice)
- Backward compatibility verified (works with or without direction)
- No regressions from SHORT implementation
- Awaiting first production LONG trade for final validation
- Added 'Completed Tasks' collapsible block with count badge
- Completed section collapsed by default (prevents scrolling)
- In-progress and planned tasks always visible at top
- Click completed section header to expand/collapse
- Improves navigation to active work
- Added expandedItems state to track which items are expanded
- Auto-expands only non-complete items on load (in-progress, planned)
- Complete items collapsed by default for better overview
- Click anywhere on item header or chevron to toggle
- Smooth transitions and hover effects
- Improves readability when many items are complete
- Moved useEffect hook before return statement (proper React component structure)
- Was causing Docker build failures with 'Unexpected token' error
- useEffect must be inside component function but before JSX return
- Build now completes successfully in 71.8s
- Created PROFIT_PROJECTION_NOV24_2025.md for Feb 2026 accountability
- Built interactive dashboard at /app/projection/page.tsx
- Live Drift API integration for current capital
- 12-week projection with status indicators (🚀✅⚠️📅)
- Discovery cards showing +98 vs -53 quality 90 shorts
- System fixes documentation
- Weekly tracking table with milestone highlights
- Added projection card to homepage (yellow/orange gradient, 🚀 icon)
- Projection page includes back to home button
- Container rebuilt and deployed successfully
User can now track 01 → 00K journey with real-time comparison of
projected vs actual performance. See you Feb 16, 2026 to verify! 🎯
User Request: Replace blind 2-hour restart timer with smart monitoring that only restarts when accountUnsubscribe errors actually occur
Changes:
. Health Monitor (NEW):
- Created lib/monitoring/drift-health-monitor.ts
- Tracks accountUnsubscribe errors in 30-second sliding window
- Triggers container restart via flag file when 50+ errors detected
- Prevents unnecessary restarts when SDK healthy
. Drift Client:
- Removed blind scheduleReconnection() and 2-hour timer
- Added interceptWebSocketErrors() to catch SDK errors
- Patches console.error to monitor for accountUnsubscribe patterns
- Starts health monitor after successful initialization
- Removed unused reconnect() method and reconnectTimer field
. Health API (NEW):
- GET /api/drift/health - Check current error count and health status
- Returns: healthy boolean, errorCount, threshold, message
- Useful for external monitoring and debugging
Impact:
- System only restarts when actual memory leak detected
- Prevents unnecessary downtime every 2 hours
- More targeted response to SDK issues
- Better operational stability
Files:
- lib/monitoring/drift-health-monitor.ts (NEW - 165 lines)
- lib/drift/client.ts (removed timer, added error interception)
- app/api/drift/health/route.ts (NEW - health check endpoint)
Testing:
- Health monitor starts on initialization: ✅
- API endpoint returns healthy status: ✅
- No blind reconnection scheduled: ✅
Changes:
- Updated roadmap status from 'planned' to 'complete'
- Added checkmarks for implemented features:
✅ Position Manager tracks MFE/MAE every 2 seconds
✅ Database stores maxFavorableExcursion and maxAdverseExcursion
✅ Analytics dashboard displays avg MFE/MAE per indicator version
✅ Version comparison shows MFE/MAE trends
✅ Optimization API analyzes MFE vs TP1 rate
- Added future enhancement note for distribution charts
Evidence:
- Position Manager: lib/trading/position-manager.ts (lines 53-55, 140, 1127+)
- Database: Trade model with MFE/MAE fields
- Analytics: app/analytics/page.tsx (lines 77-78, 566-579, 654-655)
- Optimization API: app/api/optimization/analyze/route.ts
User Request: 'i think we already have this implemented?'
Confirmed: MAE/MFE tracking is fully operational
TypeScript build error: qualityScore not in interface
Fix: Added qualityScore?: number to ExecuteTradeResponse type
Files Modified:
- app/api/trading/execute/route.ts (interface update)
User Request: Show quality score in Telegram when position opened
Changes:
- Updated execute endpoint response to include qualityScore field
- n8n workflow already checks for qualityScore in response
- When present, displays: ⭐ Quality: XX/100
Impact:
- Users now see quality score immediately on position open
- Previously only saw score on blocked signals
- Better visibility into trade quality at entry
Files Modified:
- app/api/trading/execute/route.ts (added qualityScore to response)
User Request: Distinguish between SL and Trailing SL in analytics overview
Changes:
1. Position Manager:
- Updated ExitResult interface to include 'TRAILING_SL' exit reason
- Modified trailing stop exit (line 1457) to use 'TRAILING_SL' instead of 'SL'
- Enhanced external closure detection (line 937) to identify trailing stops
- Updated handleManualClosure to detect trailing SL at price target
2. Database:
- Updated UpdateTradeExitParams interface to accept 'TRAILING_SL'
3. Frontend Analytics:
- Updated last trade display to show 'Trailing SL' with special formatting
- Purple background/border for TRAILING_SL vs blue for regular SL
- Runner emoji (🏃) prefix for trailing stops
Impact:
- Users can now see when trades exit via trailing stop vs regular SL
- Better understanding of runner system performance
- Trailing stops visually distinct in analytics dashboard
Files Modified:
- lib/trading/position-manager.ts (4 locations)
- lib/database/trades.ts (UpdateTradeExitParams interface)
- app/analytics/page.tsx (exit reason display)
- .github/copilot-instructions.md (Common Pitfalls #61, #62)
- Fixed tp1Hit/tp2Hit -> tp1Filled/tp2Filled in Runner Performance query
- Fixed atr -> atrAtEntry in ATR vs MFE Correlation and Data Collection queries
- Added Analytics card to homepage with link to /analytics/optimization
- Added Home button to optimization page header
- All 7 analyses now working without SQL errors
- Created /api/optimization/analyze endpoint with 7 SQL analyses
- Replaced old TP/SL page with comprehensive dashboard
- Analyses: Quality Score Distribution, Direction Performance, Blocked Signals, Runner Performance, ATR vs MFE, Indicator Versions, Data Collection Status
- Real-time refresh capability
- Actionable recommendations based on data thresholds
- Roadmap links at bottom
- Addresses user request for automated SQL analysis dashboard
- Added MIN_SIGNAL_QUALITY_SCORE_LONG and _SHORT fields to Settings interface
- Replaced single quality score field with three fields:
1. Global Fallback (91) - for BTC and other symbols
2. LONG Signals (90) - based on 71.4% WR data analysis
3. SHORT Signals (95) - based on toxic 28.6% WR data, blocks low-quality shorts
- Updated app/api/settings/route.ts GET/POST handlers to support direction-specific fields
- Fixed field naming consistency (MIN_SIGNAL_QUALITY_SCORE vs MIN_QUALITY_SCORE)
- User can now adjust direction-specific thresholds via settings UI without .env editing
- Container deployed: 2025-11-23T14:25:34 UTC
Critical Bug Fix:
- archivedVersions was used before declaration (line 147 vs line 165)
- Caused 'Cannot access before initialization' error
- Moved versionDescriptions and archivedVersions declarations to top
- Now defined BEFORE usage in resultsWithArchived.map()
Impact: Analytics page was completely broken (stuck on loading)
Resolution: API now returns data correctly, UI functional
Error: ReferenceError: Cannot access 'g' before initialization
Fix: Proper variable ordering in route.ts
**BUG:** Telegram 'short sol' blocked by multi-timeframe data collection filter
- Filter checked 'timeframe !== 5' which blocked 'manual' timeframe
- Manual trades from Telegram should execute, not be saved for analysis
**FIX:** Updated condition to 'timeframe !== 5 && timeframe !== manual'
- Allows both 5min TradingView signals AND manual Telegram trades
- Only blocks 15min/1H/4H/Daily for data collection
**FILES:** app/api/trading/execute/route.ts line 114
**DEPLOYED:** Nov 20, 2025 15:42 CET
Stats API was recalculating P&L from entry/exit prices which didn't
account for TP1+runner partial closes. This caused incorrect P&L
display (-$26.10 instead of +$46.97).
Fixed:
- Use database realizedPnL (now corrected to match Drift UI)
- Added debug logging to show trade count and total
- Stats now correctly show v8 performance: +$46.97
Note: Database P&L values were corrected in previous commit (cd6f590)
to match Drift UI's actual TP1+runner close values.
- Query Drift Protocol's cumulativeDeposits for ground truth
- Discovered user deposited ,440.61 (not hardcoded 46)
- Previous withdrawals: ~26.78 (not tracked in ENV)
- Database P&L -.01 is correct
- Reconciliation: ,440.61 deposits - .01 P&L - 26.78 withdrawn = 11.82 current
- Available profit now shows current balance (includes all trading results)
- Created /api/drift/account-summary endpoint for account reconciliation
- Statistics now mathematically consistent and trustworthy
- Fixed LAST_WITHDRAWAL_TIME type (null | string)
- Removed parseFloat on health.freeCollateral (already number)
- Fixed getDriftClient() → getClient() method name
- Build now compiles successfully
Deployed: Withdrawal system now live on dashboard
Implemented comprehensive price tracking for multi-timeframe signal analysis.
**Components Added:**
- lib/analysis/blocked-signal-tracker.ts - Background job tracking prices
- app/api/analytics/signal-tracking/route.ts - Status/metrics endpoint
**Features:**
- Automatic price tracking at 1min, 5min, 15min, 30min intervals
- TP1/TP2/SL hit detection using ATR-based targets
- Max favorable/adverse excursion tracking (MFE/MAE)
- Analysis completion after 30 minutes
- Background job runs every 5 minutes
- Entry price captured from signal time
**Database Changes:**
- Added entryPrice field to BlockedSignal (for price tracking baseline)
- Added maxFavorablePrice, maxAdversePrice fields
- Added maxFavorableExcursion, maxAdverseExcursion fields
**Integration:**
- Auto-starts on container startup
- Tracks all DATA_COLLECTION_ONLY signals
- Uses same TP/SL calculation as live trades (ATR-based)
- Calculates profit % based on direction (long vs short)
**API Endpoints:**
- GET /api/analytics/signal-tracking - View tracking status and metrics
- POST /api/analytics/signal-tracking - Manually trigger update (auth required)
**Purpose:**
Enables data-driven multi-timeframe comparison. After 50+ signals per
timeframe, can analyze which timeframe (5min vs 15min vs 1H vs 4H vs Daily)
has best win rate, profit potential, and signal quality.
**What It Tracks:**
- Price at 1min, 5min, 15min, 30min after signal
- Would TP1/TP2/SL have been hit?
- Maximum profit/loss during 30min window
- Complete analysis of signal profitability
**How It Works:**
1. Signal comes in (15min, 1H, 4H, Daily) → saved to BlockedSignal
2. Background job runs every 5min
3. Queries current price from Pyth
4. Calculates profit % from entry
5. Checks if TP/SL thresholds crossed
6. Updates MFE/MAE if new highs/lows
7. After 30min, marks analysisComplete=true
**Future Analysis:**
After 50+ signals per timeframe:
- Compare TP1 hit rates across timeframes
- Identify which timeframe has highest win rate
- Determine optimal signal frequency vs quality trade-off
- Switch production to best-performing timeframe
User requested: "i want all the bells and whistles. lets make the
powerhouse more powerfull. i cant see any reason why we shouldnt"
Settings UI was using wrong variable name (MIN_QUALITY_SCORE) while
code reads MIN_SIGNAL_QUALITY_SCORE. This caused quality score changes
in settings UI to have no effect.
Fixed:
- Settings API now reads/writes MIN_SIGNAL_QUALITY_SCORE
- Updated .env file to use correct variable name
- User's quality score increase to 81 will now work
Related: User increased min quality from 60 to 81 to filter out
small chop trades (avoiding -$99 trade with quality score 80).
- Changed 'Avg MFE: +X.XX%' to 'Avg MFE: +$X.XX'
- Changed 'Avg MAE: -X.XX%' to 'Avg MAE: -$X.XX'
- Database already stores dollar values (fixed Nov 19)
- UI was incorrectly displaying dollars with % suffix
- V8 trades: Avg MFE $34.23, Avg MAE -$11.06 (correct)
- Part of MAE/MFE data corruption fix series
Also corrected existing v8 trade in database:
- Before: MFE 19.73% (account %), MAE -1.53% (account %)
- After: MFE $95.11, MAE -$7.39 (actual dollars)
- SQL: UPDATE based on maxFavorablePrice/maxAdversePrice
- TypeScript build error: currentPrice not in interface
- Correct field name is signalPrice (already defined)
- Fixes multi-timeframe data collection compilation
- Only 5min signals execute trades (production)
- 15min/1H/4H/Daily signals saved to BlockedSignal table for analysis
- Enables cross-timeframe performance comparison
- Zero financial risk - non-5min signals just collect data
- blockReason: 'DATA_COLLECTION_ONLY' for easy filtering
- Returns HTTP 200 (not 400) since this is expected behavior
- Prepares for future timeframe optimization decisions
- Moved positionManager.addTrade() to AFTER database save succeeds
- Changed database error handling to return HTTP 500 (not silent fail)
- Test endpoint now enforces same pattern as execute endpoint
- Prevents untracked positions when database save fails
- Root cause of trade manual-1763391075992 compounding to -19.43
Before: Test endpoint added to Position Manager first, saved to DB after
After: Test endpoint saves to DB first, only adds to PM if DB succeeds
Impact: No more untracked positions from test trades with failed DB saves
- Created /api/trading/place-exit-orders endpoint
- Created restore-orders.mjs script
- Issue: Next.js creates separate Drift instances per route
- Workaround: Use /api/trading/cancel-orders to remove orphaned orders
Current situation:
- 32 orphaned orders existed and were cancelled
- Position Manager should auto-place new orders
- Manual order placement endpoint needs refactoring
