Files
trading_bot_v4/docs/deployments/SMART_ENTRY_TIMING_STATUS.md
mindesbunister 4c36fa2bc3 docs: Major documentation reorganization + ENV variable reference
**Documentation Structure:**
- Created docs/ subdirectory organization (analysis/, architecture/, bugs/,
  cluster/, deployments/, roadmaps/, setup/, archived/)
- Moved 68 root markdown files to appropriate categories
- Root directory now clean (only README.md remains)
- Total: 83 markdown files now organized by purpose

**New Content:**
- Added comprehensive Environment Variable Reference to copilot-instructions.md
- 100+ ENV variables documented with types, defaults, purpose, notes
- Organized by category: Required (Drift/RPC/Pyth), Trading Config (quality/
  leverage/sizing), ATR System, Runner System, Risk Limits, Notifications, etc.
- Includes usage examples (correct vs wrong patterns)

**File Distribution:**
- docs/analysis/ - Performance analyses, blocked signals, profit projections
- docs/architecture/ - Adaptive leverage, ATR trailing, indicator tracking
- docs/bugs/ - CRITICAL_*.md, FIXES_*.md bug reports (7 files)
- docs/cluster/ - EPYC setup, distributed computing docs (3 files)
- docs/deployments/ - *_COMPLETE.md, DEPLOYMENT_*.md status (12 files)
- docs/roadmaps/ - All *ROADMAP*.md strategic planning files (7 files)
- docs/setup/ - TradingView guides, signal quality, n8n setup (8 files)
- docs/archived/2025_pre_nov/ - Obsolete verification checklist (1 file)

**Key Improvements:**
- ENV variable reference: Single source of truth for all configuration
- Common Pitfalls #68-71: Already complete, verified during audit
- Better findability: Category-based navigation vs 68 files in root
- Preserves history: All files git mv (rename), not copy/delete
- Zero broken functionality: Only documentation moved, no code changes

**Verification:**
- 83 markdown files now in docs/ subdirectories
- Root directory cleaned: 68 files → 0 files (except README.md)
- Git history preserved for all moved files
- Container running: trading-bot-v4 (no restart needed)

**Next Steps:**
- Create README.md files in each docs subdirectory
- Add navigation index
- Update main README.md with new structure
- Consolidate duplicate deployment docs
- Archive truly obsolete files (old SQL backups)

See: docs/analysis/CLEANUP_PLAN.md for complete reorganization strategy
2025-12-04 08:29:59 +01:00

12 KiB
Raw Blame History

Smart Entry Timing - Implementation Status

PHASE 2 IMPLEMENTATION COMPLETE

Date: November 26, 2025
Status: Code complete, TypeScript compilation clean (0 errors)
Expected Value: $1,600-4,000 improvement over 100 trades (0.2-0.5% per trade)


Implementation Summary

Core Service: lib/trading/smart-entry-timer.ts (616 lines)

Architecture:

  • Singleton pattern via getSmartEntryTimer() getter
  • Queue-based signal management (Map of QueuedSignal objects)
  • Monitoring loop runs every 15 seconds when queue active
  • Automatic cleanup of expired/executed signals

Key Features:

  1. Queue Management

    • queueSignal(signalData) - Adds signal to queue with pullback targets
    • startMonitoring() - Begins 15s interval checks
    • stopMonitoring() - Stops when queue empty
    • getQueueStatus() - Debug/monitoring endpoint
  2. Smart Entry Logic

    • LONG: Wait for 0.15-0.5% dip below signal price
    • SHORT: Wait for 0.15-0.5% bounce above signal price
    • ADX validation: Trend strength hasn't degraded >2 points
    • Timeout: 2 minutes → execute at current price regardless
  3. Execution Flow

    • Gets fresh market data from cache (1-min updates)
    • Gets real-time price from Pyth oracle
    • Calculates pullback magnitude
    • Validates ADX via fresh TradingView data
    • Opens position via Drift SDK
    • Places ATR-based exit orders (TP1/TP2/SL)
    • Saves to database with smart entry metadata
    • Adds to Position Manager for monitoring
  4. Configuration (.env variables)

    SMART_ENTRY_ENABLED=false           # Disabled by default
    SMART_ENTRY_MAX_WAIT_MS=120000      # 2 minutes
    SMART_ENTRY_PULLBACK_MIN=0.15       # 0.15% minimum
    SMART_ENTRY_PULLBACK_MAX=0.50       # 0.50% maximum
    SMART_ENTRY_ADX_TOLERANCE=2         # 2 points max drop
    

Integration: app/api/trading/execute/route.ts

Smart Entry Decision Tree (lines 422-478):

Signal arrives → Check if smart entry enabled
  ↓ NO: Execute immediately (existing flow)
  ↓ YES: Get current price from Pyth
    ↓ Calculate pullback from signal price
    ↓ Already at favorable level? (0.15-0.5% pullback achieved)
      ↓ YES: Execute immediately
      ↓ NO: Queue signal for monitoring
        ↓ Return HTTP 200 to n8n (workflow continues)
        ↓ Background monitoring every 15s
        ↓ Execute when:
          - Pullback target hit + ADX valid
          - OR timeout (2 minutes)

Key Behaviors:

  • Preserves existing immediate execution when smart entry disabled
  • Returns success to n8n even when queued (workflow completes)
  • No blocking waits - fully asynchronous monitoring
  • Works with both 5-minute signals (production) and multi-timeframe data collection

Database Tracking

Smart Entry Metadata (saved in configSnapshot.smartEntry):

{
  used: boolean,              // Was smart entry used?
  improvement: number,        // % improvement (positive = better entry)
  waitTime: number,           // Seconds waited before execution
  reason: string,             // 'pullback_confirmed' | 'timeout' | 'manual_override'
  checksPerformed: number     // How many 15s checks ran
}

Purpose: Enable post-trade analysis to measure actual improvement vs immediate entry.


Testing Plan

Phase 1: TypeScript Compilation

  • Zero TypeScript errors
  • All interfaces correctly matched
  • Dependencies properly imported
  • Git committed and pushed

Phase 2: Development Testing (TODO)

  1. Enable smart entry:

    echo "SMART_ENTRY_ENABLED=true" >> .env
    docker restart trading-bot-v4
    
  2. Send test signal via n8n or manual API:

    curl -X POST http://localhost:3001/api/trading/execute \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $API_SECRET_KEY" \
      -d '{
        "symbol": "SOL-PERP",
        "direction": "long",
        "signalPrice": 142.50,
        "atr": 0.43,
        "adx": 26,
        "rsi": 58,
        "volumeRatio": 1.2,
        "pricePosition": 45,
        "qualityScore": 95
      }'
    
  3. Verify logs:

    docker logs -f trading-bot-v4 | grep "Smart Entry"
    

    Expected log sequence:

    • 📥 Smart Entry: Queued signal SOL-PERP-{timestamp}
    • 🔍 Smart Entry: Checking 1 queued signals...
    • ✅ Smart Entry: Pullback confirmed! (if price dipped)
    • OR ⏰ Smart Entry: Timeout - executing at current price (after 2 min)
  4. Test scenarios:

    • Signal arrives when price already at favorable level → immediate execution
    • Signal arrives when price unfavorable → queued → pullback detected → execution
    • Signal arrives when price unfavorable → queued → timeout → execution at current
    • ADX degrades >2 points during wait → signal cancelled

Phase 3: Production Deployment (TODO)

  1. Docker build:

    cd /home/icke/traderv4
    docker compose build trading-bot
    docker compose up -d --force-recreate trading-bot
    
  2. Verify container timestamp:

    docker logs trading-bot-v4 | grep "Server starting" | head -1
    # Must be AFTER commit timestamp: a8c1b2c (Nov 26, 2025)
    
  3. Monitor first 5-10 signals:

    • Watch for "Smart Entry" logs
    • Verify queuing behavior
    • Confirm execution timing (pullback vs timeout)
    • Check database configSnapshot.smartEntry fields
  4. Compare entry prices:

    • Query last 20 trades: 10 with smart entry ON, 10 with smart entry OFF
    • Calculate average entry improvement
    • Expected: 0.2-0.5% better entries with smart entry

Phase 4: Performance Analysis (TODO - After 50+ trades)

-- Compare smart entry vs immediate entry performance
SELECT 
  CASE 
    WHEN "configSnapshot"::jsonb->'smartEntry'->>'used' = 'true' 
    THEN 'Smart Entry' 
    ELSE 'Immediate Entry' 
  END as entry_type,
  COUNT(*) as trades,
  ROUND(AVG("realizedPnL")::numeric, 2) as avg_pnl,
  ROUND(100.0 * SUM(CASE WHEN "realizedPnL" > 0 THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate,
  ROUND(AVG(("configSnapshot"::jsonb->'smartEntry'->>'improvement')::float), 3) as avg_improvement
FROM "Trade"
WHERE "exitReason" IS NOT NULL
  AND "createdAt" > NOW() - INTERVAL '30 days'
GROUP BY entry_type;

Expected Results:

  • Smart Entry avg_improvement: +0.2% to +0.5%
  • Smart Entry win_rate: 2-3% higher than immediate (due to better entries)
  • Smart Entry avg_pnl: $16-40 more per trade

Configuration Tuning

Pullback Range

Current: 0.15-0.5%

  • Too narrow: Misses opportunities, high timeout rate
  • Too wide: Risks reversal, delays entry
  • Optimal: Market-dependent, analyze timeout vs pullback hit rate

Wait Time

Current: 2 minutes (120,000ms)

  • Too short: Misses pullbacks that take longer
  • Too long: Delays entry, risks missed moves
  • Optimal: 90-180 seconds based on 5min candle timing

ADX Tolerance

Current: 2 points

  • Too strict: High cancellation rate, misses valid entries
  • Too loose: Enters weak trends
  • Optimal: 2-3 points based on ADX volatility during pullbacks

Tuning Process:

  1. Collect 50+ smart entry trades
  2. Analyze:
    • Timeout rate vs pullback hit rate
    • Cancelled signals (ADX degraded) - were they correct cancellations?
    • Entry improvement distribution (0.15%, 0.30%, 0.50%)
  3. Adjust parameters based on data
  4. Re-test for 50 more trades
  5. Compare performance

Monitoring & Debugging

Queue Status Endpoint

const smartEntryTimer = getSmartEntryTimer()
const queueStatus = smartEntryTimer.getQueueStatus()
console.log('Queued signals:', queueStatus)

Key Log Messages

  • 💡 Smart Entry Timer initialized: {enabled, maxWait, pullback, adxTolerance}
  • 📥 Smart Entry: Queued signal {id} - Signal added to queue
  • 🔍 Smart Entry: Checking {count} queued signals... - Monitoring loop running
  • ✅ Smart Entry: Pullback confirmed! {direction} {symbol} - Optimal entry detected
  • ⏰ Smart Entry: Timeout - executing at current price - 2min timeout reached
  • ❌ Smart Entry: ADX degraded from {start} to {current} - Signal cancelled
  • 💰 Smart Entry: Improvement: {percent}% - Entry vs signal price comparison

Common Issues

Issue: Signals timeout frequently (>50% timeout rate)

  • Cause: Pullback targets too tight for market volatility
  • Solution: Widen SMART_ENTRY_PULLBACK_MAX from 0.50% to 0.75%

Issue: Signals cancelled due to ADX degradation

  • Cause: ADX tolerance too strict for natural fluctuations
  • Solution: Increase SMART_ENTRY_ADX_TOLERANCE from 2 to 3

Issue: Smart entry improves price but trades still lose

  • Cause: Entry improvement doesn't fix bad signal quality
  • Solution: Focus on improving signal quality thresholds first
  • Note: Smart entry optimizes entry on GOOD signals, doesn't fix BAD signals

Issue: Monitoring loop not running (no "Checking" logs)

  • Cause: Queue empty or monitoring interval not started
  • Solution: Check queueSignal() was called, verify enabled=true

Success Criteria

Phase 2 Complete

  • Zero TypeScript compilation errors
  • Smart entry service implemented (616 lines)
  • Execute endpoint integrated
  • Configuration variables added to .env
  • Git committed and pushed
  • Ready for testing

Phase 3 Success (Development Testing)

  • Smart entry queues signals correctly
  • Monitoring loop detects pullbacks
  • Timeout execution works after 2 minutes
  • ADX degradation cancels signals
  • Database records smart entry metadata
  • No TypeScript runtime errors

Phase 4 Success (Production Validation)

  • 50+ trades executed with smart entry enabled
  • Average entry improvement: 0.2-0.5% measured
  • No adverse effects on win rate
  • No system stability issues
  • User satisfied with results

Phase 5 Success (Performance Analysis)

  • 100+ trades analyzed
  • $1,600-4,000 cumulative profit improvement confirmed
  • Optimal configuration parameters determined
  • Documentation updated with tuning recommendations
  • Feature declared production-ready

Financial Impact Projection

Based on 100 trades at $8,000 average position size:

Entry Improvement Profit per Trade Total Improvement
0.2% (conservative) +$16 +$1,600
0.35% (expected) +$28 +$2,800
0.5% (optimistic) +$40 +$4,000

Assumptions:

  • Position size: $8,000 (current capital $540 × 15x leverage)
  • Pullback hit rate: 40-60% (rest timeout at current price)
  • ADX cancellation rate: <10% (mostly valid cancellations)
  • Win rate maintained or slightly improved (better entries)

Comparison to Phase 1:

  • Phase 1: 1-minute data collection (infrastructure)
  • Phase 2: Smart entry timing (CURRENT - profit generation)
  • Phase 3: ATR-based dynamic targets (planned - further optimization)

Cumulative Impact:

  • Phase 2 alone: +$1,600-4,000 over 100 trades
  • Phase 2 + Phase 3: +$3,000-7,000 expected (combined improvements)
  • All phases complete: +35-40% P&L improvement (per master roadmap)

Next Steps

  1. Immediate (Today):

    • Enable SMART_ENTRY_ENABLED=true in development .env
    • Send test signal via n8n or manual API call
    • Verify logs show queuing and monitoring behavior
    • Test timeout scenario (wait 2+ minutes)
  2. This Week:

    • Execute 5-10 test trades with smart entry enabled
    • Monitor for errors, crashes, unexpected behavior
    • Measure entry improvement on test trades
    • Fix any bugs discovered during testing
  3. Next Week:

    • Deploy to production if testing successful
    • Monitor first 20 production trades closely
    • Compare smart entry vs immediate entry performance
    • Adjust configuration parameters if needed
  4. Month 1:

    • Collect 50+ smart entry trades
    • Run SQL analysis comparing entry types
    • Calculate actual profit improvement
    • Tune pullback range, wait time, ADX tolerance
  5. Month 2:

    • Collect 100+ trades total
    • Confirm $1,600-4,000 improvement achieved
    • Document optimal configuration
    • Proceed to Phase 3: ATR-based dynamic targets

References

  • Roadmap: 1MIN_DATA_ENHANCEMENTS_ROADMAP.md
  • Master Plan: OPTIMIZATION_MASTER_ROADMAP.md
  • Phase 1 Status: Complete (1-min data collection working)
  • Phase 3 Roadmap: ATR_BASED_TP_ROADMAP.md
  • Git Commit: a8c1b2c (Nov 26, 2025)

Status: READY FOR TESTING
Next Action: Enable in development and execute first test trade
Expected Result: 0.2-0.5% entry improvement per trade = $16-40 additional profit