Commit Graph

77 Commits

Author SHA1 Message Date
mindesbunister
01aaa0932a feat: Direction-specific quality thresholds (long=90, short=95)
- DATA-DRIVEN: 227 trades analysis showed longs 71.4% WR vs shorts 28.6% WR at quality 90-94
- LONG threshold: 90 (captures profitable 90-94 signals: +4.77 total, +.40 avg)
- SHORT threshold: 95 (blocks toxic 90-94 signals: -53.76 total, -9.11 avg)
- Historical validation: Quality 90+ longs +00.62 vs shorts -77.90

Modified files:
- config/trading.ts: Added minSignalQualityScoreLong/Short fields + getMinQualityScoreForDirection()
- lib/trading/signal-quality.ts: Accept direction-specific minScore parameter
- app/api/trading/check-risk/route.ts: Use direction-specific thresholds
- .env: Added MIN_SIGNAL_QUALITY_SCORE_LONG=90 and _SHORT=95

Fallback logic: direction-specific → global → 60 default
Backward compatible with existing code
2025-11-23 15:01:56 +01:00
mindesbunister
a07485c21f feat: Add comprehensive database save protection system
INVESTIGATION RESULT: No database failure occurred - trade was saved correctly.
However, implemented 5-layer protection against future failures:

1. Persistent File Logger (lib/utils/persistent-logger.ts)
   - Survives container restarts
   - Logs to /app/logs/errors.log
   - Daily rotation, 30-day retention

2. Database Save Retry Logic (lib/database/trades.ts)
   - 3 retry attempts with exponential backoff (1s, 2s, 4s)
   - Immediate verification query after each create
   - Persistent logging of all attempts

3. Orphan Position Detection (lib/startup/init-position-manager.ts)
   - Runs on every container startup
   - Queries Drift for positions without database records
   - Creates retroactive Trade records
   - Sends Telegram alerts
   - Restores Position Manager monitoring

4. Critical Logging (app/api/trading/execute/route.ts)
   - Database failures logged with full trade details
   - Stack traces preserved for debugging

5. Infrastructure (logs directory + Docker volume)
   - Mounted at /home/icke/traderv4/logs
   - Configured in docker-compose.yml

Trade from Nov 21 00:40:14 CET:
- Found in database: cmi82qg590001tn079c3qpw4r
- SHORT SOL-PERP 33.69 → 34.67 SL
- P&L: -9.17
- Closed at 01:17:03 CET (37 minutes duration)
- No database failure occurred

Future Protection:
- Retry logic catches transient failures
- Verification prevents silent failures
- Orphan detection catches anything missed
- Persistent logs enable post-mortem analysis
- System now bulletproof for 16 → 00k journey
2025-11-21 09:47:00 +01:00
mindesbunister
b511211f59 fix: Allow manual Telegram trades through timeframe filter
**BUG:** Telegram 'short sol' blocked by multi-timeframe data collection filter
- Filter checked 'timeframe !== 5' which blocked 'manual' timeframe
- Manual trades from Telegram should execute, not be saved for analysis

**FIX:** Updated condition to 'timeframe !== 5 && timeframe !== manual'
- Allows both 5min TradingView signals AND manual Telegram trades
- Only blocks 15min/1H/4H/Daily for data collection

**FILES:** app/api/trading/execute/route.ts line 114
**DEPLOYED:** Nov 20, 2025 15:42 CET
2025-11-20 16:35:09 +01:00
mindesbunister
60fc571aa6 feat: Automated multi-timeframe price tracking system
Implemented comprehensive price tracking for multi-timeframe signal analysis.

**Components Added:**
- lib/analysis/blocked-signal-tracker.ts - Background job tracking prices
- app/api/analytics/signal-tracking/route.ts - Status/metrics endpoint

**Features:**
- Automatic price tracking at 1min, 5min, 15min, 30min intervals
- TP1/TP2/SL hit detection using ATR-based targets
- Max favorable/adverse excursion tracking (MFE/MAE)
- Analysis completion after 30 minutes
- Background job runs every 5 minutes
- Entry price captured from signal time

**Database Changes:**
- Added entryPrice field to BlockedSignal (for price tracking baseline)
- Added maxFavorablePrice, maxAdversePrice fields
- Added maxFavorableExcursion, maxAdverseExcursion fields

**Integration:**
- Auto-starts on container startup
- Tracks all DATA_COLLECTION_ONLY signals
- Uses same TP/SL calculation as live trades (ATR-based)
- Calculates profit % based on direction (long vs short)

**API Endpoints:**
- GET /api/analytics/signal-tracking - View tracking status and metrics
- POST /api/analytics/signal-tracking - Manually trigger update (auth required)

**Purpose:**
Enables data-driven multi-timeframe comparison. After 50+ signals per
timeframe, can analyze which timeframe (5min vs 15min vs 1H vs 4H vs Daily)
has best win rate, profit potential, and signal quality.

**What It Tracks:**
- Price at 1min, 5min, 15min, 30min after signal
- Would TP1/TP2/SL have been hit?
- Maximum profit/loss during 30min window
- Complete analysis of signal profitability

**How It Works:**
1. Signal comes in (15min, 1H, 4H, Daily) → saved to BlockedSignal
2. Background job runs every 5min
3. Queries current price from Pyth
4. Calculates profit % from entry
5. Checks if TP/SL thresholds crossed
6. Updates MFE/MAE if new highs/lows
7. After 30min, marks analysisComplete=true

**Future Analysis:**
After 50+ signals per timeframe:
- Compare TP1 hit rates across timeframes
- Identify which timeframe has highest win rate
- Determine optimal signal frequency vs quality trade-off
- Switch production to best-performing timeframe

User requested: "i want all the bells and whistles. lets make the
powerhouse more powerfull. i cant see any reason why we shouldnt"
2025-11-19 17:18:47 +01:00
mindesbunister
9b9d80779d fix: Use signalPrice instead of currentPrice in CreateBlockedSignalParams
- TypeScript build error: currentPrice not in interface
- Correct field name is signalPrice (already defined)
- Fixes multi-timeframe data collection compilation
2025-11-18 20:30:07 +01:00
mindesbunister
325f8d0482 feat: Add multi-timeframe data collection to execute endpoint
- Only 5min signals execute trades (production)
- 15min/1H/4H/Daily signals saved to BlockedSignal table for analysis
- Enables cross-timeframe performance comparison
- Zero financial risk - non-5min signals just collect data
- blockReason: 'DATA_COLLECTION_ONLY' for easy filtering
- Returns HTTP 200 (not 400) since this is expected behavior
- Prepares for future timeframe optimization decisions
2025-11-18 20:24:26 +01:00
mindesbunister
6f85fee1da critical: Fix test endpoint violating database-first pattern (Common Pitfall #29)
- Moved positionManager.addTrade() to AFTER database save succeeds
- Changed database error handling to return HTTP 500 (not silent fail)
- Test endpoint now enforces same pattern as execute endpoint
- Prevents untracked positions when database save fails
- Root cause of trade manual-1763391075992 compounding to -19.43

Before: Test endpoint added to Position Manager first, saved to DB after
After: Test endpoint saves to DB first, only adds to PM if DB succeeds
Impact: No more untracked positions from test trades with failed DB saves
2025-11-17 16:47:07 +01:00
mindesbunister
3aeb00f998 critical: Fix P&L calculation and TP1 false detection bugs
- Add originalPositionSize tracking to prevent stale size usage
- Add price validation to TP1 detection (prevents manual closes misidentified as TP1)
- Fix external closure P&L to use originalPositionSize not currentSize
- Add handleManualClosure method for proper exit reason detection
- Add isPriceAtTarget helper for TP/SL price validation (0.2% tolerance)
- Update all ActiveTrade creation points (execute, test, sync-positions, test-db)

Bug fixes:
- Manual close at 42.34 was detected as TP1 (target 40.71) - FIXED
- P&L showed -$1.71 instead of actual -$2.92 - FIXED
- Exit reason showed SL instead of manual - FIXED

Root cause: Position Manager detected size reduction without validating
price was actually at TP1 level. Used stale currentSize for P&L calculation.

Files modified:
- lib/trading/position-manager.ts (core fixes)
- app/api/trading/execute/route.ts
- app/api/trading/test/route.ts
- app/api/trading/sync-positions/route.ts
- app/api/trading/test-db/route.ts
2025-11-17 15:10:15 +01:00
mindesbunister
141022243a feat: Implement ATR-based TP/SL system for regime-agnostic trading
CRITICAL UPGRADE - Nov 17, 2025

Problem Solved:
- v6 shorts averaging +20.74% MFE but TP exits at +0.7% (leaving 95% on table)
- Fixed % targets don't adapt to bull/bear regime changes
- User must manually adjust settings when sentiment flips
- Market-regime bias in optimization (bearish now ≠ bullish later)

Solution - ATR-Based Dynamic TP/SL:
- TP1 = ATR × 2.0 (adaptive to volatility)
- TP2 = ATR × 4.0 (captures extended moves)
- SL = ATR × 3.0 (proportional risk)
- Safety bounds prevent extremes (min/max caps)

Example with SOL ATR = 0.45%:
- TP1: 0.45% × 2.0 = 0.90% (vs old fixed 0.4%)
- TP2: 0.45% × 4.0 = 1.80% (vs old fixed 0.7%)
- SL: 0.45% × 3.0 = 1.35% (vs old fixed 1.5%)

Benefits:
 Adapts automatically to bull/bear regime changes
 Asset-agnostic (SOL vs BTC have different ATR)
 Captures more profit in volatile conditions
 Tighter risk in calm conditions
 No manual intervention when sentiment shifts
 Consistent with existing ATR-based trailing stop

Implementation:
- Added TradingConfig fields: atrMultiplierTp1/Tp2/Sl with min/max bounds
- New calculatePercentFromAtr() helper function
- Execute endpoint calculates dynamic % from ATR, falls back to fixed % if unavailable
- ENV variables: ATR_MULTIPLIER_TP1/TP2/SL, MIN_TP1/TP2/SL_PERCENT, MAX_TP1/TP2/SL_PERCENT
- Updated .env with new defaults based on v6 MAE/MFE analysis

Configuration:
- USE_ATR_BASED_TARGETS=true (enabled by default)
- Runner: 40% (TAKE_PROFIT_1_SIZE_PERCENT=60)
- Trailing: 1.3x ATR (existing system, unchanged)
- Legacy fixed % used as fallback when ATR unavailable

Files Modified:
- config/trading.ts (interface + defaults + ENV reading)
- app/api/trading/execute/route.ts (ATR calculation logic)
- .env (new ATR multiplier variables)

Expected Impact:
- Capture 2-3x more profit per winning trade
- Maintain same risk management rigor
- Perform well in BOTH bull and bear markets
- Fix v6 underperformance (-$47.70 → positive)

Testing Required:
- Monitor first 10 trades with ATR-based targets
- Verify TP/SL prices match ATR calculations in logs
- Compare P&L to historical fixed-% performance
2025-11-17 11:41:13 +01:00
mindesbunister
40d69b13ef wip: Emergency order restoration endpoint (has singleton issues)
- Created /api/trading/place-exit-orders endpoint
- Created restore-orders.mjs script
- Issue: Next.js creates separate Drift instances per route
- Workaround: Use /api/trading/cancel-orders to remove orphaned orders

Current situation:
- 32 orphaned orders existed and were cancelled
- Position Manager should auto-place new orders
- Manual order placement endpoint needs refactoring
2025-11-16 22:10:15 +01:00
mindesbunister
e8a1ce972d critical: Prevent hedge positions during signal flips
**The 4 Loss Problem:**
Multiple trades today opened opposite positions before previous closed:
- 11:15 SHORT manual close
- 11:21 LONG opened + hit SL (-.84)
- 11:21 SHORT opened same minute (both positions live)
- Result: Hedge with limited capital = double risk

**Root Cause:**
- Execute endpoint had 2-second delay after close
- During rate limiting, close takes 30+ seconds
- New position opened before old one confirmed closed
- Both positions live = hedge you can't afford at 100% capital

**Fix Applied:**
1. Block flip if close fails (don't open new position)
2. Wait for Drift confirmation (up to 15s), not just tx confirmation
3. Poll Drift every 2s to verify position actually closed
4. Only proceed with new position after verified closure
5. Return HTTP 500 if position still exists after 15s

**Impact:**
-  NO MORE accidental hedges
-  Guaranteed old position closed before new opens
-  Protects limited capital from double exposure
-  Fails safe (blocks flip rather than creating hedge)

**Trade-off:**
- Flips now take 2-15s longer (verification wait)
- But eliminates hedge risk that caused -4 losses

Files modified:
- app/api/trading/execute/route.ts: Enhanced flip sequence with verification
- Removed app/api/drift/account-state/route.ts (had TypeScript errors)
2025-11-16 20:51:26 +01:00
mindesbunister
25776413d0 feat: Add signalSource field to identify manual vs TradingView trades
- Set signalSource='manual' for Telegram trades, 'tradingview' for TradingView
- Updated analytics queries to exclude manual trades from indicator analysis
- getTradingStats() filters manual trades (TradingView performance only)
- Version comparison endpoint filters manual trades
- Created comprehensive filtering guide: docs/MANUAL_TRADE_FILTERING.md
- Ensures clean data for indicator optimization without contamination
2025-11-14 22:55:14 +01:00
mindesbunister
78ab9e1a94 fix: Increase transaction confirmation timeout to 60s for Alchemy Growth
- Alchemy Growth (10,000 CU/s) can handle longer confirmation waits
- Increased timeout from 30s to 60s in both openPosition() and closePosition()
- Added debug logging to execute endpoint to trace hang points
- Configured dual RPC: Alchemy primary (transactions), Helius fallback (subscriptions)
- Previous 30s timeout was causing premature failures during Solana congestion
- This should resolve 'Transaction was not confirmed in 30.00 seconds' errors

Related: User reported n8n webhook returning 500 with timeout error
2025-11-14 20:42:59 +01:00
mindesbunister
7afd7d5aa1 feat: switch from Helius to Alchemy RPC provider
Changes:
- Updated SOLANA_RPC_URL to use Alchemy (https://solana-mainnet.g.alchemy.com/v2/...)
- Migrated from Helius free tier to Alchemy free tier
- Includes previous rate limit fixes (8s backoff, 2s operation delays)

Context:
- Helius free tier: 10 req/sec sustained, 100 req/sec burst
- Alchemy free tier: 300M compute units/month (more generous)
- User hit 239 rate limit errors in 10 minutes on Helius
- User registered Alchemy account and provided API key

Impact:
- Should significantly reduce 429 rate limit errors
- Better free tier limits for trading bot operations
- Combined with delay fixes for optimal RPC usage
2025-11-14 14:01:52 +01:00
mindesbunister
795026aed1 fix: use Pyth price data for flip-flop context check
CRITICAL FIX: Previous implementation showed incorrect price movements
(100% instead of 0.2%) because currentPrice wasn't available in
check-risk endpoint.

Changes:
- app/api/trading/check-risk/route.ts: Fetch current price from Pyth
  price monitor before quality scoring
- lib/trading/signal-quality.ts: Added validation and detailed logging
  - Check if currentPrice available, apply penalty if missing
  - Log actual prices: $X → $Y = Z%
  - Include prices in penalty/allowance messages

Example outputs:
 Flip-flop in tight range: 4min ago, only 0.20% move ($143.86 → $143.58) (-25 pts)
 Direction change after 10.2% move ($170.00 → $153.00, 12min ago) - reversal allowed

This fixes the false positive that allowed a 0.2% flip-flop earlier today.

Deployed: 09:42 CET Nov 14, 2025
2025-11-14 08:23:04 +01:00
mindesbunister
77a9437d26 feat: add price movement context to flip-flop detection
Improved flip-flop penalty logic to distinguish between:
- Chop (bad): <2% price move from opposite signal → -25 penalty
- Reversal (good): ≥2% price move from opposite signal → allowed

Changes:
- lib/database/trades.ts: getRecentSignals() now returns oppositeDirectionPrice
- lib/trading/signal-quality.ts: Added currentPrice parameter, price movement check
- app/api/trading/check-risk/route.ts: Added currentPrice to RiskCheckRequest interface
- app/api/trading/execute/route.ts: Pass openResult.fillPrice as currentPrice
- app/api/analytics/reentry-check/route.ts: Pass currentPrice from metrics

Example scenarios:
- ETH $170 SHORT → $153 LONG (10% move) = reversal allowed 
- ETH $154.50 SHORT → $154.30 LONG (0.13% move) = chop blocked ⚠️

Deployed: 09:18 CET Nov 14, 2025
Container: trading-bot-v4
2025-11-14 07:46:28 +01:00
mindesbunister
111e3ed12a feat: implement signal frequency penalties for flip-flop detection
PHASE 1 IMPLEMENTATION:
Signal quality scoring now checks database for recent trading patterns
and applies penalties to prevent overtrading and flip-flop losses.

NEW PENALTIES:
1. Overtrading: 3+ signals in 30min → -20 points
   - Detects consolidation zones where system generates excessive signals
   - Counts both executed trades AND blocked signals

2. Flip-flop: Opposite direction in last 15min → -25 points
   - Prevents rapid long→short→long whipsaws
   - Example: SHORT at 10:00, LONG at 10:12 = blocked

3. Alternating pattern: Last 3 trades flip directions → -30 points
   - Detects choppy market conditions
   - Pattern like long→short→long = system getting chopped

DATABASE INTEGRATION:
- New function: getRecentSignals() in lib/database/trades.ts
- Queries last 30min of trades + blocked signals
- Checks last 3 executed trades for alternating pattern
- Zero performance impact (fast indexed queries)

ARCHITECTURE:
- scoreSignalQuality() now async (requires database access)
- All callers updated: check-risk, execute, reentry-check
- skipFrequencyCheck flag available for special cases
- Frequency penalties included in qualityResult breakdown

EXPECTED IMPACT:
- Eliminate overnight flip-flop losses (like SOL $141-145 chop)
- Reduce overtrading during sideways consolidation
- Better capital preservation in non-trending markets
- Should improve win rate by 5-10% by avoiding worst setups

TESTING:
- Deploy and monitor next 5 signals in choppy markets
- Check logs for frequency penalty messages
- Analyze if blocked signals would have been losers

Files changed:
- lib/database/trades.ts: Added getRecentSignals()
- lib/trading/signal-quality.ts: Made async, added frequency checks
- app/api/trading/check-risk/route.ts: await + symbol parameter
- app/api/trading/execute/route.ts: await + symbol parameter
- app/api/analytics/reentry-check/route.ts: await + skipFrequencyCheck
2025-11-14 06:41:03 +01:00
mindesbunister
6590f4fb1e feat: phantom trade auto-closure system
- Auto-close phantom positions immediately via market order
- Return HTTP 200 (not 500) to allow n8n workflow continuation
- Save phantom trades to database with full P&L tracking
- Exit reason: 'manual' category for phantom auto-closes
- Protects user during unavailable hours (sleeping, no phone)
- Add Docker build best practices to instructions (background + tail)
- Document phantom system as Critical Component #1
- Add Common Pitfall #30: Phantom notification workflow

Why auto-close:
- User can't always respond to phantom alerts
- Unmonitored position = unlimited risk exposure
- Better to exit with small loss/gain than leave exposed
- Re-entry possible if setup actually good

Files changed:
- app/api/trading/execute/route.ts: Auto-close logic
- .github/copilot-instructions.md: Documentation + build pattern
2025-11-14 05:37:51 +01:00
mindesbunister
bd9633fbc2 CRITICAL FIX: Prevent unprotected positions via database-first pattern
Root Cause:
- Execute endpoint saved to database AFTER adding to Position Manager
- Database save failures were silently caught and ignored
- API returned success even when DB save failed
- Container restarts lost in-memory Position Manager state
- Result: Unprotected positions with no TP/SL monitoring

Fixes Applied:

1. Database-First Pattern (app/api/trading/execute/route.ts):
   - MOVED createTrade() BEFORE positionManager.addTrade()
   - If database save fails, return HTTP 500 with critical error
   - Error message: 'CLOSE POSITION MANUALLY IMMEDIATELY'
   - Position Manager only tracks database-persisted trades
   - Ensures container restarts can restore all positions

2. Transaction Timeout (lib/drift/orders.ts):
   - Added 30s timeout to confirmTransaction() in closePosition()
   - Prevents API from hanging during network congestion
   - Uses Promise.race() pattern for timeout enforcement

3. Telegram Error Messages (telegram_command_bot.py):
   - Parse JSON for ALL responses (not just 200 OK)
   - Extract detailed error messages from 'message' field
   - Shows critical warnings to user immediately
   - Fail-open: proceeds if analytics check fails

4. Position Manager (lib/trading/position-manager.ts):
   - Move lastPrice update to TOP of monitoring loop
   - Ensures /status endpoint always shows current price

Verification:
- Test trade cmhxj8qxl0000od076m21l58z executed successfully
- Database save completed BEFORE Position Manager tracking
- SL triggered correctly at -$4.21 after 15 minutes
- All protection systems working as expected

Impact:
- Eliminates risk of unprotected positions
- Provides immediate critical warnings if DB fails
- Enables safe container restarts with full position recovery
- Verified with live test trade on production

See: CRITICAL_INCIDENT_UNPROTECTED_POSITION.md for full incident report
2025-11-13 15:56:28 +01:00
mindesbunister
74df461556 fix: use actual symbol-specific leverage in notifications
Fixed Telegram notification showing wrong leverage (10x instead of 20x).

Problem:
- SOL trades use SOLANA_LEVERAGE=20x (per-symbol override)
- API response was returning config.leverage (global default 10x)
- n8n workflow displayed incorrect leverage value

Changes:
- Line 345: Use 'leverage' variable (from getPositionSizeForSymbol)
- Line 448: ActiveTrade uses actual leverage
- Line 522: ExecuteTradeResponse uses actual leverage
- Line 557: Database createTrade() uses actual leverage

Now notifications correctly show 20x for SOL trades.
2025-11-12 11:42:51 +01:00
mindesbunister
2c6295367c feat: add indicatorVersion tracking to backend
Added indicatorVersion field to track which TradingView indicator version
generated each signal (v5, v6, etc.)

Changes:
- Updated ExecuteTradeRequest interface to include indicatorVersion field
- Added indicatorVersion to both createTrade() calls with default 'v5' fallback
- Field already exists in Prisma schema (indicatorVersion String?)
- Defaults to 'v5' for backward compatibility with old alerts

This enables comparison of indicator performance:
- v5: Original Money Line indicator
- v6: Improved version with 100-bar price position filter

Works alongside existing signalQualityVersion (v4) which tracks backend
scoring algorithm changes. Two separate version fields:
1. indicatorVersion = TradingView Pine Script version (v5/v6)
2. signalQualityVersion = Backend scoring logic version (v4)

Frontend can now filter/compare trades by indicator version in analytics.
2025-11-12 08:22:06 +01:00
mindesbunister
ba13c20c60 feat: implement blocked signals tracking system
- Add BlockedSignal table with 25 fields for comprehensive signal analysis
- Track all blocked signals with metrics (ATR, ADX, RSI, volume, price position)
- Store quality scores, block reasons, and detailed breakdowns
- Include future fields for automated price analysis (priceAfter1/5/15/30Min)
- Restore signalQualityVersion field to Trade table

Database changes:
- New table: BlockedSignal with indexes on symbol, createdAt, score, blockReason
- Fixed schema drift from manual changes

API changes:
- Modified check-risk endpoint to save blocked signals automatically
- Fixed hasContextMetrics variable scope (moved to line 209)
- Save blocks for: quality score too low, cooldown period, hourly limit
- Use config.minSignalQualityScore instead of hardcoded 60

Database helpers:
- Added createBlockedSignal() function with try/catch safety
- Added getRecentBlockedSignals(limit) for queries
- Added getBlockedSignalsForAnalysis(olderThanMinutes) for automation

Documentation:
- Created BLOCKED_SIGNALS_TRACKING.md with SQL queries and analysis workflow
- Created SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md with 5-phase plan
- Documented data-first approach: collect 10-20 signals before optimization

Rationale:
Only 2 historical trades scored 60-64 (insufficient sample size for threshold decision).
Building data collection infrastructure before making premature optimizations.

Phase 1 (current): Collect blocked signals for 1-2 weeks
Phase 2 (next): Analyze patterns and make data-driven threshold decision
Phase 3-5 (future): Automation and ML optimization
2025-11-11 11:49:21 +01:00
mindesbunister
ee89d15b8b Use percentage aware sizing in execute endpoint 2025-11-10 20:27:52 +01:00
mindesbunister
43b688d9f2 Fix test trade endpoint to honor TP2 runner configuration 2025-11-10 19:55:24 +01:00
mindesbunister
c3a053df63 CRITICAL FIX: Use ?? instead of || for tp2SizePercent to allow 0 value
BUG FOUND:
Line 558: tp2SizePercent: config.takeProfit2SizePercent || 100

When config.takeProfit2SizePercent = 0 (TP2-as-runner system), JavaScript's ||
operator treats 0 as falsy and falls back to 100, causing TP2 to close 100%
of remaining position instead of activating trailing stop.

IMPACT:
- On-chain orders placed correctly (line 481 uses ?? correctly)
- Position Manager reads from DB and expects TP2 to close position
- Result: User sees TWO take-profit orders instead of runner system

FIX:
Changed both tp1SizePercent and tp2SizePercent to use ?? operator:
- tp1SizePercent: config.takeProfit1SizePercent ?? 75
- tp2SizePercent: config.takeProfit2SizePercent ?? 0

This allows 0 value to be saved correctly for TP2-as-runner system.

VERIFICATION NEEDED:
Current open SHORT position in database has tp2SizePercent=100 from before
this fix. Next trade will use correct runner system.
2025-11-10 19:46:03 +01:00
mindesbunister
089308a07e Add Position Sync feature for recovering tracking after partial fills
- New /api/trading/sync-positions endpoint (no auth)
- Fetches actual Drift positions and compares with Position Manager
- Removes stale tracking, adds missing positions with calculated TP/SL
- Settings UI: Orange 'Sync Positions' button added
- CLI script: scripts/sync-positions.sh for terminal access
- Full documentation in docs/guides/POSITION_SYNC_GUIDE.md
- Quick reference: POSITION_SYNC_QUICK_REF.md
- Updated AI instructions with pitfall #23

Problem solved: Manual Telegram trades with partial fills can cause
Position Manager to lose tracking, leaving positions without software-
based stop loss protection. This feature restores dual-layer protection.

Note: Docker build not picking up route yet (cache issue), needs investigation
2025-11-10 17:05:32 +01:00
mindesbunister
988fdb9ea4 Fix runner system + strengthen anti-chop filter
Three critical bugs fixed:
1. P&L calculation (65x inflation) - now uses collateralUSD not notional
2. handlePostTp1Adjustments() - checks tp2SizePercent===0 for runner mode
3. JavaScript || operator bug - changed to ?? for proper 0 handling

Signal quality improvements:
- Added anti-chop filter: price position <40% + ADX <25 = -25 points
- Prevents range-bound flip-flops (caught all 3 today)
- Backtest: 43.8% → 55.6% win rate, +86% profit per trade

Changes:
- lib/trading/signal-quality.ts: RANGE-BOUND CHOP penalty
- lib/drift/orders.ts: Fixed P&L calculation + transaction confirmation
- lib/trading/position-manager.ts: Runner system logic
- app/api/trading/execute/route.ts: || to ?? for tp2SizePercent
- app/api/trading/test/route.ts: || to ?? for tp1/tp2SizePercent
- prisma/schema.prisma: Added collateralUSD field
- scripts/fix_pnl_calculations.sql: Historical P&L correction
2025-11-10 15:36:51 +01:00
mindesbunister
6f0a1bb49b feat: Implement percentage-based position sizing
- Add usePercentageSize flag to SymbolSettings and TradingConfig
- Add calculateActualPositionSize() and getActualPositionSizeForSymbol() helpers
- Update execute and test endpoints to calculate position size from free collateral
- Add SOLANA_USE_PERCENTAGE_SIZE, ETHEREUM_USE_PERCENTAGE_SIZE, USE_PERCENTAGE_SIZE env vars
- Configure SOL to use 100% of portfolio (auto-adjusts to available balance)
- Fix TypeScript errors: replace fillNotionalUSD with actualSizeUSD
- Remove signalQualityVersion and fullyClosed references (not in interfaces)
- Add comprehensive documentation in PERCENTAGE_SIZING_FEATURE.md

Benefits:
- Prevents insufficient collateral errors by using available balance
- Auto-scales positions as account grows/shrinks
- Maintains risk proportional to capital
- Flexible per-symbol configuration (SOL percentage, ETH fixed)
2025-11-10 13:35:10 +01:00
mindesbunister
d2fbd125a0 fix: Make minSignalQualityScore configurable via settings + anti-chop improvements
CRITICAL BUG FIX:
- Settings page saved MIN_SIGNAL_QUALITY_SCORE to .env but check-risk had hardcoded value
- Now reads from config.minSignalQualityScore (defaults to 65, editable via /settings)
- Prevents settings changes from being ignored after restart

ANTI-CHOP FILTER FIXES:
- Fixed volume breakout bonus conflicting with anti-chop filter
- Volume breakout now requires ADX > 18 (trending market)
- Prevents high volume + low ADX from getting rewarded instead of penalized
- Anti-chop filter now properly blocks whipsaw traps at score 60

TESTING INFRASTRUCTURE:
- Added backtest script showing +17.1% P&L improvement (saved $242 in losses)
- Added test-signals.sh for comprehensive signal quality validation
- Added test-recent-signals.sh for analyzing actual trading session signals
- All tests passing: timeframe awareness, anti-chop, score thresholds

CHANGES:
- config/trading.ts: Added minSignalQualityScore to interface and defaults
- app/api/trading/check-risk/route.ts: Use config value instead of hardcoded 65
- lib/trading/signal-quality.ts: Fixed volume breakout bonus logic
- .env: Added MIN_SIGNAL_QUALITY_SCORE=65
- scripts/: Added comprehensive testing tools

BACKTEST RESULTS (Last 30 trades):
- Old system (score ≥60): $1,412.79 P&L
- New system (score ≥65 + anti-chop): $1,654.79 P&L
- Improvement: +$242.00 (+17.1%)
- Blocked 5 losing trades, missed 0 winners
2025-11-10 11:22:52 +01:00
mindesbunister
4b11186d16 Fix: Add timeframe-aware signal quality scoring for 5min charts
PROBLEM:
- Long signal (ADX 15.7, ATR 0.35%) blocked with score 45/100
- Missed major +3% runup, lost -2 on short that didn't flip
- Scoring logic treated all timeframes identically (daily chart thresholds)

ROOT CAUSE:
- ADX < 18 always scored -15 points regardless of timeframe
- 5min charts naturally have lower ADX (12-22 healthy range)
- copilot-instructions mentioned timeframe awareness but wasn't implemented

FIX:
- Add timeframe parameter to RiskCheckRequest interface
- Update scoreSignalQuality() with timeframe-aware ADX thresholds:
  * 5min/15min: ADX 12-22 healthy (+5), <12 weak (-15), >22 strong (+15)
  * Higher TF: ADX 18-25 healthy (+5), <18 weak (-15), >25 strong (+15)
- Pass timeframe from n8n workflow through check-risk and execute
- Update both Check Risk nodes in Money Machine workflow

IMPACT:
Your blocked signal (ADX 15.7 on 5min) now scores:
- Was: 50 + 5 - 15 + 0 + 0 + 5 = 45 (BLOCKED)
- Now: 50 + 5 + 5 + 0 + 0 + 5 = 65 (PASSES)

This 20-point improvement from timeframe awareness would have caught the runup.
2025-11-10 07:34:21 +01:00
mindesbunister
22195ed34c Fix P&L calculation and signal flip detection
- Fix external closure P&L using tp1Hit flag instead of currentSize
- Add direction change detection to prevent false TP1 on signal flips
- Signal flips now recorded with accurate P&L as 'manual' exits
- Add retry logic with exponential backoff for Solana RPC rate limits
- Create /api/trading/cancel-orders endpoint for manual cleanup
- Improves data integrity for win/loss statistics
2025-11-09 17:59:50 +01:00
mindesbunister
4d533ccb53 fix: Remove obsolete fields from test-db ActiveTrade object
- Remove atrAtEntry and runnerTrailingPercent fields
- These don't exist in ActiveTrade interface
- Fixes TypeScript build error
2025-11-08 11:05:19 +01:00
mindesbunister
2f80c2133c fix: Remove fallback that breaks TP2-as-runner system
- Change tp2SizePercent fallback from || 100 to ?? 0
- Allows 0 value to pass through (means 'activate trailing stop, don't close')
- Fixes bug where TP2 was closing 100% of remaining position
- Now correctly leaves 25% runner after TP1 closes 75%
- Applied to both execute and test endpoints
2025-11-08 10:59:58 +01:00
mindesbunister
9b767342dc feat: Implement re-entry analytics system with fresh TradingView data
- Add market data cache service (5min expiry) for storing TradingView metrics
- Create /api/trading/market-data webhook endpoint for continuous data updates
- Add /api/analytics/reentry-check endpoint for validating manual trades
- Update execute endpoint to auto-cache metrics from incoming signals
- Enhance Telegram bot with pre-execution analytics validation
- Support --force flag to override analytics blocks
- Use fresh ADX/ATR/RSI data when available, fallback to historical
- Apply performance modifiers: -20 for losing streaks, +10 for winning
- Minimum re-entry score 55 (vs 60 for new signals)
- Fail-open design: proceeds if analytics unavailable
- Show data freshness and source in Telegram responses
- Add comprehensive setup guide in docs/guides/REENTRY_ANALYTICS_QUICKSTART.md

Phase 1 implementation for smart manual trade validation.
2025-11-07 20:40:07 +01:00
mindesbunister
6d5991172a feat: Implement ATR-based dynamic TP2 system and fix P&L calculation
- Add ATR-based dynamic TP2 scaling from 0.7% to 3.0% based on volatility
- New config options: useAtrBasedTargets, atrMultiplierForTp2, minTp2Percent, maxTp2Percent
- Enhanced settings UI with ATR controls and updated risk calculator
- Fix external closure P&L calculation using unrealized P&L instead of volatile current price
- Update execute and test endpoints to use calculateDynamicTp2() function
- Maintain 25% runner system for capturing extended moves (4-5% targets)
- Add environment variables for ATR-based configuration
- Better P&L accuracy for manual position closures
2025-11-07 17:01:22 +01:00
mindesbunister
625dc44c59 Add signal quality version tracking to database
- Added signalQualityVersion field to Trade model
- Tracks which scoring logic version was used for each trade
- v1: Original logic (price position < 5% threshold)
- v2: Added volume compensation for low ADX
- v3: CURRENT - Stricter logic requiring ADX > 18 for extreme positions (< 15%)

This enables future analysis to:
- Compare performance between logic versions
- Filter trades by scoring algorithm
- Data-driven improvements based on clean datasets

All new trades will be marked as v3. Old trades remain null/v1 for comparison.
2025-11-07 12:56:35 +01:00
mindesbunister
0365560c5b Add timeframe-aware signal quality scoring for 5min charts
- Lower ADX/ATR thresholds for 5min timeframe (ADX 12-22, ATR 0.2-0.7%)
- Add anti-chop filter: -20 points for extreme sideways (ADX<10, ATR<0.25, Vol<0.9)
- Pass timeframe parameter through check-risk and execute endpoints
- Fixes flip-flop losses from overly strict 5min filters
- Higher timeframes unchanged (still use ADX 18+, ATR 0.4+)

5min scoring now:
- ADX 12-15: moderate trend (+5)
- ADX 22+: strong trend (+15)
- ATR 0.2-0.35: acceptable (+5)
- ATR 0.35+: healthy (+10)
- Extreme chop penalty prevents whipsaw trades
2025-11-07 08:56:19 +01:00
mindesbunister
6c7eaf5f04 Add TP1/SL consistency check on trade restore 2025-11-06 12:18:31 +01:00
mindesbunister
a100945864 Enhance trailing stop with ATR-based sizing 2025-11-05 15:28:12 +01:00
mindesbunister
cbb6592153 fix: correct PnL math and add health probe 2025-11-05 07:58:27 +01:00
mindesbunister
02193b7dce fix(critical): Unify quality score calculation across check-risk and execute
PROBLEM:
- check-risk calculated quality score: 60, 70 (PASSED)
- execute calculated quality score: 35, 45 (should have BLOCKED)
- Two different functions with different logic caused trades to bypass validation

ROOT CAUSE:
Two separate scoring functions existed:
1. scoreSignalQuality() in check-risk (detailed, 95% price threshold)
2. calculateQualityScore() in execute (simpler, 90% price threshold)

Example with pricePosition=96.4%, volumeRatio=0.9:
- check-risk: Checks >95, volumeRatio>1.4 failed → -15 + bonuses = 60  PASSED
- execute: Checks >90 → -15 + bonuses = 35  Should block but already opened

SOLUTION:
1. Created lib/trading/signal-quality.ts with unified scoreSignalQuality()
2. Both endpoints now import and use SAME function
3. Consistent scoring logic: 95% price threshold, volume breakout bonus
4. Returns detailed reasons for debugging

IMPACT:
- Quality scores now MATCH between check-risk and execute
- No more trades bypassing validation due to calculation differences
- Better debugging with quality reasons logged

Files changed:
- NEW: lib/trading/signal-quality.ts (unified scoring function)
- MODIFIED: app/api/trading/check-risk/route.ts (import shared function)
- MODIFIED: app/api/trading/execute/route.ts (import shared function)
- REMOVED: Duplicate calculateQualityScore() from execute
- REMOVED: Duplicate scoreSignalQuality() from check-risk
2025-11-04 11:40:25 +01:00
mindesbunister
8bc08955cc feat: Add phantom trade detection and database tracking
- Detect position size mismatches (>50% variance) after opening
- Save phantom trades to database with expectedSizeUSD, actualSizeUSD, phantomReason
- Return error from execute endpoint to prevent Position Manager tracking
- Add comprehensive documentation of phantom trade issue and solution
- Enable data collection for pattern analysis and future optimization

Fixes oracle price lag issue during volatile markets where transactions
confirm but positions don't actually open at expected size.
2025-11-04 10:34:38 +01:00
mindesbunister
f682b93a1e Fix: Signal flip race condition - properly coordinate Position Manager during opposite signal closure
- Remove trade from Position Manager BEFORE closing Drift position (prevents race condition)
- Explicitly save closure to database with proper P&L calculation
- Mark flipped positions as 'manual' exit reason
- Increase delay from 1s to 2s for better on-chain confirmation
- Preserve MAE/MFE data in closure records

Fixes issue where SHORT signal would close LONG but not properly track the new SHORT position.
Database now correctly records both old position closure and new position opening.
2025-11-03 20:23:42 +01:00
mindesbunister
8a8d4a348c feat: Add position scaling for strong confirmation signals
**Feature: Position Scaling**
Allows adding to existing profitable positions when high-quality signals confirm trend strength.

**Configuration (config/trading.ts):**
- enablePositionScaling: false (disabled by default - enable after testing)
- minScaleQualityScore: 75 (higher bar than initial 60)
- minProfitForScale: 0.4% (must be at/past TP1)
- maxScaleMultiplier: 2.0 (max 200% of original size)
- scaleSizePercent: 50% (add 50% of original position)
- minAdxIncrease: 5 (ADX must strengthen)
- maxPricePositionForScale: 70% (don't chase resistance)

**Validation Logic (check-risk endpoint):**
Same-direction signal triggers scaling check if enabled:
1. Quality score ≥75 (stronger than initial entry)
2. Position profitable ≥0.4% (at/past TP1)
3. ADX increased ≥5 points (trend strengthening)
4. Price position <70% (not near resistance)
5. Total size <2x original (risk management)
6. Returns 'allowed: true, reason: Position scaling' if all pass

**Execution (execute endpoint):**
- Opens additional position at scale size (50% of original)
- Updates ActiveTrade: timesScaled, totalScaleAdded, currentSize
- Tracks originalAdx from first entry for comparison
- Returns 'action: scaled' with scale details

**ActiveTrade Interface:**
Added fields:
- originalAdx?: number (for scaling validation)
- timesScaled?: number (track scaling count)
- totalScaleAdded?: number (total USD added)

**Example Scenario:**
1. LONG SOL at $176 (quality: 45, ADX: 13.4) - weak but entered
2. Price hits $176.70 (+0.4%) - at TP1
3. New LONG signal (quality: 78, ADX: 19) - strong confirmation
4. Scaling validation:  Quality 78  Profit +0.4%  ADX +5.6  Price 68%
5. Adds 50% more position at $176.70
6. Total position: 150% of original size

**Conservative Design:**
- Disabled by default (requires manual enabling)
- Only scales INTO profitable positions (never averaging down)
- Requires significant quality improvement (75 vs 60)
- Requires trend confirmation (ADX increase)
- Hard cap at 2x original size
- Won't chase near resistance levels

**Next Steps:**
1. Enable in settings: ENABLE_POSITION_SCALING=true
2. Test with small positions first
3. Monitor data: do scaled positions outperform?
4. Adjust thresholds based on results

**Safety:**
- All existing duplicate prevention logic intact
- Flip logic unchanged (still requires quality check)
- Position Manager tracks scaling state
- Can be toggled on/off without code changes
2025-11-03 15:35:33 +01:00
mindesbunister
57f0457f95 fix: Require signal quality check for position flips
**Problem:**
- Signal flips (SHORT→LONG or LONG→SHORT) were auto-approved
- Bypassed signal quality scoring, cooldown, drawdown checks
- User wanted flips ONLY if new signal has strong quality (score ≥60)

**Solution:**
- Removed early return for opposite-direction signals in check-risk
- Flips now go through FULL validation: quality score, cooldown, limits
- Execute endpoint still handles flip logic (close opposite + open new)

**New Flow:**
1. n8n sends flip signal → check-risk endpoint
2. Detects potential flip, logs 'checking quality score'
3. Continues to quality checks (not early return)
4. If score ≥60 AND all checks pass → execute handles flip
5. If score <60 → BLOCKS flip with 'Signal quality too low'

**Result:**
Flips now require signal strength, not just direction change
2025-11-03 14:34:26 +01:00
mindesbunister
6b1d32a72d fix: Add phantom trade detection and prevention safeguards
**Root Causes:**
1. Auto-flip logic could create phantom trades if close failed
2. Position size mismatches (0.01 SOL vs 11.92 SOL expected) not caught
3. Multiple trades for same symbol+direction in database

**Preventive Measures:**

1. **Startup Validation (lib/startup/init-position-manager.ts)**
   - Validates all open trades against Drift positions on startup
   - Auto-closes phantom trades with <50% expected size
   - Logs size mismatches for manual review
   - Prevents Position Manager from tracking ghost positions

2. **Duplicate Position Prevention (app/api/trading/execute/route.ts)**
   - Blocks opening same-direction position on same symbol
   - Returns 400 error if duplicate detected
   - Only allows auto-flip (opposite direction close + open)

3. **Runtime Phantom Detection (lib/trading/position-manager.ts)**
   - Checks position size every 2s monitoring cycle
   - Auto-closes if size ratio <50% (extreme mismatch)
   - Logs as 'manual' exit with AUTO_CLEANUP tx
   - Removes from monitoring immediately

4. **Quality Score Fix (app/api/trading/check-risk/route.ts)**
   - Hardcoded minScore=60 (removed non-existent config reference)

**Prevention Summary:**
-  Startup validation catches historical phantoms
-  Duplicate check prevents new phantoms
-  Runtime detection catches size mismatches <30s after they occur
-  All three layers work together for defense-in-depth

Issue: User had LONG (phantom) + SHORT (undersized 0.01 SOL vs 11.92 expected)
Fix: Both detected and closed, bot now clean with 0 active trades
2025-11-03 13:53:12 +01:00
mindesbunister
881a99242d feat: Add per-symbol trading controls for SOL and ETH
- Add SymbolSettings interface with enabled/positionSize/leverage fields
- Implement per-symbol ENV variables (SOLANA_*, ETHEREUM_*)
- Add SOL and ETH sections to settings UI with enable/disable toggles
- Add symbol-specific test buttons (SOL LONG/SHORT, ETH LONG/SHORT)
- Update execute and test endpoints to check symbol enabled status
- Add real-time risk/reward calculator per symbol
- Rename 'Position Sizing' to 'Global Fallback' for clarity
- Fix position manager P&L calculation for externally closed positions
- Fix zero P&L bug affecting 12 historical trades
- Add SQL scripts for recalculating historical P&L data
- Move archive TypeScript files to .archive to fix build

Defaults:
- SOL: 10 base × 10x leverage = 100 notional (profit trading)
- ETH:  base × 1x leverage =  notional (data collection)
- Global: 10 × 10x for BTC and other symbols

Configuration priority: Per-symbol ENV > Market config > Global ENV > Defaults
2025-11-03 10:28:48 +01:00
mindesbunister
0ed2e89c7e feat: implement per-symbol cooldown period
CRITICAL: Cooldown was global across ALL symbols, causing missed opportunities
Example: ETH trade at 10:00 blocked SOL trade at 10:04 (5min cooldown)

Changes:
- Added getLastTradeTimeForSymbol() function to query last trade per symbol
- Updated check-risk endpoint to use symbol-specific cooldown
- Each coin (SOL/ETH/BTC) now has independent cooldown timer
- Cooldown message shows symbol: 'Must wait X min before next SOL-PERP trade'

Result: Can trade ETH and SOL simultaneously without interference
Example: ETH LONG at 10:00, SOL SHORT at 10:01 = both allowed
2025-11-03 08:01:30 +01:00
mindesbunister
d4aeeb4f99 fix: add MAE/MFE fields to all ActiveTrade initializations
Updated execute, test, and test-db endpoints to include:
- maxFavorableExcursion: 0
- maxAdverseExcursion: 0
- maxFavorablePrice: entryPrice
- maxAdversePrice: entryPrice

Required for TypeScript compilation after adding MAE/MFE tracking
2025-11-02 23:04:02 +01:00
mindesbunister
32e88c3823 fix: improve signal quality scoring for volume breakouts
- Lower ATR threshold from 0.6% to 0.15% (allows low volatility breakouts)
- Increase volume bonus: +15 for very strong volume (1.5x+), was +10 for 1.2x+
- Add volume breakout logic: High volume (1.4x+) at 95%+ range gets +5 instead of -15 penalty
- Add volume compensation: +10 bonus when volume >1.8x and ATR <0.6%
- Example: SOL signal with 0.18% ATR, 1.74x volume at 95.6% range now scores 70/100 (PASS) instead of 25/100 (BLOCK)
- This signal moved +0.97% and would have hit TP1 (+1.5%) - proves quality scoring was too conservative
- Changes apply globally to all symbols (SOL, ETH, BTC) using same scoring algorithm
2025-11-02 09:10:03 +01:00