Commit Graph

20 Commits

Author SHA1 Message Date
mindesbunister
25776413d0 feat: Add signalSource field to identify manual vs TradingView trades
- Set signalSource='manual' for Telegram trades, 'tradingview' for TradingView
- Updated analytics queries to exclude manual trades from indicator analysis
- getTradingStats() filters manual trades (TradingView performance only)
- Version comparison endpoint filters manual trades
- Created comprehensive filtering guide: docs/MANUAL_TRADE_FILTERING.md
- Ensures clean data for indicator optimization without contamination
2025-11-14 22:55:14 +01:00
mindesbunister
c4c0c63de1 feat: Add Alchemy RPC diagnostic endpoint + complete investigation
- Created /api/testing/drift-init endpoint for systematic RPC testing
- Tested Alchemy: 17-71 subscription errors per init (49 avg over 5 runs)
- Tested Helius: 0 subscription errors, 800ms init time
- DEFINITIVE PROOF: Alchemy rate limits break Drift SDK initialization
- Root cause: Burst subscription pattern hits CUPS limits
- SDK doesn't retry failed subscriptions → unstable state
- Documented complete findings in docs/ALCHEMY_RPC_INVESTIGATION_RESULTS.md
- Investigation CLOSED - Helius is the only reliable solution
2025-11-14 22:20:04 +01:00
mindesbunister
a49db192f4 feat: complete Nextcloud Deck integration with English emoji stack names
- Renamed all stacks to English with emojis (Backlog, Planning, In Progress, Complete)
- Updated sync script to use new stack names
- Created all 3 initiative cards (IDs 189-191)
- Enhanced error handling with detailed debug output
- Updated documentation with API limitations and troubleshooting
- Fixed stack fallback from 'eingang' to '📥 Backlog'

Changes:
- scripts/sync-roadmap-to-deck.py: Updated STATUS_TO_STACK mapping, added verbose logging
- docs/NEXTCLOUD_DECK_SYNC.md: Updated stack table, added Known Limitations section, enhanced troubleshooting

Note: 6 duplicate/test cards (184-188, 192) must be deleted manually from Nextcloud UI
      due to API limitations (DELETE returns 405)
2025-11-14 11:25:09 +01:00
mindesbunister
77a22bae3f feat: add Nextcloud Deck roadmap sync system
- Create discover-deck-ids.sh to find board/stack configuration
- Implement sync-roadmap-to-deck.py for roadmap → Deck sync
- Parse OPTIMIZATION_MASTER_ROADMAP.md and extract initiatives
- Map roadmap status to Deck stacks (eingang/planung/arbeit/erledigt)
- Create cards with titles, descriptions, due dates, progress
- Support dry-run mode for testing before actual sync
- Add comprehensive documentation in NEXTCLOUD_DECK_SYNC.md

**Benefits:**
- Visual kanban board for roadmap management
- Drag & drop to prioritize tasks
- Single source of truth (markdown files)
- Easy task tracking and status updates
- No manual duplication between systems

**Initial Sync:**
- Created 1 card: Initiative 1 (Signal Quality Optimization)
- Placed in 'eingang' (FUTURE status)

**Future Work:**
- Bidirectional sync (Deck → Roadmap)
- Phase-level cards parsing
- Manual card creation → roadmap entry
- Automated cron sync
2025-11-14 11:09:37 +01:00
mindesbunister
9973feb742 docs: Add Helius RPC rate limit documentation
- Comprehensive guide to Helius rate limit tiers
- Current bot configuration and retry strategy
- Estimated daily RPC usage calculation
- Monitoring queries and warning signs
- Optimization strategies (short/medium/long-term)
- Cost-benefit analysis for tier upgrades

Key insights:
- Free tier: 10 RPS sustained, 100k/month
- Current usage: ~8,880 requests/day (24 trades)
- Free tier capacity: 3,300/day → DEFICIT
- Recommendation: Upgrade to Developer tier at 200+ trades/month
2025-11-14 09:57:06 +01:00
mindesbunister
83f1d1e5b6 Add DNS retry logic documentation 2025-11-13 16:06:26 +01:00
mindesbunister
a21ae6d622 Add v7-momentum indicator (experimental, disabled)
- Created momentum scalper indicator for catching rapid price acceleration
- ROC-based detection: 2.0% threshold over 5 bars
- Volume confirmation: 2.0x spike (checks last 3 bars)
- ADX filter: Requires 12+ minimum directional movement
- Anti-chop filter: Blocks signals in dead markets
- Debug table: Real-time metric display for troubleshooting

Status: Functional but signal quality inferior to v6 HalfTrend
Decision: Shelved for now, continue with proven v6 strategy
File: docs/guides/MOMENTUM_INDICATOR_V1.pine (239 lines)

Lessons learned:
- Momentum indicators inherently noisy (40-50% WR expected)
- Signals either too early (false breakouts) or too late (miss move)
- Volume spike timing issue: Often lags price move by 1-2 bars
- Better to optimize proven strategy than add complexity

Related: Position Manager duplicate update bug fixed (awaiting verification)
2025-11-12 19:55:19 +01:00
mindesbunister
03e91fc18d feat: ATR-based trailing stop + rate limit monitoring
MAJOR FIXES:
- ATR-based trailing stop for runners (was fixed 0.3%, now adapts to volatility)
- Fixes runners with +7-9% MFE exiting for losses
- Typical improvement: 2.24x more room (0.3% → 0.67% at 0.45% ATR)
- Enhanced rate limit logging with database tracking
- New /api/analytics/rate-limits endpoint for monitoring

DETAILS:
- Position Manager: Calculate trailing as (atrAtEntry / price × 100) × multiplier
- Config: TRAILING_STOP_ATR_MULTIPLIER=1.5, MIN=0.25%, MAX=0.9%
- Settings UI: Added ATR multiplier controls
- Rate limits: Log hits/recoveries/exhaustions to SystemEvent table
- Documentation: ATR_TRAILING_STOP_FIX.md + RATE_LIMIT_MONITORING.md

IMPACT:
- Runners can now capture big moves (like morning's $172→$162 SOL drop)
- Rate limit visibility prevents silent failures
- Data-driven optimization for RPC endpoint health
2025-11-11 14:51:41 +01:00
mindesbunister
089308a07e Add Position Sync feature for recovering tracking after partial fills
- New /api/trading/sync-positions endpoint (no auth)
- Fetches actual Drift positions and compares with Position Manager
- Removes stale tracking, adds missing positions with calculated TP/SL
- Settings UI: Orange 'Sync Positions' button added
- CLI script: scripts/sync-positions.sh for terminal access
- Full documentation in docs/guides/POSITION_SYNC_GUIDE.md
- Quick reference: POSITION_SYNC_QUICK_REF.md
- Updated AI instructions with pitfall #23

Problem solved: Manual Telegram trades with partial fills can cause
Position Manager to lose tracking, leaving positions without software-
based stop loss protection. This feature restores dual-layer protection.

Note: Docker build not picking up route yet (cache issue), needs investigation
2025-11-10 17:05:32 +01:00
mindesbunister
988fdb9ea4 Fix runner system + strengthen anti-chop filter
Three critical bugs fixed:
1. P&L calculation (65x inflation) - now uses collateralUSD not notional
2. handlePostTp1Adjustments() - checks tp2SizePercent===0 for runner mode
3. JavaScript || operator bug - changed to ?? for proper 0 handling

Signal quality improvements:
- Added anti-chop filter: price position <40% + ADX <25 = -25 points
- Prevents range-bound flip-flops (caught all 3 today)
- Backtest: 43.8% → 55.6% win rate, +86% profit per trade

Changes:
- lib/trading/signal-quality.ts: RANGE-BOUND CHOP penalty
- lib/drift/orders.ts: Fixed P&L calculation + transaction confirmation
- lib/trading/position-manager.ts: Runner system logic
- app/api/trading/execute/route.ts: || to ?? for tp2SizePercent
- app/api/trading/test/route.ts: || to ?? for tp1/tp2SizePercent
- prisma/schema.prisma: Added collateralUSD field
- scripts/fix_pnl_calculations.sql: Historical P&L correction
2025-11-10 15:36:51 +01:00
mindesbunister
9b767342dc feat: Implement re-entry analytics system with fresh TradingView data
- Add market data cache service (5min expiry) for storing TradingView metrics
- Create /api/trading/market-data webhook endpoint for continuous data updates
- Add /api/analytics/reentry-check endpoint for validating manual trades
- Update execute endpoint to auto-cache metrics from incoming signals
- Enhance Telegram bot with pre-execution analytics validation
- Support --force flag to override analytics blocks
- Use fresh ADX/ATR/RSI data when available, fallback to historical
- Apply performance modifiers: -20 for losing streaks, +10 for winning
- Minimum re-entry score 55 (vs 60 for new signals)
- Fail-open design: proceeds if analytics unavailable
- Show data freshness and source in Telegram responses
- Add comprehensive setup guide in docs/guides/REENTRY_ANALYTICS_QUICKSTART.md

Phase 1 implementation for smart manual trade validation.
2025-11-07 20:40:07 +01:00
mindesbunister
711ff9aaf4 Add signal quality version comparison to analytics dashboard
- Created /api/analytics/version-comparison endpoint
- Shows performance metrics for v1, v2, v3 scoring logic
- Compares: trade count, win rate, P&L, quality scores, MFE/MAE
- Special focus on extreme positions (< 15% or > 85% range)
- Tracks weak ADX count (< 18) for each version
- Visual indicators for current version (v3)
- Data collection progress notice for v3 (need 20+ trades)
- Legend explaining MFE, MAE, extreme positions, weak ADX

Enables data-driven optimization by comparing algorithm performance
with clean, version-tagged datasets.
2025-11-07 13:05:48 +01:00
mindesbunister
fdbb474e68 fix(n8n): CRITICAL - Add quality score validation to old workflow path
PROBLEM:
- Trades with quality score 35 and 45 were executed (threshold: 60)
- Position opened without risk management after signal flips
- "Parse Signal" node didn't extract ATR/ADX/RSI/volumeRatio/pricePosition
- "Check Risk" node only sent symbol+direction, skipped quality validation
- "Execute Trade" node didn't forward metrics to backend

ROOT CAUSE:
n8n workflow had TWO paths:
1. NEW: Parse Signal Enhanced → Check Risk1 → Execute Trade1 (working)
2. OLD: Parse Signal → Check Risk → Execute Trade (broken)

Old path bypassed quality check because check-risk endpoint saw
hasContextMetrics=false and allowed trade without validation.

FIX:
1. Changed "Parse Signal" from 'set' to 'code' node with metric extraction
2. Updated "Check Risk" to send atr/adx/rsi/volumeRatio/pricePosition
3. Updated "Execute Trade" to forward all metrics to backend

IMPACT:
- All trades now validated against quality score threshold (60)
- Low-quality signals properly blocked before execution
- Prevents positions opening without proper risk management

Evidence from database showed 3 trades in 2 hours with scores <60:
- 10:00:31 SOL LONG - Score 35 (phantom detected)
- 09:55:30 SOL SHORT - Score 35 (executed)
- 09:35:14 SOL LONG - Score 45 (executed)

All three should have been blocked. Fix prevents future bypasses.
2025-11-04 11:18:57 +01:00
mindesbunister
8bc08955cc feat: Add phantom trade detection and database tracking
- Detect position size mismatches (>50% variance) after opening
- Save phantom trades to database with expectedSizeUSD, actualSizeUSD, phantomReason
- Return error from execute endpoint to prevent Position Manager tracking
- Add comprehensive documentation of phantom trade issue and solution
- Enable data collection for pattern analysis and future optimization

Fixes oracle price lag issue during volatile markets where transactions
confirm but positions don't actually open at expected size.
2025-11-04 10:34:38 +01:00
mindesbunister
f682b93a1e Fix: Signal flip race condition - properly coordinate Position Manager during opposite signal closure
- Remove trade from Position Manager BEFORE closing Drift position (prevents race condition)
- Explicitly save closure to database with proper P&L calculation
- Mark flipped positions as 'manual' exit reason
- Increase delay from 1s to 2s for better on-chain confirmation
- Preserve MAE/MFE data in closure records

Fixes issue where SHORT signal would close LONG but not properly track the new SHORT position.
Database now correctly records both old position closure and new position opening.
2025-11-03 20:23:42 +01:00
mindesbunister
881a99242d feat: Add per-symbol trading controls for SOL and ETH
- Add SymbolSettings interface with enabled/positionSize/leverage fields
- Implement per-symbol ENV variables (SOLANA_*, ETHEREUM_*)
- Add SOL and ETH sections to settings UI with enable/disable toggles
- Add symbol-specific test buttons (SOL LONG/SHORT, ETH LONG/SHORT)
- Update execute and test endpoints to check symbol enabled status
- Add real-time risk/reward calculator per symbol
- Rename 'Position Sizing' to 'Global Fallback' for clarity
- Fix position manager P&L calculation for externally closed positions
- Fix zero P&L bug affecting 12 historical trades
- Add SQL scripts for recalculating historical P&L data
- Move archive TypeScript files to .archive to fix build

Defaults:
- SOL: 10 base × 10x leverage = 100 notional (profit trading)
- ETH:  base × 1x leverage =  notional (data collection)
- Global: 10 × 10x for BTC and other symbols

Configuration priority: Per-symbol ENV > Market config > Global ENV > Defaults
2025-11-03 10:28:48 +01:00
mindesbunister
26f70c6426 feat: implement symbol-specific position sizing for multi-asset trading
- Extended MarketConfig with optional positionSize and leverage fields
- Configured ETH-PERP at  @ 1x leverage for minimal-risk data collection
- Created getPositionSizeForSymbol() helper function in config/trading.ts
- Integrated symbol-specific sizing into execute endpoint
- Added comprehensive guide in docs/guides/SYMBOL_SPECIFIC_SIZING.md

Purpose: Enable ETH trading for faster signal quality data collection
         while preserving SOL's profit-generation sizing (0 @ 10x)

Next: Create ETH alert in TradingView and restart bot
2025-10-31 16:16:03 +01:00
mindesbunister
d3f385deac Add ATR-based position scaling guide
Comprehensive guide covering:
- How ATR is captured and stored (entry value frozen)
- Static ATR approach (Phases 1-3): Use entry ATR for entire trade
- Dynamic ATR approach (Phase 5+): Real-time updates via TradingView or bot calculation
- Use cases: Dynamic TP/SL, trailing stops, scaling in/out decisions
- Implementation path: Start simple with entry ATR, add real-time later if data supports
- Code examples for all approaches
- Troubleshooting common ATR issues
- Database schema considerations

Explains why waiting for data is critical before implementing advanced ATR features.
2025-10-31 13:34:18 +01:00
mindesbunister
eeb90ad455 Add documentation for duplicate position fix 2025-10-27 19:08:52 +01:00
mindesbunister
14d5de2c64 chore: Organize workspace structure - move docs, workflows, scripts to subdirectories
Organization:
- Created docs/ with setup/, guides/, history/ subdirectories
- Created workflows/ with trading/, analytics/, telegram/, archive/ subdirectories
- Created scripts/ with docker/, setup/, testing/ subdirectories
- Created tests/ for TypeScript test files
- Created archive/ for unused reference files

Moved files:
- 17 documentation files → docs/
- 16 workflow JSON files → workflows/
- 10 shell scripts → scripts/
- 4 test files → tests/
- 5 unused files → archive/

Updated:
- README.md with new file structure and documentation paths

Deleted:
- data/ (empty directory)
- screenshots/ (empty directory)

Critical files remain in root:
- telegram_command_bot.py (active bot - used by Dockerfile)
- watch-restart.sh (systemd service dependency)
- All Dockerfiles and docker-compose files
- All environment files

Validation:
 Containers running (trading-bot-v4, telegram-trade-bot, postgres)
 API responding (positions endpoint tested)
 Telegram bot functional (/status command tested)
 All critical files present in root

No code changes - purely organizational.
System continues running without interruption.

Recovery: git revert HEAD or git reset --hard cleanup-before
2025-10-27 12:59:25 +01:00