Add signal quality version tracking to database

- Added signalQualityVersion field to Trade model - Tracks which scoring logic version was used for each trade - v1: Original logic (price position < 5% threshold) - v2: Added volume compensation for low ADX - v3: CURRENT - Stricter logic requiring ADX > 18 for extreme positions (< 15%) This enables future analysis to: - Compare performance between logic versions - Filter trades by scoring algorithm - Data-driven improvements based on clean datasets All new trades will be marked as v3. Old trades remain null/v1 for comparison.
2025-11-07 12:56:35 +01:00
parent 3c9da22a8a
commit 625dc44c59
6 changed files with 107 additions and 13 deletions
--- a/.env
+++ b/.env
@@ -355,7 +355,7 @@ TRAILING_STOP_ACTIVATION=0.4
 MIN_QUALITY_SCORE=60
 SOLANA_ENABLED=true
 SOLANA_POSITION_SIZE=210
-SOLANA_LEVERAGE=10
+SOLANA_LEVERAGE=5
 ETHEREUM_ENABLED=false
 ETHEREUM_POSITION_SIZE=50
 ETHEREUM_LEVERAGE=1
@@ -365,4 +365,7 @@ MIN_PROFIT_FOR_SCALE=0.4
 MAX_SCALE_MULTIPLIER=2
 SCALE_SIZE_PERCENT=50
 MIN_ADX_INCREASE=5
-MAX_PRICE_POSITION_FOR_SCALE=70
+MAX_PRICE_POSITION_FOR_SCALE=70
+TRAILING_STOP_ATR_MULTIPLIER=1.5
+TRAILING_STOP_MIN_PERCENT=0.25
+TRAILING_STOP_MAX_PERCENT=0.9
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -21,11 +21,45 @@

 **Signal Quality System:** Filters trades based on 5 metrics (ATR, ADX, RSI, volumeRatio, pricePosition) scored 0-100. Only trades scoring 60+ are executed. Scores stored in database for future optimization.

+**Timeframe-Aware Scoring:** Signal quality thresholds adjust based on timeframe (5min vs daily):
+- 5min: ADX 12+ trending (vs 18+ for daily), ATR 0.2-0.7% healthy (vs 0.4%+ for daily)
+- Anti-chop filter: -20 points for extreme sideways (ADX <10, ATR <0.25%, Vol <0.9x)
+- Pass `timeframe` param to `scoreSignalQuality()` from TradingView alerts (e.g., `timeframe: "5"`)
+
 **MAE/MFE Tracking:** Every trade tracks Maximum Favorable Excursion (best profit %) and Maximum Adverse Excursion (worst loss %) updated every 2s. Used for data-driven optimization of TP/SL levels.

+**Manual Trading via Telegram:** Send plain-text messages like `long sol`, `short eth`, `long btc` to open positions instantly (bypasses n8n, calls `/api/trading/execute` directly with preset healthy metrics).
+
 ## Critical Components

-### 1. Position Manager (`lib/trading/position-manager.ts`)
+### 1. Signal Quality Scoring (`lib/trading/signal-quality.ts`)
+**Purpose:** Unified quality validation system that scores trading signals 0-100 based on 5 market metrics
+
+**Timeframe-aware thresholds:**
+```typescript
+scoreSignalQuality({ 
+  atr, adx, rsi, volumeRatio, pricePosition, 
+  timeframe?: string // "5" for 5min, undefined for higher timeframes
+})
+```
+
+**5min chart adjustments:**
+- ADX healthy range: 12-22 (vs 18-30 for daily)
+- ATR healthy range: 0.2-0.7% (vs 0.4%+ for daily)
+- Anti-chop filter: -20 points for extreme sideways (ADX <10, ATR <0.25%, Vol <0.9x)
+
+**Price position penalties (all timeframes):**
+- Long at 90-95%+ range: -15 to -30 points (chasing highs)
+- Short at <5-10% range: -15 to -30 points (chasing lows)
+- Prevents flip-flop losses from entering range extremes
+
+**Key behaviors:**
+- Returns score 0-100 and detailed breakdown object
+- Minimum score 60 required to execute trade
+- Called by both `/api/trading/check-risk` and `/api/trading/execute`
+- Scores saved to database for post-trade analysis
+
+### 2. Position Manager (`lib/trading/position-manager.ts`)
 **Purpose:** Software-based monitoring loop that checks prices every 2 seconds and closes positions via market orders

 **Singleton pattern:** Always use `getInitializedPositionManager()` - never instantiate directly
@@ -46,18 +80,45 @@ await positionManager.addTrade(activeTrade)
 - **Grace period for new trades:** Skips "external closure" detection for positions <30 seconds old (Drift positions take 5-10s to propagate)
 - **Exit reason detection:** Uses trade state flags (`tp1Hit`, `tp2Hit`) and realized P&L to determine exit reason, NOT current price (avoids misclassification when price moves after order fills)

-### 2. Drift Client (`lib/drift/client.ts`)
-**Purpose:** Solana/Drift Protocol SDK wrapper for order execution
+### 3. Telegram Bot (`telegram_command_bot.py`)
+**Purpose:** Python-based Telegram bot for manual trading commands and position status monitoring

-**Singleton pattern:** Use `initializeDriftService()` and `getDriftService()` - maintains single connection
+**Manual trade commands via plain text:**
+```python
+# User sends plain text message (not slash commands)
+"long sol"   → Opens SOL-PERP long position
+"short eth"  → Opens ETH-PERP short position
+"long btc"   → Opens BTC-PERP long position
+```
+
+**Key behaviors:**
+- MessageHandler processes all text messages (not just commands)
+- Maps user-friendly symbols (sol, eth, btc) to Drift format (SOL-PERP, etc.)
+- Calls `/api/trading/execute` directly with preset healthy metrics (ATR=1.0, ADX=25, RSI=50, volumeRatio=1.2)
+- Bypasses n8n workflow and TradingView requirements
+- 60-second timeout for API calls
+- Responds with trade confirmation or error message
+
+**Status command:**
+```python
+/status → Returns JSON of open positions from Drift
+```
+
+**Implementation details:**
+- Uses `python-telegram-bot` library
+- Deployed via `docker-compose.telegram-bot.yml`
+- Requires `TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHANNEL_ID` in .env
+- API calls to `http://trading-bot:3000/api/trading/execute`
+
+**Drift client integration:**
+- Singleton pattern: Use `initializeDriftService()` and `getDriftService()` - maintains single connection
 ```typescript
 const driftService = await initializeDriftService()
 const health = await driftService.getAccountHealth()
 ```
+- Wallet handling: Supports both JSON array `[91,24,...]` and base58 string formats from Phantom wallet

-**Wallet handling:** Supports both JSON array `[91,24,...]` and base58 string formats from Phantom wallet
-
-### 3. Order Placement (`lib/drift/orders.ts`)
+### 4. Order Placement (`lib/drift/orders.ts`)
 **Critical functions:** 
 - `openPosition()` - Opens market position with transaction confirmation
 - `closePosition()` - Closes position with transaction confirmation
@@ -90,7 +151,7 @@ Without this, the SDK returns signatures for transactions that never execute, ca
 - Soft SL: TRIGGER_LIMIT reduce-only
 - Hard SL: TRIGGER_MARKET reduce-only

-### 4. Database (`lib/database/trades.ts` + `prisma/schema.prisma`)
+### 5. Database (`lib/database/trades.ts` + `prisma/schema.prisma`)
 **Purpose:** PostgreSQL via Prisma ORM for trade history and analytics

 **Models:** Trade, PriceUpdate, SystemEvent, DailyStats
@@ -164,7 +225,7 @@ const driftSymbol = normalizeTradingViewSymbol(body.symbol)

 ### Execute Trade (Production)
 ```
-TradingView alert → n8n Parse Signal Enhanced (extracts metrics)
+TradingView alert → n8n Parse Signal Enhanced (extracts metrics + timeframe)
  ↓ /api/trading/check-risk [validates quality score ≥60, checks duplicates, per-symbol cooldown]
  ↓ /api/trading/execute
  ↓ normalize symbol (SOLUSDT → SOL-PERP)
@@ -173,7 +234,7 @@ TradingView alert → n8n Parse Signal Enhanced (extracts metrics)
  ↓ openPosition() [MARKET order]
  ↓ calculate dual stop prices if enabled
  ↓ placeExitOrders() [on-chain TP1/TP2/SL orders]
-  ↓ calculateQualityScore() [compute 0-100 score from metrics]
+  ↓ scoreSignalQuality({ ..., timeframe }) [compute 0-100 score with timeframe-aware thresholds]
  ↓ createTrade() [save to database with signalQualityScore]
  ↓ positionManager.addTrade() [start monitoring]
 ```
@@ -333,6 +394,24 @@ docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\dt"

 14. **Per-symbol cooldown:** Cooldown period is per-symbol, NOT global. ETH trade at 10:00 does NOT block SOL trade at 10:01. Each coin (SOL/ETH/BTC) has independent cooldown timer to avoid missing opportunities on different assets.

+15. **Timeframe-aware scoring crucial:** Signal quality thresholds MUST adjust for 5min vs higher timeframes:
+    - 5min charts naturally have lower ADX (12-22 healthy) and ATR (0.2-0.7% healthy) than daily charts
+    - Without timeframe awareness, valid 5min breakouts get blocked as "low quality"
+    - Anti-chop filter applies -20 points for extreme sideways regardless of timeframe
+    - Always pass `timeframe` parameter from TradingView alerts to `scoreSignalQuality()`
+
+16. **Price position chasing causes flip-flops:** Opening longs at 90%+ range or shorts at <10% range reliably loses money:
+    - Database analysis showed overnight flip-flop losses all had price position 9-94% (chasing extremes)
+    - These trades had valid ADX (16-18) but entered at worst possible time
+    - Quality scoring now penalizes -15 to -30 points for range extremes
+    - Prevents rapid reversals when price is already overextended
+
+17. **TradingView ADX minimum for 5min:** Set ADX filter to 15 (not 20+) in TradingView alerts for 5min charts:
+    - Higher timeframes can use ADX 20+ for strong trends
+    - 5min charts need lower threshold to catch valid breakouts
+    - Bot's quality scoring provides second-layer filtering with context-aware metrics
+    - Two-stage filtering (TradingView + bot) prevents both overtrading and missing valid signals
+
 ## File Conventions

 - **API routes:** `app/api/[feature]/[action]/route.ts` (Next.js 15 App Router)
@@ -382,8 +461,9 @@ if (!enabled) {
 3. **Changing order logic:** Test with DRY_RUN=true first, use small position sizes ($10)
 4. **API endpoint changes:** Update both endpoint + corresponding n8n workflow JSON (Check Risk and Execute Trade nodes)
 5. **Docker changes:** Rebuild with `docker compose build trading-bot` then restart container
-6. **Modifying quality score logic:** Update BOTH `/api/trading/check-risk` and `/api/trading/execute` endpoints
+6. **Modifying quality score logic:** Update BOTH `/api/trading/check-risk` and `/api/trading/execute` endpoints, ensure timeframe-aware thresholds are synchronized
 7. **Exit strategy changes:** Modify Position Manager logic + update on-chain order placement in `placeExitOrders()`
+8. **TradingView alert changes:** Ensure alerts pass `timeframe` field (e.g., `"timeframe": "5"`) to enable proper signal quality scoring

 ## Development Roadmap

--- a/app/api/trading/execute/route.ts
+++ b/app/api/trading/execute/route.ts
@@ -347,6 +347,7 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
          volumeAtEntry: body.volumeRatio,
          pricePositionAtEntry: body.pricePosition,
          signalQualityScore: qualityResult.score,
+          signalQualityVersion: 'v3', // Stricter logic with ADX > 18 requirement
          // Phantom-specific fields
          status: 'phantom',
          isPhantom: true,
@@ -584,6 +585,7 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
        volumeAtEntry: body.volumeRatio,
        pricePositionAtEntry: body.pricePosition,
        signalQualityScore: qualityResult.score,
+        signalQualityVersion: 'v3', // Stricter logic with ADX > 18 requirement for extreme positions
        expectedSizeUSD: requestedPositionSizeUSD,
        actualSizeUSD: actualPositionSizeUSD,
      })
--- a/lib/database/trades.ts
+++ b/lib/database/trades.ts
@@ -52,6 +52,7 @@ export interface CreateTradeParams {
  volumeAtEntry?: number
  pricePositionAtEntry?: number
  signalQualityScore?: number
+  signalQualityVersion?: string // Track which scoring logic version was used
  // Phantom trade fields
  status?: string
  isPhantom?: boolean
--- a/prisma/migrations/20251107114250_add_signal_quality_version/migration.sql
+++ b/prisma/migrations/20251107114250_add_signal_quality_version/migration.sql
@@ -0,0 +1,2 @@
+-- AlterTable
+ALTER TABLE "Trade" ADD COLUMN     "signalQualityVersion" TEXT DEFAULT 'v1';
--- a/prisma/schema.prisma
+++ b/prisma/schema.prisma
@@ -102,6 +102,12 @@ model Trade {
  signalStrength   String?  // "strong", "moderate", "weak"
  timeframe        String?  // "5", "15", "60"
  
+  // Signal quality logic version tracking
+  signalQualityVersion String? @default("v1") // Track which scoring logic was used
+  // v1: Original logic with price position < 5% threshold
+  // v2: Added volume compensation for low ADX (2025-11-07)
+  // v3: Stricter - price position < 15% requires ADX > 18 (2025-11-07)
+  
  // Status
  status           String   @default("open") // "open", "closed", "failed", "phantom"
  isTestTrade      Boolean  @default(false) // Flag test trades for exclusion from analytics