feat: Add signalSource field to identify manual vs TradingView trades
- Set signalSource='manual' for Telegram trades, 'tradingview' for TradingView - Updated analytics queries to exclude manual trades from indicator analysis - getTradingStats() filters manual trades (TradingView performance only) - Version comparison endpoint filters manual trades - Created comprehensive filtering guide: docs/MANUAL_TRADE_FILTERING.md - Ensures clean data for indicator optimization without contamination
This commit is contained in:
@@ -56,6 +56,7 @@ export async function GET() {
|
|||||||
WHERE "exitReason" IS NOT NULL
|
WHERE "exitReason" IS NOT NULL
|
||||||
AND "exitReason" NOT LIKE '%CLEANUP%'
|
AND "exitReason" NOT LIKE '%CLEANUP%'
|
||||||
AND "isTestTrade" = false
|
AND "isTestTrade" = false
|
||||||
|
AND ("signalSource" IS NULL OR "signalSource" != 'manual')
|
||||||
GROUP BY "indicatorVersion"
|
GROUP BY "indicatorVersion"
|
||||||
ORDER BY version DESC
|
ORDER BY version DESC
|
||||||
`
|
`
|
||||||
@@ -78,6 +79,7 @@ export async function GET() {
|
|||||||
WHERE "exitReason" IS NOT NULL
|
WHERE "exitReason" IS NOT NULL
|
||||||
AND "exitReason" NOT LIKE '%CLEANUP%'
|
AND "exitReason" NOT LIKE '%CLEANUP%'
|
||||||
AND "isTestTrade" = false
|
AND "isTestTrade" = false
|
||||||
|
AND ("signalSource" IS NULL OR "signalSource" != 'manual')
|
||||||
AND "pricePositionAtEntry" < 15
|
AND "pricePositionAtEntry" < 15
|
||||||
GROUP BY "indicatorVersion"
|
GROUP BY "indicatorVersion"
|
||||||
ORDER BY version DESC
|
ORDER BY version DESC
|
||||||
|
|||||||
@@ -650,6 +650,7 @@ export async function POST(request: NextRequest): Promise<NextResponse<ExecuteTr
|
|||||||
hardStopOrderTx: config.useDualStops ? exitOrderSignatures[3] : undefined,
|
hardStopOrderTx: config.useDualStops ? exitOrderSignatures[3] : undefined,
|
||||||
softStopPrice,
|
softStopPrice,
|
||||||
hardStopPrice,
|
hardStopPrice,
|
||||||
|
signalSource: body.timeframe === 'manual' ? 'manual' : 'tradingview', // Identify manual Telegram trades
|
||||||
signalStrength: body.signalStrength,
|
signalStrength: body.signalStrength,
|
||||||
timeframe: body.timeframe,
|
timeframe: body.timeframe,
|
||||||
// Context metrics from TradingView
|
// Context metrics from TradingView
|
||||||
|
|||||||
231
docs/MANUAL_TRADE_FILTERING.md
Normal file
231
docs/MANUAL_TRADE_FILTERING.md
Normal file
@@ -0,0 +1,231 @@
|
|||||||
|
# Manual Trade Filtering Guide
|
||||||
|
|
||||||
|
**Date:** November 14, 2025
|
||||||
|
**Purpose:** Ensure TradingView indicator analysis excludes manual Telegram trades
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Manual trades entered via Telegram bot use preset metrics and should **NOT** be included in TradingView indicator performance analysis. Including them would contaminate the data and make indicator optimization impossible.
|
||||||
|
|
||||||
|
## Database Fields
|
||||||
|
|
||||||
|
Trades from different sources are identified by:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
signalSource VARCHAR -- 'tradingview', 'manual', or NULL (old trades)
|
||||||
|
timeframe VARCHAR -- 'manual' for Telegram trades, '5'/'15'/'60' for TradingView
|
||||||
|
isTestTrade BOOLEAN -- Test trades from settings UI
|
||||||
|
```
|
||||||
|
|
||||||
|
## Filtering Pattern
|
||||||
|
|
||||||
|
**ALWAYS use this WHERE clause for indicator analysis:**
|
||||||
|
|
||||||
|
```sql
|
||||||
|
WHERE
|
||||||
|
"isTestTrade" = false -- Exclude test trades
|
||||||
|
AND ("signalSource" IS NULL OR "signalSource" != 'manual') -- Exclude manual trades
|
||||||
|
AND "exitReason" IS NOT NULL -- Closed trades only
|
||||||
|
AND "exitReason" NOT LIKE '%CLEANUP%' -- Exclude cleanup trades
|
||||||
|
```
|
||||||
|
|
||||||
|
## Common Analysis Queries (CORRECT)
|
||||||
|
|
||||||
|
### Win Rate by Indicator Version
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT
|
||||||
|
COALESCE("indicatorVersion", 'unknown') as version,
|
||||||
|
COUNT(*) as total_trades,
|
||||||
|
SUM(CASE WHEN "realizedPnL" > 0 THEN 1 ELSE 0 END) as wins,
|
||||||
|
ROUND(100.0 * SUM(CASE WHEN "realizedPnL" > 0 THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate,
|
||||||
|
ROUND(SUM("realizedPnL")::numeric, 2) as total_pnl,
|
||||||
|
ROUND(AVG("realizedPnL")::numeric, 2) as avg_pnl
|
||||||
|
FROM "Trade"
|
||||||
|
WHERE
|
||||||
|
"isTestTrade" = false
|
||||||
|
AND ("signalSource" IS NULL OR "signalSource" != 'manual')
|
||||||
|
AND "exitReason" IS NOT NULL
|
||||||
|
AND "exitReason" NOT LIKE '%CLEANUP%'
|
||||||
|
GROUP BY "indicatorVersion"
|
||||||
|
ORDER BY version DESC;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Signal Quality Score Distribution (TradingView Only)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT
|
||||||
|
CASE
|
||||||
|
WHEN "signalQualityScore" >= 70 THEN '70-79'
|
||||||
|
WHEN "signalQualityScore" >= 65 THEN '65-69'
|
||||||
|
WHEN "signalQualityScore" >= 60 THEN '60-64'
|
||||||
|
ELSE '<60'
|
||||||
|
END as score_tier,
|
||||||
|
COUNT(*) as trades,
|
||||||
|
ROUND(100.0 * SUM(CASE WHEN "realizedPnL" > 0 THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate,
|
||||||
|
ROUND(AVG("realizedPnL")::numeric, 2) as avg_pnl
|
||||||
|
FROM "Trade"
|
||||||
|
WHERE
|
||||||
|
"isTestTrade" = false
|
||||||
|
AND ("signalSource" IS NULL OR "signalSource" != 'manual')
|
||||||
|
AND "exitReason" IS NOT NULL
|
||||||
|
AND "signalQualityScore" IS NOT NULL
|
||||||
|
GROUP BY score_tier
|
||||||
|
ORDER BY MIN("signalQualityScore") DESC;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Extreme Position Analysis (TradingView Only)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Trades at range extremes (< 15% price position)
|
||||||
|
SELECT
|
||||||
|
"direction",
|
||||||
|
COUNT(*) as trades,
|
||||||
|
ROUND(100.0 * SUM(CASE WHEN "realizedPnL" > 0 THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate,
|
||||||
|
ROUND(AVG("adxAtEntry")::numeric, 1) as avg_adx,
|
||||||
|
ROUND(AVG("pricePositionAtEntry")::numeric, 1) as avg_price_pos,
|
||||||
|
ROUND(SUM("realizedPnL")::numeric, 2) as total_pnl
|
||||||
|
FROM "Trade"
|
||||||
|
WHERE
|
||||||
|
"isTestTrade" = false
|
||||||
|
AND ("signalSource" IS NULL OR "signalSource" != 'manual')
|
||||||
|
AND "exitReason" IS NOT NULL
|
||||||
|
AND "pricePositionAtEntry" < 15
|
||||||
|
GROUP BY "direction";
|
||||||
|
```
|
||||||
|
|
||||||
|
### Timeframe Performance (TradingView Only)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT
|
||||||
|
"timeframe",
|
||||||
|
COUNT(*) as trades,
|
||||||
|
ROUND(100.0 * SUM(CASE WHEN "realizedPnL" > 0 THEN 1 ELSE 0 END) / COUNT(*), 1) as win_rate,
|
||||||
|
ROUND(AVG("signalQualityScore")::numeric, 1) as avg_score,
|
||||||
|
ROUND(SUM("realizedPnL")::numeric, 2) as total_pnl
|
||||||
|
FROM "Trade"
|
||||||
|
WHERE
|
||||||
|
"isTestTrade" = false
|
||||||
|
AND ("signalSource" IS NULL OR "signalSource" != 'manual')
|
||||||
|
AND "exitReason" IS NOT NULL
|
||||||
|
AND "timeframe" IS NOT NULL
|
||||||
|
GROUP BY "timeframe"
|
||||||
|
ORDER BY timeframe;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Queries INCLUDING Manual Trades
|
||||||
|
|
||||||
|
Sometimes you DO want to see manual trades (e.g., overall account performance):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ALL trades (manual + TradingView)
|
||||||
|
SELECT
|
||||||
|
"signalSource",
|
||||||
|
COUNT(*) as trades,
|
||||||
|
ROUND(SUM("realizedPnL")::numeric, 2) as total_pnl
|
||||||
|
FROM "Trade"
|
||||||
|
WHERE
|
||||||
|
"isTestTrade" = false
|
||||||
|
AND "exitReason" IS NOT NULL
|
||||||
|
GROUP BY "signalSource"
|
||||||
|
ORDER BY "signalSource";
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```
|
||||||
|
signalSource | trades | total_pnl
|
||||||
|
-------------|--------|----------
|
||||||
|
tradingview | 150 | +45.23
|
||||||
|
manual | 11 | -12.56
|
||||||
|
(null) | 10 | +8.90 (old trades before signalSource added)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Application Code
|
||||||
|
|
||||||
|
### TypeScript/Prisma Queries
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// TradingView indicator analysis only
|
||||||
|
const trades = await prisma.trade.findMany({
|
||||||
|
where: {
|
||||||
|
isTestTrade: false,
|
||||||
|
OR: [
|
||||||
|
{ signalSource: null }, // Old trades
|
||||||
|
{ signalSource: { not: 'manual' } }, // Exclude Telegram
|
||||||
|
],
|
||||||
|
exitReason: { not: null },
|
||||||
|
NOT: {
|
||||||
|
exitReason: { contains: 'CLEANUP' },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
// All real trades (including manual)
|
||||||
|
const allTrades = await prisma.trade.findMany({
|
||||||
|
where: {
|
||||||
|
isTestTrade: false,
|
||||||
|
exitReason: { not: null },
|
||||||
|
},
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration for Old Trades
|
||||||
|
|
||||||
|
Old trades don't have `signalSource` set (NULL). The filter `OR signalSource IS NULL` includes them in TradingView analysis, which is correct because they came from TradingView before the field existed.
|
||||||
|
|
||||||
|
To backfill old trades:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Set signalSource for old trades (if timeframe is known)
|
||||||
|
UPDATE "Trade"
|
||||||
|
SET "signalSource" = CASE
|
||||||
|
WHEN "timeframe" = 'manual' THEN 'manual'
|
||||||
|
WHEN "timeframe" IS NOT NULL AND "timeframe" != 'manual' THEN 'tradingview'
|
||||||
|
ELSE 'tradingview' -- Default assumption for old trades
|
||||||
|
END
|
||||||
|
WHERE "signalSource" IS NULL;
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note:** Only run this if you want to clean up old data. Current filter handles NULL correctly.
|
||||||
|
|
||||||
|
## Implementation Status
|
||||||
|
|
||||||
|
**Updated files (Nov 14, 2025):**
|
||||||
|
- ✅ `app/api/trading/execute/route.ts` - Sets `signalSource` on new trades
|
||||||
|
- ✅ `app/api/analytics/version-comparison/route.ts` - Filters manual trades in SQL
|
||||||
|
- ✅ `lib/database/views.ts` - Filters manual trades in `getTradingStats()`
|
||||||
|
|
||||||
|
**Future updates needed:**
|
||||||
|
- Analytics dashboard frontend (if it queries trades directly)
|
||||||
|
- Any custom SQL analysis scripts in `scripts/` directory
|
||||||
|
- Performance analysis queries in optimization roadmaps
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Verify filtering works:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Should return 0 (no manual trades in TradingView analysis)
|
||||||
|
SELECT COUNT(*)
|
||||||
|
FROM "Trade"
|
||||||
|
WHERE
|
||||||
|
"isTestTrade" = false
|
||||||
|
AND ("signalSource" IS NULL OR "signalSource" != 'manual')
|
||||||
|
AND "exitReason" IS NOT NULL
|
||||||
|
AND "signalSource" = 'manual';
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected result: `0` (manual trades correctly excluded)
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
**Rule of thumb:**
|
||||||
|
- **Indicator optimization:** Exclude manual trades (use filter above)
|
||||||
|
- **Account performance:** Include all real trades (exclude only `isTestTrade`)
|
||||||
|
- **Position sizing analysis:** Can include manual trades (they use same TP/SL system)
|
||||||
|
|
||||||
|
**When in doubt:** Use the standard filter shown at the top of this document.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*This filtering ensures clean data for TradingView indicator analysis while preserving manual trades for account-level reporting.*
|
||||||
@@ -143,6 +143,10 @@ export async function getTradingStats(days: number = 30) {
|
|||||||
createdAt: { gte: since },
|
createdAt: { gte: since },
|
||||||
status: 'closed',
|
status: 'closed',
|
||||||
isTestTrade: false, // Real trades only
|
isTestTrade: false, // Real trades only
|
||||||
|
OR: [
|
||||||
|
{ signalSource: null }, // Old trades without signalSource
|
||||||
|
{ signalSource: { not: 'manual' } }, // Exclude manual Telegram trades
|
||||||
|
],
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user