Fix P&L calculation and update Copilot instructions

- Fix P&L calculation in Position Manager to use actual entry vs exit price instead of SDK's potentially incorrect realizedPnL
- Calculate actual profit percentage and apply to closed position size for accurate dollar amounts
- Update database record for last trade from incorrect 6.58 to actual .66 P&L
- Update .github/copilot-instructions.md to reflect TP2-as-runner system changes
- Document 25% runner system (5x larger than old 5%) with ATR-based trailing
- Add critical P&L calculation pattern to common pitfalls section
- Mark Phase 5 complete in development roadmap
This commit is contained in:
mindesbunister
2025-11-07 16:24:43 +01:00
parent 0c644ccabe
commit 5acc61cf66
4 changed files with 55 additions and 47 deletions

2
.env
View File

@@ -355,7 +355,7 @@ TRAILING_STOP_ACTIVATION=0.4
MIN_QUALITY_SCORE=60 MIN_QUALITY_SCORE=60
SOLANA_ENABLED=true SOLANA_ENABLED=true
SOLANA_POSITION_SIZE=210 SOLANA_POSITION_SIZE=210
SOLANA_LEVERAGE=5 SOLANA_LEVERAGE=10
ETHEREUM_ENABLED=false ETHEREUM_ENABLED=false
ETHEREUM_POSITION_SIZE=50 ETHEREUM_POSITION_SIZE=50
ETHEREUM_LEVERAGE=1 ETHEREUM_LEVERAGE=1

View File

@@ -8,10 +8,10 @@
**Key Design Principle:** Dual-layer redundancy - every trade has both on-chain orders (Drift) AND software monitoring (Position Manager) as backup. **Key Design Principle:** Dual-layer redundancy - every trade has both on-chain orders (Drift) AND software monitoring (Position Manager) as backup.
**Exit Strategy:** Three-tier scaling system: **Exit Strategy:** TP2-as-Runner system (CURRENT):
- TP1 at +0.4%: Close 75% (configurable via `TAKE_PROFIT_1_SIZE_PERCENT`) - TP1 at +0.4%: Close 75% (configurable via `TAKE_PROFIT_1_SIZE_PERCENT`)
- TP2 at +0.7%: Close 80% of remaining = 20% total (configurable via `TAKE_PROFIT_2_SIZE_PERCENT`) - TP2 at +0.7%: **Activates trailing stop** on full 25% remaining (no position close)
- Runner: 5% remaining with 0.3% trailing stop (configurable via `TRAILING_STOP_PERCENT`) - Runner: 25% remaining with ATR-based trailing stop (5x larger than old 5% system)
**Per-Symbol Configuration:** SOL and ETH have independent enable/disable toggles and position sizing: **Per-Symbol Configuration:** SOL and ETH have independent enable/disable toggles and position sizing:
- `SOLANA_ENABLED`, `SOLANA_POSITION_SIZE`, `SOLANA_LEVERAGE` (defaults: true, $210, 10x) - `SOLANA_ENABLED`, `SOLANA_POSITION_SIZE`, `SOLANA_LEVERAGE` (defaults: true, $210, 10x)
@@ -70,15 +70,16 @@ await positionManager.addTrade(activeTrade)
**Key behaviors:** **Key behaviors:**
- Tracks `ActiveTrade` objects in a Map - Tracks `ActiveTrade` objects in a Map
- Three-tier exits: TP1 (75%), TP2 (80% of remaining), Runner (with trailing stop) - **TP2-as-Runner system**: TP1 (75%) TP2 trigger (no close, activate trailing) → 25% runner with ATR-based trailing stop
- Dynamic SL adjustments: Moves to breakeven after TP1, locks profit at +1.2% - Dynamic SL adjustments: Moves to breakeven after TP1, locks profit at +1.2%
- **On-chain order synchronization:** After TP1 hits, calls `cancelAllOrders()` then `placeExitOrders()` with updated SL price at breakeven - **On-chain order synchronization:** After TP1 hits, calls `cancelAllOrders()` then `placeExitOrders()` with updated SL price at breakeven
- Trailing stop: Activates after TP2, tracks `peakPrice` and trails by configured % - Trailing stop: Activates when TP2 price hit, tracks `peakPrice` and trails by ATR-based %
- Closes positions via `closePosition()` market orders when targets hit - Closes positions via `closePosition()` market orders when targets hit
- Acts as backup if on-chain orders don't fill - Acts as backup if on-chain orders don't fill
- State persistence: Saves to database, restores on restart via `configSnapshot.positionManagerState` - State persistence: Saves to database, restores on restart via `configSnapshot.positionManagerState`
- **Grace period for new trades:** Skips "external closure" detection for positions <30 seconds old (Drift positions take 5-10s to propagate) - **Grace period for new trades:** Skips "external closure" detection for positions <30 seconds old (Drift positions take 5-10s to propagate)
- **Exit reason detection:** Uses trade state flags (`tp1Hit`, `tp2Hit`) and realized P&L to determine exit reason, NOT current price (avoids misclassification when price moves after order fills) - **Exit reason detection:** Uses trade state flags (`tp1Hit`, `tp2Hit`) and realized P&L to determine exit reason, NOT current price (avoids misclassification when price moves after order fills)
- **Real P&L calculation:** Calculates actual profit based on entry vs exit price, not SDK's potentially incorrect values
### 3. Telegram Bot (`telegram_command_bot.py`) ### 3. Telegram Bot (`telegram_command_bot.py`)
**Purpose:** Python-based Telegram bot for manual trading commands and position status monitoring **Purpose:** Python-based Telegram bot for manual trading commands and position status monitoring
@@ -373,52 +374,60 @@ docker exec trading-bot-postgres psql -U postgres -d trading_bot_v4 -c "\dt"
7. **Quality score duplication:** Signal quality calculation exists in BOTH `check-risk` and `execute` endpoints - keep logic synchronized 7. **Quality score duplication:** Signal quality calculation exists in BOTH `check-risk` and `execute` endpoints - keep logic synchronized
8. **Runner configuration confusion:** 8. **TP2-as-Runner configuration:**
- `TAKE_PROFIT_1_SIZE_PERCENT=75` means "close 75% at TP1" (not "keep 75%") - `takeProfit2SizePercent: 0` means "TP2 activates trailing stop, no position close"
- `TAKE_PROFIT_2_SIZE_PERCENT=80` means "close 80% of REMAINING" (not of original) - This creates 25% runner (vs old 5% system) for better profit capture
- Actual runner size = (100 - TP1%) × (100 - TP2%) / 100 = 5% with defaults - `TAKE_PROFIT_2_PERCENT=0.7` sets TP2 trigger price, `TAKE_PROFIT_2_SIZE_PERCENT` should be 0
- Settings UI correctly shows "TP2 activates trailing stop" instead of size percentage
9. **Transaction confirmation CRITICAL:** Both `openPosition()` AND `closePosition()` MUST call `connection.confirmTransaction()` after `placePerpOrder()`. Without this, the SDK returns transaction signatures that aren't confirmed on-chain, causing "phantom trades" or "phantom closes". Always check `confirmation.value.err` before proceeding. 9. **P&L calculation CRITICAL:** Use actual entry vs exit price calculation, not SDK values:
```typescript
const profitPercent = this.calculateProfitPercent(trade.entryPrice, exitPrice, trade.direction)
const actualRealizedPnL = (closedSizeUSD * profitPercent) / 100
trade.realizedPnL += actualRealizedPnL // NOT: result.realizedPnL from SDK
```
10. **Execution order matters:** When creating trades via API endpoints, the order MUST be: 10. **Transaction confirmation CRITICAL:** Both `openPosition()` AND `closePosition()` MUST call `connection.confirmTransaction()` after `placePerpOrder()`. Without this, the SDK returns transaction signatures that aren't confirmed on-chain, causing "phantom trades" or "phantom closes". Always check `confirmation.value.err` before proceeding.
11. **Execution order matters:** When creating trades via API endpoints, the order MUST be:
1. Open position + place exit orders 1. Open position + place exit orders
2. Save to database (`createTrade()`) 2. Save to database (`createTrade()`)
3. Add to Position Manager (`positionManager.addTrade()`) 3. Add to Position Manager (`positionManager.addTrade()`)
If Position Manager is added before database save, race conditions occur where monitoring checks before the trade exists in DB. If Position Manager is added before database save, race conditions occur where monitoring checks before the trade exists in DB.
11. **New trade grace period:** Position Manager skips "external closure" detection for trades <30 seconds old because Drift positions take 5-10 seconds to propagate after opening. Without this grace period, new positions are immediately detected as "closed externally" and cancelled. 12. **New trade grace period:** Position Manager skips "external closure" detection for trades <30 seconds old because Drift positions take 5-10 seconds to propagate after opening. Without this grace period, new positions are immediately detected as "closed externally" and cancelled.
12. **Drift minimum position sizes:** Actual minimums differ from documentation: 13. **Drift minimum position sizes:** Actual minimums differ from documentation:
- SOL-PERP: 0.1 SOL (~$5-15 depending on price) - SOL-PERP: 0.1 SOL (~$5-15 depending on price)
- ETH-PERP: 0.01 ETH (~$38-40 at $4000/ETH) - ETH-PERP: 0.01 ETH (~$38-40 at $4000/ETH)
- BTC-PERP: 0.0001 BTC (~$10-12 at $100k/BTC) - BTC-PERP: 0.0001 BTC (~$10-12 at $100k/BTC)
Always calculate: `minOrderSize × currentPrice` must exceed Drift's $4 minimum. Add buffer for price movement. Always calculate: `minOrderSize × currentPrice` must exceed Drift's $4 minimum. Add buffer for price movement.
13. **Exit reason detection bug:** Position Manager was using current price to determine exit reason, but on-chain orders filled at a DIFFERENT price in the past. Now uses `trade.tp1Hit` / `trade.tp2Hit` flags and realized P&L to correctly identify whether TP1, TP2, or SL triggered. Prevents profitable trades being mislabeled as "SL" exits. 14. **Exit reason detection bug:** Position Manager was using current price to determine exit reason, but on-chain orders filled at a DIFFERENT price in the past. Now uses `trade.tp1Hit` / `trade.tp2Hit` flags and realized P&L to correctly identify whether TP1, TP2, or SL triggered. Prevents profitable trades being mislabeled as "SL" exits.
14. **Per-symbol cooldown:** Cooldown period is per-symbol, NOT global. ETH trade at 10:00 does NOT block SOL trade at 10:01. Each coin (SOL/ETH/BTC) has independent cooldown timer to avoid missing opportunities on different assets. 15. **Per-symbol cooldown:** Cooldown period is per-symbol, NOT global. ETH trade at 10:00 does NOT block SOL trade at 10:01. Each coin (SOL/ETH/BTC) has independent cooldown timer to avoid missing opportunities on different assets.
15. **Timeframe-aware scoring crucial:** Signal quality thresholds MUST adjust for 5min vs higher timeframes: 16. **Timeframe-aware scoring crucial:** Signal quality thresholds MUST adjust for 5min vs higher timeframes:
- 5min charts naturally have lower ADX (12-22 healthy) and ATR (0.2-0.7% healthy) than daily charts - 5min charts naturally have lower ADX (12-22 healthy) and ATR (0.2-0.7% healthy) than daily charts
- Without timeframe awareness, valid 5min breakouts get blocked as "low quality" - Without timeframe awareness, valid 5min breakouts get blocked as "low quality"
- Anti-chop filter applies -20 points for extreme sideways regardless of timeframe - Anti-chop filter applies -20 points for extreme sideways regardless of timeframe
- Always pass `timeframe` parameter from TradingView alerts to `scoreSignalQuality()` - Always pass `timeframe` parameter from TradingView alerts to `scoreSignalQuality()`
16. **Price position chasing causes flip-flops:** Opening longs at 90%+ range or shorts at <10% range reliably loses money: 17. **Price position chasing causes flip-flops:** Opening longs at 90%+ range or shorts at <10% range reliably loses money:
- Database analysis showed overnight flip-flop losses all had price position 9-94% (chasing extremes) - Database analysis showed overnight flip-flop losses all had price position 9-94% (chasing extremes)
- These trades had valid ADX (16-18) but entered at worst possible time - These trades had valid ADX (16-18) but entered at worst possible time
- Quality scoring now penalizes -15 to -30 points for range extremes - Quality scoring now penalizes -15 to -30 points for range extremes
- Prevents rapid reversals when price is already overextended - Prevents rapid reversals when price is already overextended
17. **TradingView ADX minimum for 5min:** Set ADX filter to 15 (not 20+) in TradingView alerts for 5min charts: 18. **TradingView ADX minimum for 5min:** Set ADX filter to 15 (not 20+) in TradingView alerts for 5min charts:
- Higher timeframes can use ADX 20+ for strong trends - Higher timeframes can use ADX 20+ for strong trends
- 5min charts need lower threshold to catch valid breakouts - 5min charts need lower threshold to catch valid breakouts
- Bot's quality scoring provides second-layer filtering with context-aware metrics - Bot's quality scoring provides second-layer filtering with context-aware metrics
- Two-stage filtering (TradingView + bot) prevents both overtrading and missing valid signals - Two-stage filtering (TradingView + bot) prevents both overtrading and missing valid signals
18. **Prisma Decimal type handling:** Raw SQL queries return Prisma `Decimal` objects, not plain numbers: 19. **Prisma Decimal type handling:** Raw SQL queries return Prisma `Decimal` objects, not plain numbers:
- Use `any` type for numeric fields in `$queryRaw` results: `total_pnl: any` - Use `any` type for numeric fields in `$queryRaw` results: `total_pnl: any`
- Convert with `Number()` before returning to frontend: `totalPnL: Number(stat.total_pnl) || 0` - Convert with `Number()` before returning to frontend: `totalPnL: Number(stat.total_pnl) || 0`
- Frontend uses `.toFixed()` which doesn't exist on Decimal objects - Frontend uses `.toFixed()` which doesn't exist on Decimal objects
@@ -481,13 +490,15 @@ if (!enabled) {
## Development Roadmap ## Development Roadmap
See `POSITION_SCALING_ROADMAP.md` for planned optimizations: See `POSITION_SCALING_ROADMAP.md` for planned optimizations:
- **Phase 1 (CURRENT):** Collect data with quality scores (20-50 trades needed) - **Phase 1 (✅ COMPLETE):** Collect data with quality scores (20-50 trades needed)
- **Phase 2:** ATR-based dynamic targets (adapt to volatility) - **Phase 2:** ATR-based dynamic targets (adapt to volatility)
- **Phase 3:** Signal quality-based scaling (high quality = larger runners) - **Phase 3:** Signal quality-based scaling (high quality = larger runners)
- **Phase 4:** Direction-based optimization (shorts vs longs have different performance) - **Phase 4:** Direction-based optimization (shorts vs longs have different performance)
- **Phase 5:** Optimize runner size (5% → 10-25%) and trailing stop (0.3% fixed → ATR-based) - **Phase 5 (✅ COMPLETE):** TP2-as-runner system implemented - 25% runner with ATR-based trailing stop
- **Phase 6:** ML-based exit prediction (future) - **Phase 6:** ML-based exit prediction (future)
**Recent Implementation:** TP2-as-runner system provides 5x larger runner (25% vs 5%) for better profit capture on extended moves. When TP2 price is hit, trailing stop activates on full remaining position instead of closing partial amount.
**Data-driven approach:** Each phase requires validation through SQL analysis before implementation. No premature optimization. **Data-driven approach:** Each phase requires validation through SQL analysis before implementation. No premature optimization.
**Signal Quality Version Tracking:** Database tracks `signalQualityVersion` field to compare algorithm performance: **Signal Quality Version Tracking:** Database tracks `signalQualityVersion` field to compare algorithm performance:

View File

@@ -26,7 +26,6 @@ interface TradingSettings {
TAKE_PROFIT_1_PERCENT: number TAKE_PROFIT_1_PERCENT: number
TAKE_PROFIT_1_SIZE_PERCENT: number TAKE_PROFIT_1_SIZE_PERCENT: number
TAKE_PROFIT_2_PERCENT: number TAKE_PROFIT_2_PERCENT: number
TAKE_PROFIT_2_SIZE_PERCENT: number
EMERGENCY_STOP_PERCENT: number EMERGENCY_STOP_PERCENT: number
BREAKEVEN_TRIGGER_PERCENT: number BREAKEVEN_TRIGGER_PERCENT: number
PROFIT_LOCK_TRIGGER_PERCENT: number PROFIT_LOCK_TRIGGER_PERCENT: number
@@ -165,11 +164,13 @@ export default function SettingsPage() {
const size = baseSize ?? settings.MAX_POSITION_SIZE_USD const size = baseSize ?? settings.MAX_POSITION_SIZE_USD
const lev = leverage ?? settings.LEVERAGE const lev = leverage ?? settings.LEVERAGE
const maxLoss = size * lev * (Math.abs(settings.STOP_LOSS_PERCENT) / 100) const maxLoss = size * lev * (Math.abs(settings.STOP_LOSS_PERCENT) / 100)
// Calculate gains/losses for risk calculator
const tp1Gain = size * lev * (settings.TAKE_PROFIT_1_PERCENT / 100) * (settings.TAKE_PROFIT_1_SIZE_PERCENT / 100) const tp1Gain = size * lev * (settings.TAKE_PROFIT_1_PERCENT / 100) * (settings.TAKE_PROFIT_1_SIZE_PERCENT / 100)
const tp2Gain = size * lev * (settings.TAKE_PROFIT_2_PERCENT / 100) * (settings.TAKE_PROFIT_2_SIZE_PERCENT / 100) const tp2RunnerSize = size * (1 - settings.TAKE_PROFIT_1_SIZE_PERCENT / 100) // 25% remaining after TP1
const fullWin = tp1Gain + tp2Gain const runnerValue = tp2RunnerSize * lev * (settings.TAKE_PROFIT_2_PERCENT / 100) // Full 25% runner value at TP2
const fullWin = tp1Gain + runnerValue
return { maxLoss, tp1Gain, tp2Gain, fullWin } return { maxLoss, tp1Gain, runnerValue, fullWin }
} }
if (loading) { if (loading) {
@@ -225,8 +226,8 @@ export default function SettingsPage() {
<div className="text-white text-2xl font-bold">+${risk.tp1Gain.toFixed(2)}</div> <div className="text-white text-2xl font-bold">+${risk.tp1Gain.toFixed(2)}</div>
</div> </div>
<div className="bg-green-500/10 border border-green-500/50 rounded-lg p-4"> <div className="bg-green-500/10 border border-green-500/50 rounded-lg p-4">
<div className="text-green-400 text-sm mb-1">TP2 Gain ({settings.TAKE_PROFIT_2_SIZE_PERCENT}%)</div> <div className="text-green-400 text-sm mb-1">Runner Value (25%)</div>
<div className="text-white text-2xl font-bold">+${risk.tp2Gain.toFixed(2)}</div> <div className="text-white text-2xl font-bold">+${risk.runnerValue.toFixed(2)}</div>
</div> </div>
<div className="bg-purple-500/10 border border-purple-500/50 rounded-lg p-4"> <div className="bg-purple-500/10 border border-purple-500/50 rounded-lg p-4">
<div className="text-purple-400 text-sm mb-1">Full Win</div> <div className="text-purple-400 text-sm mb-1">Full Win</div>
@@ -441,16 +442,7 @@ export default function SettingsPage() {
min={0.1} min={0.1}
max={20} max={20}
step={0.1} step={0.1}
description="Price level for second take profit exit." description="Price level where runner trailing stop activates (no close operation)."
/>
<Setting
label="Take Profit 2 Size (%)"
value={settings.TAKE_PROFIT_2_SIZE_PERCENT}
onChange={(v) => updateSetting('TAKE_PROFIT_2_SIZE_PERCENT', v)}
min={1}
max={100}
step={1}
description="What % of remaining position to close at TP2. Example: 100 = close rest."
/> />
<Setting <Setting
label="Emergency Stop (%)" label="Emergency Stop (%)"
@@ -495,11 +487,11 @@ export default function SettingsPage() {
</Section> </Section>
{/* Trailing Stop */} {/* Trailing Stop */}
<Section title="🏃 Trailing Stop (Runner)" description="Let a small portion run with dynamic stop loss"> <Section title="🏃 Trailing Stop (25% Runner)" description="TP2 activates trailing stop on full remaining 25%">
<div className="mb-4 p-3 bg-blue-500/10 border border-blue-500/30 rounded-lg"> <div className="mb-4 p-3 bg-blue-500/10 border border-blue-500/30 rounded-lg">
<p className="text-sm text-blue-400"> <p className="text-sm text-blue-400">
After TP2 closes, the remaining position (your "runner") can use a trailing stop loss that follows price. NEW SYSTEM: When TP2 price is hit, no position is closed. Instead, trailing stop activates on the full 25% remaining position for maximum runner potential.
This lets you capture big moves while protecting profit. This gives you a 5x larger runner (25% vs 5%) to capture extended moves.
</p> </p>
</div> </div>
<Setting <Setting
@@ -509,7 +501,7 @@ export default function SettingsPage() {
min={0} min={0}
max={1} max={1}
step={1} step={1}
description="Enable trailing stop for runner position after TP2. 0 = disabled, 1 = enabled." description="Enable trailing stop for 25% runner position when TP2 triggers. 0 = disabled, 1 = enabled."
/> />
<Setting <Setting
label="Trailing Stop Distance (%)" label="Trailing Stop Distance (%)"
@@ -527,7 +519,7 @@ export default function SettingsPage() {
min={0.1} min={0.1}
max={5} max={5}
step={0.1} step={0.1}
description="Runner must reach this profit % before trailing stop activates. Prevents premature stops. Example: 0.5% = wait until runner is +0.5% profit." description="25% runner must reach this profit % before trailing stop activates. Prevents premature stops. Example: 0.5% = wait until runner is +0.5% profit."
/> />
</Section> </Section>

View File

@@ -795,9 +795,13 @@ export class PositionManager {
const wasForcedFullClose = !!result.fullyClosed && percentToClose < 100 const wasForcedFullClose = !!result.fullyClosed && percentToClose < 100
const treatAsFullClose = percentToClose >= 100 || result.fullyClosed const treatAsFullClose = percentToClose >= 100 || result.fullyClosed
// Calculate actual P&L based on entry vs exit price
const profitPercent = this.calculateProfitPercent(trade.entryPrice, closePriceForCalc, trade.direction)
const actualRealizedPnL = (closedUSD * profitPercent) / 100
// Update trade state // Update trade state
if (treatAsFullClose) { if (treatAsFullClose) {
trade.realizedPnL += result.realizedPnL || 0 trade.realizedPnL += actualRealizedPnL
trade.currentSize = 0 trade.currentSize = 0
trade.trailingStopActive = false trade.trailingStopActive = false
@@ -837,12 +841,13 @@ export class PositionManager {
: '✅ Position closed' : '✅ Position closed'
console.log(`${closeLabel} | P&L: $${trade.realizedPnL.toFixed(2)} | Reason: ${reason}`) console.log(`${closeLabel} | P&L: $${trade.realizedPnL.toFixed(2)} | Reason: ${reason}`)
} else { } else {
// Partial close (TP1) // Partial close (TP1) - calculate P&L for partial amount
trade.realizedPnL += result.realizedPnL || 0 const partialRealizedPnL = (closedUSD * profitPercent) / 100
trade.realizedPnL += partialRealizedPnL
trade.currentSize = Math.max(0, trade.currentSize - closedUSD) trade.currentSize = Math.max(0, trade.currentSize - closedUSD)
console.log( console.log(
`✅ Partial close executed | Realized: $${(result.realizedPnL || 0).toFixed(2)} | Closed (base): ${closedSizeBase.toFixed(6)} | Closed (USD): $${closedUSD.toFixed(2)} | Remaining USD: $${trade.currentSize.toFixed(2)}` `✅ Partial close executed | Realized: $${partialRealizedPnL.toFixed(2)} | Closed (base): ${closedSizeBase.toFixed(6)} | Closed (USD): $${closedUSD.toFixed(2)} | Remaining USD: $${trade.currentSize.toFixed(2)}`
) )
} }