feat: add Nextcloud Deck roadmap sync system

- Create discover-deck-ids.sh to find board/stack configuration - Implement sync-roadmap-to-deck.py for roadmap → Deck sync - Parse OPTIMIZATION_MASTER_ROADMAP.md and extract initiatives - Map roadmap status to Deck stacks (eingang/planung/arbeit/erledigt) - Create cards with titles, descriptions, due dates, progress - Support dry-run mode for testing before actual sync - Add comprehensive documentation in NEXTCLOUD_DECK_SYNC.md **Benefits:** - Visual kanban board for roadmap management - Drag & drop to prioritize tasks - Single source of truth (markdown files) - Easy task tracking and status updates - No manual duplication between systems **Initial Sync:** - Created 1 card: Initiative 1 (Signal Quality Optimization) - Placed in 'eingang' (FUTURE status) **Future Work:** - Bidirectional sync (Deck → Roadmap) - Phase-level cards parsing - Manual card creation → roadmap entry - Automated cron sync
2025-11-14 11:09:37 +01:00
parent a0dc80e96b
commit 77a22bae3f
3 changed files with 521 additions and 0 deletions
--- a/docs/NEXTCLOUD_DECK_SYNC.md
+++ b/docs/NEXTCLOUD_DECK_SYNC.md
@@ -0,0 +1,146 @@
+# Nextcloud Deck Roadmap Sync
+
+Bidirectional sync system between trading bot roadmaps (markdown files) and Nextcloud Deck kanban board.
+
+## Setup
+
+### 1. Discover Board Configuration
+
+First time setup - find your Nextcloud Deck board and stack IDs:
+
+```bash
+./scripts/discover-deck-ids.sh
+```
+
+This creates `/tmp/deck-config.json` with your board configuration.
+
+### 2. Initialize Deck with Roadmap Cards
+
+Create Deck cards from current roadmap:
+
+```bash
+python3 scripts/sync-roadmap-to-deck.py --init
+```
+
+This will:
+- Parse all roadmap files (OPTIMIZATION_MASTER_ROADMAP.md, etc.)
+- Create cards for each initiative/phase
+- Place cards in appropriate stacks based on status
+- Set due dates based on timelines
+
+## Stack Mapping
+
+| Deck Stack | Roadmap Status | Purpose |
+|------------|----------------|---------|
+| `eingang` (inbox) | FUTURE | Backlog items, ideas, future phases |
+| `in planung` (planning) | PENDING | Ready to implement, needs detailed specs |
+| `in arbeit` (in progress) | IN PROGRESS (🔄) | Currently working on |
+| `erledigt` (done) | COMPLETE (✅) | Finished and verified |
+
+## Usage
+
+### Creating Roadmap Cards
+
+**Option 1: Update markdown, sync to Deck**
+1. Edit roadmap markdown files
+2. Run: `python3 scripts/sync-roadmap-to-deck.py --init`
+3. New initiatives/phases appear as Deck cards
+
+**Option 2: Create card in Deck (future feature)**
+1. Create card in Deck "eingang" stack
+2. Title format: `[ROADMAP] Initiative Name`
+3. Script will parse and add to appropriate roadmap file
+
+### Updating Status
+
+**Current:** Manual sync
+- Move cards between stacks in Deck to reflect progress
+- Update markdown roadmap files with status emoji (🔄, ✅, etc.)
+- Re-sync to align
+
+**Future:** Automatic bidirectional sync
+- Script will update roadmap files based on card positions
+- Cron job could run sync every hour
+
+## Files
+
+- `scripts/discover-deck-ids.sh` - Find board/stack IDs
+- `scripts/sync-roadmap-to-deck.py` - Main sync script
+- `/tmp/deck-config.json` - Nextcloud configuration (auto-generated)
+
+## Configuration
+
+Edit `/tmp/deck-config.json` or set environment variables:
+
+```bash
+export NEXTCLOUD_URL="http://10.0.0.48:8089"
+export NEXTCLOUD_USER="robert.wiegand"
+export NEXTCLOUD_PASSWORD="your-password"
+```
+
+## Roadmap Files
+
+Currently syncs:
+- `OPTIMIZATION_MASTER_ROADMAP.md` - Main overview
+- `SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md` - Initiative 1
+- `POSITION_SCALING_ROADMAP.md` - Initiative 2
+- `ATR_BASED_TP_ROADMAP.md` - Initiative 3
+
+## Future Features
+
+- [ ] Bidirectional sync (Deck → Roadmap updates)
+- [ ] Manual card creation → roadmap entry
+- [ ] Automated sync via cron
+- [ ] Phase-level cards (not just initiative-level)
+- [ ] Label management (optimization, data-collection, analysis)
+- [ ] Due date calculations from timeline estimates
+- [ ] Progress tracking from card checklists
+
+## API Reference
+
+Nextcloud Deck API: `/index.php/apps/deck/api/v1.0`
+
+Key endpoints used:
+- `GET /boards` - List all boards
+- `GET /boards/{boardId}` - Get board details
+- `GET /boards/{boardId}/stacks` - Get stacks
+- `POST /boards/{boardId}/stacks/{stackId}/cards` - Create card
+
+## Troubleshooting
+
+**"Board not found"**
+- Run `discover-deck-ids.sh` first
+- Check board exists in Nextcloud Deck
+- Verify name contains "trader" (case insensitive)
+
+**"405 Method Not Allowed"**
+- Nextcloud Deck version may have different API
+- Check Nextcloud Deck version in settings
+- Some GET endpoints may not be available
+
+**"Cards not syncing"**
+- Check `/tmp/deck-config.json` exists
+- Verify credentials in config
+- Run with `--dry-run` first to test
+
+## Development
+
+**Adding new roadmap files:**
+
+Edit `sync-roadmap-to-deck.py`:
+```python
+ROADMAP_FILES = [
+    'OPTIMIZATION_MASTER_ROADMAP.md',
+    'YOUR_NEW_ROADMAP.md',  # Add here
+]
+```
+
+**Customizing status mapping:**
+
+Edit `STATUS_TO_STACK` dict:
+```python
+STATUS_TO_STACK = {
+    'IN PROGRESS': 'in arbeit',
+    'YOUR_STATUS': 'your_stack',  # Add here
+}
+```
--- a/scripts/discover-deck-ids.sh
+++ b/scripts/discover-deck-ids.sh
@@ -0,0 +1,70 @@
+#!/bin/bash
+# Discover Nextcloud Deck Board and Stack IDs
+# Usage: ./discover-deck-ids.sh
+
+set -e
+
+NEXTCLOUD_URL="${NEXTCLOUD_URL:-http://10.0.0.48:8089}"
+NEXTCLOUD_USER="${NEXTCLOUD_USER:-robert.wiegand}"
+NEXTCLOUD_PASSWORD="${NEXTCLOUD_PASSWORD:-November1985**}"
+
+echo "🔍 Discovering Nextcloud Deck configuration..."
+echo ""
+
+# Get all boards
+echo "📋 Available Boards:"
+BOARDS=$(curl -sS -X GET "${NEXTCLOUD_URL}/index.php/apps/deck/api/v1.0/boards" \
+  -u "${NEXTCLOUD_USER}:${NEXTCLOUD_PASSWORD}" \
+  -H 'OCS-APIRequest: true' \
+  -k)
+
+echo "$BOARDS" | jq -r '.[] | "  ID: \(.id) - Title: \(.title)"'
+echo ""
+
+# Find trader board
+TRADER_BOARD_ID=$(echo "$BOARDS" | jq -r '.[] | select(.title | test("trader"; "i")) | .id' | head -1)
+
+if [ -z "$TRADER_BOARD_ID" ]; then
+  echo "❌ No board with 'trader' in title found!"
+  echo "Please create a board called 'trader' in Nextcloud Deck first."
+  exit 1
+fi
+
+echo "✅ Found 'trader' board with ID: $TRADER_BOARD_ID"
+echo ""
+
+# Get stacks for trader board
+echo "📚 Stacks in 'trader' board:"
+STACKS=$(curl -sS -X GET "${NEXTCLOUD_URL}/index.php/apps/deck/api/v1.0/boards/${TRADER_BOARD_ID}/stacks" \
+  -u "${NEXTCLOUD_USER}:${NEXTCLOUD_PASSWORD}" \
+  -H 'OCS-APIRequest: true' \
+  -k)
+
+echo "$STACKS" | jq -r '.[] | "  ID: \(.id) - Title: \(.title) - Order: \(.order)"'
+echo ""
+
+# Create config snippet
+echo "📝 Configuration for sync-roadmap-to-deck.py:"
+echo ""
+echo "nextcloud:"
+echo "  url: \"${NEXTCLOUD_URL}\""
+echo "  user: \"${NEXTCLOUD_USER}\""
+echo "  board_id: ${TRADER_BOARD_ID}"
+echo "  stacks:"
+echo "$STACKS" | jq -r '.[] | "    \(.title | ascii_downcase | gsub(" "; "_")): \(.id)  # \(.title)"'
+echo ""
+
+# Save to temp file for script to use
+cat > /tmp/deck-config.json <<EOF
+{
+  "url": "${NEXTCLOUD_URL}",
+  "user": "${NEXTCLOUD_USER}",
+  "password": "${NEXTCLOUD_PASSWORD}",
+  "board_id": ${TRADER_BOARD_ID},
+  "stacks": $(echo "$STACKS" | jq '[.[] | {name: .title, id: .id, order: .order}]')
+}
+EOF
+
+echo "✅ Configuration saved to /tmp/deck-config.json"
+echo ""
+echo "Run: python3 scripts/sync-roadmap-to-deck.py --init"
--- a/scripts/sync-roadmap-to-deck.py
+++ b/scripts/sync-roadmap-to-deck.py
@@ -0,0 +1,305 @@
+#!/usr/bin/env python3
+"""
+Nextcloud Deck <-> Trading Bot Roadmap Sync
+Syncs roadmap markdown files with Nextcloud Deck kanban board
+"""
+
+import json
+import re
+import sys
+import os
+import requests
+from pathlib import Path
+from typing import Dict, List, Optional
+from datetime import datetime, timedelta
+import argparse
+
+# Configuration
+DECK_CONFIG_FILE = '/tmp/deck-config.json'
+ROADMAP_DIR = Path('/home/icke/traderv4')
+
+# Load Nextcloud Deck config
+with open(DECK_CONFIG_FILE, 'r') as f:
+    DECK_CONFIG = json.load(f)
+
+NEXTCLOUD_URL = DECK_CONFIG['url']
+NEXTCLOUD_USER = DECK_CONFIG['user']
+NEXTCLOUD_PASSWORD = DECK_CONFIG['password']
+BOARD_ID = DECK_CONFIG['board_id']
+
+# Stack mapping
+STACKS = {stack['name']: stack['id'] for stack in DECK_CONFIG['stacks']}
+
+# Status to stack mapping
+STATUS_TO_STACK = {
+    'IN PROGRESS': 'in arbeit',
+    '🔄 IN PROGRESS': 'in arbeit',
+    'COMPLETE': 'erledigt',
+    '✅ COMPLETE': 'erledigt',
+    'PENDING': 'in planung',
+    '🔜 NEXT': 'in planung',
+    'FUTURE': 'eingang',
+    '🎯 FUTURE': 'eingang',
+}
+
+# Roadmap files to parse
+ROADMAP_FILES = [
+    'OPTIMIZATION_MASTER_ROADMAP.md',
+    'SIGNAL_QUALITY_OPTIMIZATION_ROADMAP.md',
+    'POSITION_SCALING_ROADMAP.md',
+    'ATR_BASED_TP_ROADMAP.md',
+]
+
+
+class DeckAPI:
+    """Nextcloud Deck API wrapper"""
+    
+    def __init__(self):
+        self.base_url = f"{NEXTCLOUD_URL}/index.php/apps/deck/api/v1.0"
+        self.auth = (NEXTCLOUD_USER, NEXTCLOUD_PASSWORD)
+        self.headers = {
+            'OCS-APIRequest': 'true',
+            'Content-Type': 'application/json'
+        }
+    
+    def get_board_details(self) -> Dict:
+        """Get full board details including stacks and cards"""
+        url = f"{self.base_url}/boards/{BOARD_ID}"
+        response = requests.get(url, auth=self.auth, headers=self.headers, verify=False)
+        response.raise_for_status()
+        return response.json()
+    
+    def get_cards_in_stack(self, stack_id: int) -> List[Dict]:
+        """Get all cards in a specific stack"""
+        url = f"{self.base_url}/stacks/{stack_id}/cards"
+        response = requests.get(url, auth=self.auth, headers=self.headers, verify=False)
+        response.raise_for_status()
+        return response.json()
+    
+    def create_card(self, stack_id: int, title: str, description: str, 
+                   due_date: Optional[str] = None, labels: Optional[List[str]] = None) -> Dict:
+        """Create a new card"""
+        url = f"{self.base_url}/boards/{BOARD_ID}/stacks/{stack_id}/cards"
+        
+        payload = {
+            'title': title,
+            'type': 'plain',
+            'order': 999,
+            'description': description,
+        }
+        
+        if due_date:
+            payload['duedate'] = due_date
+        
+        response = requests.post(url, auth=self.auth, headers=self.headers, 
+                               json=payload, verify=False)
+        response.raise_for_status()
+        return response.json()
+    
+    def update_card(self, card_id: int, **kwargs) -> Dict:
+        """Update an existing card"""
+        url = f"{self.base_url}/boards/{BOARD_ID}/cards/{card_id}"
+        response = requests.put(url, auth=self.auth, headers=self.headers, 
+                              json=kwargs, verify=False)
+        response.raise_for_status()
+        return response.json()
+    
+    def get_all_cards(self) -> List[Dict]:
+        """Get all cards from all stacks"""
+        all_cards = []
+        for stack_name, stack_id in STACKS.items():
+            try:
+                cards = self.get_cards_in_stack(stack_id)
+                for card in cards:
+                    card['stack_name'] = stack_name
+                    card['stack_id'] = stack_id
+                all_cards.extend(cards)
+            except requests.exceptions.HTTPError as e:
+                # Stack might be empty, continue
+                print(f"⚠️  Warning: Could not get cards from stack '{stack_name}': {e}")
+                continue
+        return all_cards
+
+
+class RoadmapParser:
+    """Parse roadmap markdown files"""
+    
+    @staticmethod
+    def parse_file(filepath: Path) -> List[Dict]:
+        """Parse a roadmap file and extract phases/tasks"""
+        if not filepath.exists():
+            return []
+        
+        content = filepath.read_text()
+        tasks = []
+        
+        # Parse initiatives (main sections)
+        initiative_pattern = r'## 🎯 Initiative (\d+): (.+?)(?=\n##|\Z)'
+        initiatives = re.finditer(initiative_pattern, content, re.DOTALL)
+        
+        for initiative in initiatives:
+            initiative_num = initiative.group(1)
+            initiative_title = initiative.group(2).strip()
+            initiative_content = initiative.group(0)
+            
+            # Extract file reference
+            file_match = re.search(r'\*\*File:\*\* \[`(.+?)`\]', initiative_content)
+            file_ref = file_match.group(1) if file_match else ''
+            
+            # Extract purpose
+            purpose_match = re.search(r'### Purpose\n(.+?)(?=\n###|\Z)', initiative_content, re.DOTALL)
+            purpose = purpose_match.group(1).strip() if purpose_match else ''
+            
+            # Extract current status
+            status_match = re.search(r'### Current Status\n(.+?)(?=\n###|\Z)', initiative_content, re.DOTALL)
+            status = status_match.group(1).strip() if status_match else ''
+            
+            # Extract timeline
+            timeline_match = re.search(r'### Timeline\n(.+?)(?=\n###|\Z)', initiative_content, re.DOTALL)
+            timeline = timeline_match.group(1).strip() if timeline_match else ''
+            
+            # Extract success metrics
+            metrics_match = re.search(r'### Success Metrics\n(.+?)(?=\n###|\Z)', initiative_content, re.DOTALL)
+            metrics = metrics_match.group(1).strip() if metrics_match else ''
+            
+            # Determine phase status
+            phase_status = 'FUTURE'
+            if '✅ COMPLETE' in status or '✅ **COMPLETE' in status:
+                phase_status = 'COMPLETE'
+            elif '🔄 IN PROGRESS' in status or '🔄 **IN PROGRESS' in status:
+                phase_status = 'IN PROGRESS'
+            elif '🔜 NEXT' in status or 'Phase 2' in status:
+                phase_status = 'PENDING'
+            
+            # Extract progress
+            progress_match = re.search(r'Progress:.*?(\d+)/(\d+)', status)
+            progress = f"{progress_match.group(1)}/{progress_match.group(2)}" if progress_match else ''
+            
+            # Calculate due date from timeline
+            due_date = None
+            if 'weeks' in timeline.lower():
+                weeks = re.search(r'(\d+)-?(\d+)?\s*weeks', timeline.lower())
+                if weeks:
+                    max_weeks = int(weeks.group(2) if weeks.group(2) else weeks.group(1))
+                    due_date = (datetime.now() + timedelta(weeks=max_weeks)).strftime('%Y-%m-%d')
+            
+            task = {
+                'title': f"Initiative {initiative_num}: {initiative_title}",
+                'description': f"""**Initiative {initiative_num}: {initiative_title}**
+
+{purpose}
+
+**Current Status:**
+{status}
+
+**Timeline:**
+{timeline}
+
+**Success Metrics:**
+{metrics}
+
+**File:** {file_ref}
+""",
+                'status': phase_status,
+                'initiative_num': initiative_num,
+                'progress': progress,
+                'due_date': due_date,
+                'labels': [f'initiative-{initiative_num}', 'optimization'],
+            }
+            
+            tasks.append(task)
+        
+        return tasks
+
+
+def sync_roadmap_to_deck(dry_run: bool = False):
+    """Sync roadmap files to Nextcloud Deck"""
+    print("🔄 Syncing roadmap to Nextcloud Deck...")
+    print()
+    
+    api = DeckAPI()
+    parser = RoadmapParser()
+    
+    # Note: Nextcloud Deck API doesn't support GET cards by stack in this version
+    # We'll create cards and rely on manual deduplication for now
+    print("<EFBFBD> Creating cards from roadmap (manual deduplication required)")
+    print()
+    
+    # Parse all roadmap files
+    all_tasks = []
+    for filename in ROADMAP_FILES:
+        filepath = ROADMAP_DIR / filename
+        if filepath.exists():
+            print(f"📖 Parsing {filename}...")
+            tasks = parser.parse_file(filepath)
+            all_tasks.extend(tasks)
+            print(f"   Found {len(tasks)} tasks")
+    
+    print()
+    print(f"📝 Total tasks from roadmaps: {len(all_tasks)}")
+    print()
+    
+    # Create/update cards
+    created = 0
+    skipped = 0
+    
+    for task in all_tasks:
+        title = task['title']
+        
+        # Create new card
+        stack_name = STATUS_TO_STACK.get(task['status'], 'eingang')
+        stack_id = STACKS[stack_name]
+        
+        print(f"✨ Create: {title}")
+        print(f"   Stack: {stack_name}")
+        if task['progress']:
+            print(f"   Progress: {task['progress']}")
+        
+        if not dry_run:
+            try:
+                api.create_card(
+                    stack_id=stack_id,
+                    title=title,
+                    description=task['description'],
+                    due_date=task['due_date'],
+                )
+                created += 1
+            except Exception as e:
+                print(f"   ❌ Error: {e}")
+                skipped += 1
+        else:
+            created += 1
+    
+    print()
+    print("=" * 60)
+    print(f"✅ Sync complete!")
+    print(f"   Created: {created}")
+    print(f"   Skipped: {skipped}")
+    if dry_run:
+        print()
+        print("   (Dry run - no changes made)")
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Sync roadmap with Nextcloud Deck')
+    parser.add_argument('--init', action='store_true', help='Initialize: create cards from roadmap')
+    parser.add_argument('--sync', action='store_true', help='Sync: update existing cards')
+    parser.add_argument('--dry-run', action='store_true', help='Show what would be done without making changes')
+    
+    args = parser.parse_args()
+    
+    if not args.init and not args.sync:
+        parser.print_help()
+        sys.exit(1)
+    
+    try:
+        sync_roadmap_to_deck(dry_run=args.dry_run)
+    except Exception as e:
+        print(f"❌ Error: {e}")
+        import traceback
+        traceback.print_exc()
+        sys.exit(1)
+
+
+if __name__ == '__main__':
+    main()