mortimer/.github/copilot-instructions.md

# GitHub Copilot Instructions - Mortimer Telegram Assistant

## Project Overview

This is a Telegram bot assistant ("@mortimer_assi_bot") built on n8n that provides AI-powered task management, email search, and conversational AI capabilities. The bot is hosted at https://flow.egonetix.de/ and integrates with Nextcloud Deck, IMAP email, and OpenAI.

## Architecture

### n8n Environment
- **Version**: 1.123.5
- **Container**: n8n-new (Docker)
- **Database**: PostgreSQL 15 (container: postgres-n8n)
- **External URL**: https://flow.egonetix.de/
- **API Endpoint**: https://flow.egonetix.de/api/v1/
- **API Key**: n8n_api_42f1838c1e2de90cadcb669f78083de92697a85322c0b6009ad2e55760db992ab0bf61515a3cf0e1

### Telegram Bot
- **Bot Name**: @mortimer_assi_bot
- **Token**: 8506559707:AAGn9dYm2PEuSGMbJ7jtiuIfGbl1ScaCxQk
- **Webhook URL**: https://flow.egonetix.de/webhook/8f3f59db-aaa5-4762-9416-94be04131fd2
- **Credential ID**: 6Rim8odDFpQoHxCM (name: "mortimer")

### Credentials in n8n
- **Telegram API**: 6Rim8odDFpQoHxCM (mortimer) - Used for all Telegram operations
- **OpenAI API**: GPzxHwwXxeZg07o5 - Used for AI classification and chat

## Workflow Structure

### Current Active Workflows (Simplified Architecture)

1. **Telegram Assistant - Receiver** (ID: gDao7SBRyJiIBF7c)
   - Purpose: Entry point for all Telegram messages via webhook
   - Webhook Path: `/webhook/8f3f59db-aaa5-4762-9416-94be04131fd2`
   - Data Flow:
     - Receives webhook POST from Telegram
     - Checks if message starts with `/` (command vs natural language)
     - For commands: Parses command and arguments
     - For natural language: Uses OpenAI to classify intent
     - Calls Router workflow via webhook
   
2. **Telegram Router - Direct** (ID: Ys9shWiWlbvxRFh4)
   - Purpose: Simplified router that directly sends Telegram responses
   - Webhook Path: `/telegram-router-direct`
   - Data Flow:
     - Receives data from Receiver
     - Sends response directly via Telegram API
     - No complex respondToWebhook structure

### Planned Workflows (Not Yet Implemented)

3. **telegram-deck.json** - Nextcloud Deck integration
   - Purpose: Create tasks in Nextcloud Deck from Telegram commands
   - Command: `/deck <task description>`
   - Uses: `/home/node/create_card_from_ai.sh` script

4. **telegram-email.json** - IMAP email search
   - Purpose: Search emails and return summaries
   - Command: `/email <search query>`

5. **telegram-ai.json** - AI chat
   - Purpose: General AI conversations
   - Command: `/ask <question>`

## Critical Code Patterns

### Webhook Data Structure
**IMPORTANT**: Telegram webhook delivers data directly in `$json`, NOT in `$json.body`:
```javascript
// ✅ CORRECT
const text = $json.message?.text || '';
const chatId = $json.message.chat.id;
const userId = $json.message.from.id;

// ❌ WRONG (common mistake)
const text = $json.body.message.text;
```

### Optional Chaining is Essential
Always use optional chaining (`?.`) when accessing nested webhook properties:
```javascript
// ✅ CORRECT - Won't crash on null/undefined
const text = $input.item.json.message?.text || '';
const username = $input.item.json.message?.from?.username || 'unknown';

// ❌ WRONG - Will crash with "Cannot read properties of undefined"
const text = $input.item.json.message.text;
```

### Credential References
Always use the credential ID `6Rim8odDFpQoHxCM` for Telegram operations:
```json
{
  "credentials": {
    "telegramApi": {
      "id": "6Rim8odDFpQoHxCM",
      "name": "mortimer"
    }
  }
}
```

### Router Architecture Lessons
The simplified Router workflow that works:
- Receives data via webhook (simple HTTP request)
- Directly sends Telegram message using Telegram node
- Does NOT use `respondToWebhook` nodes (they cause "Invalid JSON" errors)

Previous complex Router with `respondToWebhook` structure failed consistently. Keep it simple.

## Testing Patterns

### Direct Telegram API Testing
Test bot responses without manually sending Telegram messages:
```bash
curl -X POST "https://api.telegram.org/bot8506559707:AAGn9dYm2PEuSGMbJ7jtiuIfGbl1ScaCxQk/sendMessage" \
  -H "Content-Type: application/json" \
  -d '{
    "chat_id": "YOUR_CHAT_ID",
    "text": "Test message"
  }'
```

### Webhook Testing
Test webhook receiver directly:
```bash
curl -X POST "https://flow.egonetix.de/webhook/8f3f59db-aaa5-4762-9416-94be04131fd2" \
  -H "Content-Type: application/json" \
  -d '{
    "message": {
      "message_id": 123,
      "from": {"id": 123456789, "username": "test"},
      "chat": {"id": 123456789},
      "text": "/test command"
    }
  }'
```

### n8n Workflow Import
Import workflows via API (when manual import fails):
```bash
curl -X POST "https://flow.egonetix.de/api/v1/workflows" \
  -H "X-N8N-API-KEY: n8n_api_42f1838c1e2de90cadcb669f78083de92697a85322c0b6009ad2e55760db992ab0bf61515a3cf0e1" \
  -H "Content-Type: application/json" \
  -d @workflow.json
```

## Common Pitfalls and Solutions

### Problem: n8n API Doesn't Apply Changes Exactly
**Symptom**: File edits don't reflect in workflow execution
**Solution**: Delete and recreate the workflow via API, don't just update

### Problem: "Invalid JSON in response body"
**Symptom**: Router workflow fails with JSON parse error
**Solution**: Remove `respondToWebhook` nodes, use direct Telegram send instead

### Problem: JavaScript Errors - "Cannot read properties of undefined"
**Symptom**: Workflow crashes on webhook data access
**Solution**: Add optional chaining (`?.`) to all nested property access

### Problem: Polling Trigger Interferes with Webhook
**Symptom**: Multiple workflows respond to same message
**Solution**: Delete all workflows with Telegram polling triggers, use only webhook

### Problem: Wrong Credential ID
**Symptom**: "Credentials not found" errors
**Solution**: Always use `6Rim8odDFpQoHxCM`, update all references in workflow JSON

### Problem: Workflow Not Found After Import
**Symptom**: Workflow shows in UI but API can't find it
**Solution**: Activate the workflow, check if it's in correct project/folder

## Database Direct Access

For emergency fixes or debugging:
```bash
# Connect to PostgreSQL
docker exec -it postgres-n8n psql -U n8n -d n8n

# List all workflows
SELECT id, name, active FROM workflow_entity;

# Find credential ID by name
SELECT id, name FROM credentials_entity WHERE name = 'mortimer';

# Delete workflow by ID (permanent)
DELETE FROM workflow_entity WHERE id = 'OLD_WORKFLOW_ID';
```

## Development Workflow

1. **Make Changes**: Edit workflow JSON files in `/home/icke/assistant/workflows/`
2. **Test Locally**: Validate JSON syntax, check credential IDs
3. **Import to n8n**: Use API to import or update workflow
4. **Activate**: Ensure workflow is activated in n8n UI
5. **Test Webhook**: Use curl to send test payload
6. **Verify**: Check execution logs in n8n UI
7. **Commit**: Git commit and push to Gitea

## File Structure

```
/home/icke/assistant/
├── .github/
│   └── copilot-instructions.md (this file)
├── workflows/
│   ├── telegram-receiver.json (active)
│   ├── telegram-router.json (deprecated)
│   ├── telegram-deck.json (planned)
│   ├── telegram-email.json (planned)
│   └── telegram-ai.json (planned)
├── README.md
├── SETUP_COMPLETE.md
├── N8N_UPGRADE_LESSONS_2025-12-03.md
├── NATURAL_LANGUAGE_UPDATE.md
└── import_workflows.sh
```

## Quick Reference

### Import All Workflows
```bash
cd /home/icke/assistant
./import_workflows.sh
```

### Configure Telegram Webhook
```bash
curl -X POST "https://api.telegram.org/bot8506559707:AAGn9dYm2PEuSGMbJ7jtiuIfGbl1ScaCxQk/setWebhook" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://flow.egonetix.de/webhook/8f3f59db-aaa5-4762-9416-94be04131fd2"}'
```

### Check Webhook Status
```bash
curl "https://api.telegram.org/bot8506559707:AAGn9dYm2PEuSGMbJ7jtiuIfGbl1ScaCxQk/getWebhookInfo"
```

## AI Agent Guidelines

When working on this project:
1. **Always test autonomously** - Use curl for testing, don't rely on manual Telegram messages
2. **Trust the architecture** - Simplified 2-workflow system is working, don't overcomplicate
3. **Use optional chaining everywhere** - Webhook data can be null at any level
4. **Verify credentials first** - Wrong credential ID is the most common issue
5. **Check execution logs** - n8n UI shows detailed execution data
6. **Delete when stuck** - If API won't update workflow, delete and recreate
7. **Document changes** - Update README.md with any architectural changes

## Current Status (as of 2025-12-19)

✅ **Working**: Receiver → Router → Telegram response flow
✅ **Working**: Command parsing and natural language detection
✅ **Working**: Telegram webhook integration
⚠️ **Pending**: Deck, Email, and AI command implementations
⚠️ **Pending**: Full error handling and logging
⚠️ **Pending**: User authentication/authorization

## External Resources

- n8n Documentation: https://docs.n8n.io/
- Telegram Bot API: https://core.telegram.org/bots/api
- n8n Community: https://community.n8n.io/
- Project Gitea: http://gitea.egonetix.de:3000/root/mortimer