# Copilot Instructions for Network Scanner Tool

## 🚨 MANDATORY: Read Before Making Changes

### Documentation-First Workflow (ENFORCED)

**BEFORE suggesting any changes:**

1. **Check [docs/index.md](../docs/index.md)** - Find relevant documentation
2. **Search [docs/guides/troubleshooting.md](../docs/guides/troubleshooting.md)** - Known issues and solutions
3. **Review [CONTRIBUTING.md](../CONTRIBUTING.md)** - Development workflow and standards
4. **Verify [docs/project-status.md](../docs/project-status.md)** - Current feature status

**AFTER making changes:**

1. **Update relevant documentation** in `docs/` directory
2. **Add troubleshooting entry** if fixing a bug
3. **Update [docs/project-status.md](../docs/project-status.md)** if feature status changes
4. **Never create ad-hoc markdown files** - use existing `docs/` structure

---

## Quick Overview

This is a **containerized full-stack network scanning and visualization tool**:
- **Backend**: Python 3.11 + FastAPI (async) + SQLite with SQLAlchemy ORM
- **Frontend**: React 18 + TypeScript + TailwindCSS + React Flow visualization
- **Deployment**: Docker Compose with nginx reverse proxy
- **Purpose**: Discover hosts, detect open ports/services, visualize network topology with Visio-style interactive diagrams

**Key Path**: `/home/rwiegand/Nextcloud/entwicklung/Werkzeuge/teamleader_test/`

---

## Critical Architecture Patterns

### 1. Data Flow: Backend → Frontend

```
Database (SQLite) → SQLAlchemy ORM → Pydantic Schemas → REST API/JSON
                                                              ↓
                                                    Frontend React Components
                                                    (via hooks + axios)
```

**Important**: Schema changes in `app/schemas.py` must match TypeScript types in `frontend/src/types/api.ts`. Mismatch causes 500 errors or type failures.

### 2. Session Management (Critical Bug Source)

**Problem**: Async background tasks can't use scoped sessions—they close after request completion.

**Solution** (used in `app/api/endpoints/scans.py`):
```python
# ❌ DON'T: Use shared session from request
background_tasks.add_task(scan_service.execute_scan, scan_id, db)

# ✅ DO: Create new session in background task wrapper
def scan_wrapper(scan_id: int):
    db = SessionLocal()  # New session!
    try:
        scan_service.execute_scan(scan_id, db)
    finally:
        db.close()

background_tasks.add_task(scan_wrapper, scan_id)
```

### 3. Database Constraints (Critical)

**Rule**: Always `commit()` and `refresh()` host objects **before** adding dependent records (services, connections).

**Example** (from `app/services/scan_service.py`):
```python
host = self._get_or_create_host(ip)
self.db.commit()  # ← CRITICAL: Ensure host.id is set
self.db.refresh(host)

# NOW safe to add services
service = Service(host_id=host.id, port=80)  # host.id exists
self.db.add(service)
```

### 4. WebSocket Broadcasting

Located in `app/api/endpoints/websocket.py`. Pattern:
```python
# Broadcast to all connected clients
await connection_manager.broadcast({
    "type": "scan_progress",
    "data": {"progress": 0.75, "current_host": "192.168.1.5"}
})
```

Integrated in `ScanService` to push real-time updates during scans.

---

## Project Structure Deep Dive

### Backend (`app/`)

| Component | Purpose | Critical Points |
|-----------|---------|-----------------|
| `models.py` | SQLAlchemy ORM | Use `Connection.extra_data` not `.metadata` (reserved). Check cascade rules. |
| `schemas.py` | Pydantic validation | Must have `.model_rebuild()` for forward refs (e.g., `HostDetailResponse`). |
| `services/scan_service.py` | Scan orchestration | Uses WebSocket callbacks. Commit after host creation. |
| `services/topology_service.py` | Graph generation | Simplified to return flat TopologyNode/TopologyEdge structures. |
| `api/endpoints/` | REST routes | Use `SessionLocal()` wrapper for background tasks. |
| `scanner/` | Network scanning | Socket-based (default, no root) + optional nmap integration. |

### Frontend (`frontend/src/`)

| Component | Purpose | Critical Points |
|-----------|---------|-----------------|
| `pages/Dashboard.tsx` | Home + scan control | Displays real-time progress bar via WebSocket. |
| `pages/NetworkPage.tsx` | Network map page | Integrated click handler via `HostDetailsPanel`. |
| `components/NetworkMap.tsx` | React Flow visualization | Creates nodes with circular layout, handles node clicks. |
| `components/HostDetailsPanel.tsx` | Host detail sidebar | Right-side panel, fetches via `hostApi.getHost()`. |
| `hooks/useTopology.ts` | Topology data | Fetches from `/api/topology`. |
| `services/api.ts` | Axios client | Base URL from `VITE_API_URL` env var. |
| `types/api.ts` | TypeScript interfaces | **Must match backend schemas exactly**. |

---

## Build & Deployment

### Development
```bash
cd teamleader_test
docker compose up -d --build
# Frontend: http://localhost (nginx reverse proxy)
# Backend API: http://localhost:8000
# Docs: http://localhost:8000/docs
```

### Common Issues & Fixes

| Error | Cause | Fix |
|-------|-------|-----|
| `500 Error on /api/topology` | `TopologyNode` missing `position` field | Remove `_calculate_layout()` call (positions not in simplified schema) |
| Frontend type mismatch | Backend returns `network_range`, frontend expects `target` | Update `frontend/src/types/api.ts` interface |
| `DetachedInstanceError` in async task | Session closed after request | Use `SessionLocal()` wrapper in background task |
| `NOT NULL constraint failed: host_id` | Services added before host committed | Add `db.commit()` + `db.refresh(host)` after creation |
| Scan progress not updating | WebSocket not connected | Verify `useWebSocket()` hook in Dashboard, check `/api/ws` endpoint |

---

## Database Schema (Key Tables)

```python
# 5 main tables:
Scan        # Scan operations (status, network_range, timestamps)
Host        # Discovered hosts (ip_address UNIQUE, status, hostname)
Service     # Open ports (host_id FK, port, state, banner)
Connection  # Relationships (source_host_id, target_host_id, confidence)
scan_hosts  # Many-to-many (scan_id, host_id)
```

**Column rename**: `Connection.metadata` → `Connection.extra_data` (metadata is SQLAlchemy reserved).

---

## API Response Contract

### Topology Response (`/api/topology`)
```typescript
{
  nodes: TopologyNode[],    // id, ip, hostname, type, status, service_count, connections
  edges: TopologyEdge[],    // source, target, type, confidence
  statistics: {
    total_nodes: number,
    total_edges: number,
    isolated_nodes: number,
    avg_connections: number
  }
}
```

### Scan Start Response (`POST /api/scans/start`)
```typescript
{
  scan_id: number,
  message: string,
  status: 'pending' | 'running'
}
```

**Note**: Frontend uses `scan.scan_id` not `scan.id`.

---

## Docker Compose Setup

- **Backend container**: `network-scanner-backend` on port 8000
- **Frontend container**: `network-scanner-frontend` on port 80 (nginx)
- **Shared volume**: `./data/` for SQLite database
- **Network**: `scanner-network` (bridge)

```bash
# Rebuild after code changes
docker compose up -d --build

# View logs
docker compose logs backend --tail=50
docker compose logs frontend --tail=50

# Stop all
docker compose down
```

---

## Common Development Tasks

### Add New API Endpoint

1. Create handler in `app/api/endpoints/new_feature.py`
2. Add to `app/api/__init__.py` router
3. Define request/response Pydantic models in `app/schemas.py`
4. Add TypeScript type in `frontend/src/types/api.ts`
5. Add service method in `frontend/src/services/api.ts`
6. Create React hook or component to call it

### Modify Database Schema

1. Update SQLAlchemy model in `app/models.py`
2. Delete `network_scanner.db` (dev only) or create migration
3. Restart backend: `docker compose up -d --build backend`

### Add Frontend Component

1. Create `.tsx` file in `frontend/src/components/`
2. Import API types from `frontend/src/types/api.ts`
3. Use Axios via `frontend/src/services/api.ts`
4. Integrate into page/parent component
5. Rebuild: `docker compose up -d --build frontend`

---

## Debugging Checklist

- **Backend not responding?** Check `docker compose logs backend` for errors
- **Frontend showing blank page?** Open browser console (F12) for errors, check `VITE_API_URL`
- **Type errors on build?** Verify `frontend/src/types/api.ts` matches backend response
- **Database locked?** Only one writer at a time with SQLite—check for stuck processes
- **WebSocket not connecting?** Verify `/api/ws` endpoint exists, check backend logs
- **Scan never completes?** Check `docker compose logs backend` for exceptions, verify `cancel_requested` flag

---

## Conventions

1. **Logging**: Use `logger.info()` for key events, `logger.error()` for failures
2. **Database**: Use SQLAlchemy relationships, avoid raw SQL
3. **API Responses**: Always wrap in Pydantic models, validate input
4. **Async**: Use `asyncio.Task` for background scans, new `SessionLocal()` for session
5. **Frontend**: Use TypeScript strict mode, never bypass type checking
6. **Styling**: TailwindCSS only, no inline CSS (except React Flow inline styles)
7. **Comments**: Document "why", not "what"—code is self-documenting

---

## Key Files to Reference

- **Architecture overview**: [docs/architecture/overview.md](../docs/architecture/overview.md) (comprehensive design decisions)
- **Database models**: [app/models.py](../app/models.py)
- **API schemas**: [app/schemas.py](../app/schemas.py)
- **Scan orchestration**: [app/services/scan_service.py](../app/services/scan_service.py) (async patterns, WebSocket integration)
- **Topology service**: [app/services/topology_service.py](../app/services/topology_service.py) (graph generation logic)
- **Frontend types**: [frontend/src/types/api.ts](../frontend/src/types/api.ts)
- **Network Map**: [frontend/src/components/NetworkMap.tsx](../frontend/src/components/NetworkMap.tsx) (React Flow usage, click handling)
- **Docker setup**: [docker-compose.yml](../docker-compose.yml)

---

## Testing Commands

```bash
# Health check
curl -s http://localhost/health | jq .

# Start scan (quick 192.168.1.0/24)
curl -X POST http://localhost:8000/api/scans/start \
  -H "Content-Type: application/json" \
  -d '{"network_range":"192.168.1.0/24","scan_type":"quick"}'

# Get topology
curl -s http://localhost/api/topology | jq '.nodes | length'

# List hosts
curl -s http://localhost/api/hosts | jq 'length'

# Get specific host
curl -s http://localhost/api/hosts/1 | jq .
```

---

## Version Info

- **Python**: 3.11
- **FastAPI**: 0.109.0
- **React**: 18.2.0
- **TypeScript**: 5.2.2
- **Last Updated**: December 4, 2025

---

## Contact & Escalation

For questions on:
- **Architecture**: See `ARCHITECTURE.md`
- **Build issues**: Check `docker compose logs backend/frontend`
- **Type errors**: Verify `app/schemas.py` ↔ `frontend/src/types/api.ts` alignment
- **Runtime crashes**: Enable `DEBUG=True` in `.env`, check logs directory