Initial commit: Werkzeuge-Sammlung
Enthält: - rdp_client.py: RDP Client mit GUI und Monitor-Auswahl - rdp.sh: Bash-basierter RDP Client - teamleader_test/: Network Scanner Fullstack-App - teamleader_test2/: Network Mapper CLI Subdirectories mit eigenem Repo wurden ausgeschlossen. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
root
316
teamleader_test/.github/copilot-instructions.md
vendored
Normal file
316
teamleader_test/.github/copilot-instructions.md
vendored
Normal file
@@ -0,0 +1,316 @@
|
||||
# Copilot Instructions for Network Scanner Tool
|
||||
|
||||
## 🚨 MANDATORY: Read Before Making Changes
|
||||
|
||||
### Documentation-First Workflow (ENFORCED)
|
||||
|
||||
**BEFORE suggesting any changes:**
|
||||
|
||||
1. **Check [docs/index.md](../docs/index.md)** - Find relevant documentation
|
||||
2. **Search [docs/guides/troubleshooting.md](../docs/guides/troubleshooting.md)** - Known issues and solutions
|
||||
3. **Review [CONTRIBUTING.md](../CONTRIBUTING.md)** - Development workflow and standards
|
||||
4. **Verify [docs/project-status.md](../docs/project-status.md)** - Current feature status
|
||||
|
||||
**AFTER making changes:**
|
||||
|
||||
1. **Update relevant documentation** in `docs/` directory
|
||||
2. **Add troubleshooting entry** if fixing a bug
|
||||
3. **Update [docs/project-status.md](../docs/project-status.md)** if feature status changes
|
||||
4. **Never create ad-hoc markdown files** - use existing `docs/` structure
|
||||
|
||||
---
|
||||
|
||||
## Quick Overview
|
||||
|
||||
This is a **containerized full-stack network scanning and visualization tool**:
|
||||
- **Backend**: Python 3.11 + FastAPI (async) + SQLite with SQLAlchemy ORM
|
||||
- **Frontend**: React 18 + TypeScript + TailwindCSS + React Flow visualization
|
||||
- **Deployment**: Docker Compose with nginx reverse proxy
|
||||
- **Purpose**: Discover hosts, detect open ports/services, visualize network topology with Visio-style interactive diagrams
|
||||
|
||||
**Key Path**: `/home/rwiegand/Nextcloud/entwicklung/Werkzeuge/teamleader_test/`
|
||||
|
||||
---
|
||||
|
||||
## Critical Architecture Patterns
|
||||
|
||||
### 1. Data Flow: Backend → Frontend
|
||||
|
||||
```
|
||||
Database (SQLite) → SQLAlchemy ORM → Pydantic Schemas → REST API/JSON
|
||||
↓
|
||||
Frontend React Components
|
||||
(via hooks + axios)
|
||||
```
|
||||
|
||||
**Important**: Schema changes in `app/schemas.py` must match TypeScript types in `frontend/src/types/api.ts`. Mismatch causes 500 errors or type failures.
|
||||
|
||||
### 2. Session Management (Critical Bug Source)
|
||||
|
||||
**Problem**: Async background tasks can't use scoped sessions—they close after request completion.
|
||||
|
||||
**Solution** (used in `app/api/endpoints/scans.py`):
|
||||
```python
|
||||
# ❌ DON'T: Use shared session from request
|
||||
background_tasks.add_task(scan_service.execute_scan, scan_id, db)
|
||||
|
||||
# ✅ DO: Create new session in background task wrapper
|
||||
def scan_wrapper(scan_id: int):
|
||||
db = SessionLocal() # New session!
|
||||
try:
|
||||
scan_service.execute_scan(scan_id, db)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
background_tasks.add_task(scan_wrapper, scan_id)
|
||||
```
|
||||
|
||||
### 3. Database Constraints (Critical)
|
||||
|
||||
**Rule**: Always `commit()` and `refresh()` host objects **before** adding dependent records (services, connections).
|
||||
|
||||
**Example** (from `app/services/scan_service.py`):
|
||||
```python
|
||||
host = self._get_or_create_host(ip)
|
||||
self.db.commit() # ← CRITICAL: Ensure host.id is set
|
||||
self.db.refresh(host)
|
||||
|
||||
# NOW safe to add services
|
||||
service = Service(host_id=host.id, port=80) # host.id exists
|
||||
self.db.add(service)
|
||||
```
|
||||
|
||||
### 4. WebSocket Broadcasting
|
||||
|
||||
Located in `app/api/endpoints/websocket.py`. Pattern:
|
||||
```python
|
||||
# Broadcast to all connected clients
|
||||
await connection_manager.broadcast({
|
||||
"type": "scan_progress",
|
||||
"data": {"progress": 0.75, "current_host": "192.168.1.5"}
|
||||
})
|
||||
```
|
||||
|
||||
Integrated in `ScanService` to push real-time updates during scans.
|
||||
|
||||
---
|
||||
|
||||
## Project Structure Deep Dive
|
||||
|
||||
### Backend (`app/`)
|
||||
|
||||
| Component | Purpose | Critical Points |
|
||||
|-----------|---------|-----------------|
|
||||
| `models.py` | SQLAlchemy ORM | Use `Connection.extra_data` not `.metadata` (reserved). Check cascade rules. |
|
||||
| `schemas.py` | Pydantic validation | Must have `.model_rebuild()` for forward refs (e.g., `HostDetailResponse`). |
|
||||
| `services/scan_service.py` | Scan orchestration | Uses WebSocket callbacks. Commit after host creation. |
|
||||
| `services/topology_service.py` | Graph generation | Simplified to return flat TopologyNode/TopologyEdge structures. |
|
||||
| `api/endpoints/` | REST routes | Use `SessionLocal()` wrapper for background tasks. |
|
||||
| `scanner/` | Network scanning | Socket-based (default, no root) + optional nmap integration. |
|
||||
|
||||
### Frontend (`frontend/src/`)
|
||||
|
||||
| Component | Purpose | Critical Points |
|
||||
|-----------|---------|-----------------|
|
||||
| `pages/Dashboard.tsx` | Home + scan control | Displays real-time progress bar via WebSocket. |
|
||||
| `pages/NetworkPage.tsx` | Network map page | Integrated click handler via `HostDetailsPanel`. |
|
||||
| `components/NetworkMap.tsx` | React Flow visualization | Creates nodes with circular layout, handles node clicks. |
|
||||
| `components/HostDetailsPanel.tsx` | Host detail sidebar | Right-side panel, fetches via `hostApi.getHost()`. |
|
||||
| `hooks/useTopology.ts` | Topology data | Fetches from `/api/topology`. |
|
||||
| `services/api.ts` | Axios client | Base URL from `VITE_API_URL` env var. |
|
||||
| `types/api.ts` | TypeScript interfaces | **Must match backend schemas exactly**. |
|
||||
|
||||
---
|
||||
|
||||
## Build & Deployment
|
||||
|
||||
### Development
|
||||
```bash
|
||||
cd teamleader_test
|
||||
docker compose up -d --build
|
||||
# Frontend: http://localhost (nginx reverse proxy)
|
||||
# Backend API: http://localhost:8000
|
||||
# Docs: http://localhost:8000/docs
|
||||
```
|
||||
|
||||
### Common Issues & Fixes
|
||||
|
||||
| Error | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| `500 Error on /api/topology` | `TopologyNode` missing `position` field | Remove `_calculate_layout()` call (positions not in simplified schema) |
|
||||
| Frontend type mismatch | Backend returns `network_range`, frontend expects `target` | Update `frontend/src/types/api.ts` interface |
|
||||
| `DetachedInstanceError` in async task | Session closed after request | Use `SessionLocal()` wrapper in background task |
|
||||
| `NOT NULL constraint failed: host_id` | Services added before host committed | Add `db.commit()` + `db.refresh(host)` after creation |
|
||||
| Scan progress not updating | WebSocket not connected | Verify `useWebSocket()` hook in Dashboard, check `/api/ws` endpoint |
|
||||
|
||||
---
|
||||
|
||||
## Database Schema (Key Tables)
|
||||
|
||||
```python
|
||||
# 5 main tables:
|
||||
Scan # Scan operations (status, network_range, timestamps)
|
||||
Host # Discovered hosts (ip_address UNIQUE, status, hostname)
|
||||
Service # Open ports (host_id FK, port, state, banner)
|
||||
Connection # Relationships (source_host_id, target_host_id, confidence)
|
||||
scan_hosts # Many-to-many (scan_id, host_id)
|
||||
```
|
||||
|
||||
**Column rename**: `Connection.metadata` → `Connection.extra_data` (metadata is SQLAlchemy reserved).
|
||||
|
||||
---
|
||||
|
||||
## API Response Contract
|
||||
|
||||
### Topology Response (`/api/topology`)
|
||||
```typescript
|
||||
{
|
||||
nodes: TopologyNode[], // id, ip, hostname, type, status, service_count, connections
|
||||
edges: TopologyEdge[], // source, target, type, confidence
|
||||
statistics: {
|
||||
total_nodes: number,
|
||||
total_edges: number,
|
||||
isolated_nodes: number,
|
||||
avg_connections: number
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Scan Start Response (`POST /api/scans/start`)
|
||||
```typescript
|
||||
{
|
||||
scan_id: number,
|
||||
message: string,
|
||||
status: 'pending' | 'running'
|
||||
}
|
||||
```
|
||||
|
||||
**Note**: Frontend uses `scan.scan_id` not `scan.id`.
|
||||
|
||||
---
|
||||
|
||||
## Docker Compose Setup
|
||||
|
||||
- **Backend container**: `network-scanner-backend` on port 8000
|
||||
- **Frontend container**: `network-scanner-frontend` on port 80 (nginx)
|
||||
- **Shared volume**: `./data/` for SQLite database
|
||||
- **Network**: `scanner-network` (bridge)
|
||||
|
||||
```bash
|
||||
# Rebuild after code changes
|
||||
docker compose up -d --build
|
||||
|
||||
# View logs
|
||||
docker compose logs backend --tail=50
|
||||
docker compose logs frontend --tail=50
|
||||
|
||||
# Stop all
|
||||
docker compose down
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Development Tasks
|
||||
|
||||
### Add New API Endpoint
|
||||
|
||||
1. Create handler in `app/api/endpoints/new_feature.py`
|
||||
2. Add to `app/api/__init__.py` router
|
||||
3. Define request/response Pydantic models in `app/schemas.py`
|
||||
4. Add TypeScript type in `frontend/src/types/api.ts`
|
||||
5. Add service method in `frontend/src/services/api.ts`
|
||||
6. Create React hook or component to call it
|
||||
|
||||
### Modify Database Schema
|
||||
|
||||
1. Update SQLAlchemy model in `app/models.py`
|
||||
2. Delete `network_scanner.db` (dev only) or create migration
|
||||
3. Restart backend: `docker compose up -d --build backend`
|
||||
|
||||
### Add Frontend Component
|
||||
|
||||
1. Create `.tsx` file in `frontend/src/components/`
|
||||
2. Import API types from `frontend/src/types/api.ts`
|
||||
3. Use Axios via `frontend/src/services/api.ts`
|
||||
4. Integrate into page/parent component
|
||||
5. Rebuild: `docker compose up -d --build frontend`
|
||||
|
||||
---
|
||||
|
||||
## Debugging Checklist
|
||||
|
||||
- **Backend not responding?** Check `docker compose logs backend` for errors
|
||||
- **Frontend showing blank page?** Open browser console (F12) for errors, check `VITE_API_URL`
|
||||
- **Type errors on build?** Verify `frontend/src/types/api.ts` matches backend response
|
||||
- **Database locked?** Only one writer at a time with SQLite—check for stuck processes
|
||||
- **WebSocket not connecting?** Verify `/api/ws` endpoint exists, check backend logs
|
||||
- **Scan never completes?** Check `docker compose logs backend` for exceptions, verify `cancel_requested` flag
|
||||
|
||||
---
|
||||
|
||||
## Conventions
|
||||
|
||||
1. **Logging**: Use `logger.info()` for key events, `logger.error()` for failures
|
||||
2. **Database**: Use SQLAlchemy relationships, avoid raw SQL
|
||||
3. **API Responses**: Always wrap in Pydantic models, validate input
|
||||
4. **Async**: Use `asyncio.Task` for background scans, new `SessionLocal()` for session
|
||||
5. **Frontend**: Use TypeScript strict mode, never bypass type checking
|
||||
6. **Styling**: TailwindCSS only, no inline CSS (except React Flow inline styles)
|
||||
7. **Comments**: Document "why", not "what"—code is self-documenting
|
||||
|
||||
---
|
||||
|
||||
## Key Files to Reference
|
||||
|
||||
- **Architecture overview**: [docs/architecture/overview.md](../docs/architecture/overview.md) (comprehensive design decisions)
|
||||
- **Database models**: [app/models.py](../app/models.py)
|
||||
- **API schemas**: [app/schemas.py](../app/schemas.py)
|
||||
- **Scan orchestration**: [app/services/scan_service.py](../app/services/scan_service.py) (async patterns, WebSocket integration)
|
||||
- **Topology service**: [app/services/topology_service.py](../app/services/topology_service.py) (graph generation logic)
|
||||
- **Frontend types**: [frontend/src/types/api.ts](../frontend/src/types/api.ts)
|
||||
- **Network Map**: [frontend/src/components/NetworkMap.tsx](../frontend/src/components/NetworkMap.tsx) (React Flow usage, click handling)
|
||||
- **Docker setup**: [docker-compose.yml](../docker-compose.yml)
|
||||
|
||||
---
|
||||
|
||||
## Testing Commands
|
||||
|
||||
```bash
|
||||
# Health check
|
||||
curl -s http://localhost/health | jq .
|
||||
|
||||
# Start scan (quick 192.168.1.0/24)
|
||||
curl -X POST http://localhost:8000/api/scans/start \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"network_range":"192.168.1.0/24","scan_type":"quick"}'
|
||||
|
||||
# Get topology
|
||||
curl -s http://localhost/api/topology | jq '.nodes | length'
|
||||
|
||||
# List hosts
|
||||
curl -s http://localhost/api/hosts | jq 'length'
|
||||
|
||||
# Get specific host
|
||||
curl -s http://localhost/api/hosts/1 | jq .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Version Info
|
||||
|
||||
- **Python**: 3.11
|
||||
- **FastAPI**: 0.109.0
|
||||
- **React**: 18.2.0
|
||||
- **TypeScript**: 5.2.2
|
||||
- **Last Updated**: December 4, 2025
|
||||
|
||||
---
|
||||
|
||||
## Contact & Escalation
|
||||
|
||||
For questions on:
|
||||
- **Architecture**: See `ARCHITECTURE.md`
|
||||
- **Build issues**: Check `docker compose logs backend/frontend`
|
||||
- **Type errors**: Verify `app/schemas.py` ↔ `frontend/src/types/api.ts` alignment
|
||||
- **Runtime crashes**: Enable `DEBUG=True` in `.env`, check logs directory
|
||||
|
||||
Reference in New Issue
Block a user