werkzeuge/teamleader_test/README.md

# Network Scanner & Visualization Tool

A comprehensive network scanning and visualization tool built with FastAPI, React, and Docker. Discover hosts, detect services, and visualize network topology with interactive diagrams.

**Status**: ✅ Production Ready | **Version**: 1.0.0 | **Last Updated**: December 4, 2025

---

## 📚 Documentation

**→ [Full Documentation Index](docs/index.md)** ← Start here for complete documentation

### Quick Links

- **[Quick Start Guide](QUICKSTART.md)** - Get running in 5 minutes
- **[Docker Setup](docs/setup/docker.md)** - Container deployment
- **[Troubleshooting](docs/guides/troubleshooting.md)** - Common issues & solutions
- **[Contributing](CONTRIBUTING.md)** - Development workflow
- **[Project Status](docs/project-status.md)** - Feature completeness
- **[Architecture](docs/architecture/overview.md)** - Design decisions

**For AI Agents**: Read [.github/copilot-instructions.md](.github/copilot-instructions.md) for critical patterns and mandatory documentation workflows.

---

## Features

- **Network Host Discovery**: Scan networks to discover active hosts
- **Port Scanning**: Detect open ports and running services
- **Service Detection**: Identify service types and versions
- **Network Topology**: Generate network topology graphs
- **Real-time Updates**: WebSocket support for live scan progress
- **REST API**: Complete RESTful API for all operations
- **No Root Required**: Socket-based scanning works without root privileges
- **Optional Nmap Integration**: Use nmap for advanced scanning capabilities

## Architecture

### Technology Stack

- **Backend Framework**: FastAPI (async Python web framework)
- **Database**: SQLite with SQLAlchemy ORM
- **Network Scanning**: 
  - Socket-based TCP connect scanning (no root)
  - python-nmap integration (optional)
  - Custom service detection and banner grabbing
- **Real-time Communication**: WebSockets

### Project Structure

```
teamleader_test/
├── app/
│   ├── __init__.py
│   ├── config.py              # Configuration management
│   ├── database.py            # Database setup
│   ├── models.py              # SQLAlchemy models
│   ├── schemas.py             # Pydantic schemas
│   ├── api/
│   │   ├── __init__.py
│   │   └── endpoints/
│   │       ├── scans.py       # Scan endpoints
│   │       ├── hosts.py       # Host endpoints
│   │       ├── topology.py    # Topology endpoints
│   │       └── websocket.py   # WebSocket endpoint
│   ├── scanner/
│   │   ├── __init__.py
│   │   ├── network_scanner.py # Host discovery
│   │   ├── port_scanner.py    # Port scanning
│   │   ├── service_detector.py# Service detection
│   │   └── nmap_scanner.py    # Nmap integration
│   └── services/
│       ├── __init__.py
│       ├── scan_service.py    # Scan orchestration
│       └── topology_service.py# Topology generation
├── main.py                    # Application entry point
├── requirements.txt           # Python dependencies
├── .env.example              # Example environment variables
└── README.md                 # This file
```

## Installation

### Prerequisites

- Python 3.10 or higher
- pip (Python package manager)
- Optional: nmap installed on system for advanced scanning

### Setup

1. **Clone or navigate to the project directory**:
```bash
cd /home/rwiegand/Nextcloud/entwicklung/Werkzeuge/teamleader_test
```

2. **Create a virtual environment**:
```bash
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

3. **Install dependencies**:
```bash
pip install -r requirements.txt
```

4. **Configure environment variables**:
```bash
cp .env.example .env
# Edit .env with your settings
```

5. **Initialize the database** (happens automatically on first run):
```bash
python main.py
```

## Usage

### Running the Server

**Development mode** (with auto-reload):
```bash
python main.py
```

**Production mode** with uvicorn:
```bash
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
```

The API will be available at:
- API: http://localhost:8000
- Interactive Docs: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc

### API Endpoints

#### Scan Operations

**Start a new scan**:
```bash
POST /api/scans/start
Content-Type: application/json

{
  "network_range": "192.168.1.0/24",
  "scan_type": "quick",
  "port_range": null,
  "include_service_detection": true,
  "use_nmap": false
}
```

**Get scan status**:
```bash
GET /api/scans/{scan_id}/status
```

**List all scans**:
```bash
GET /api/scans?limit=50&offset=0
```

**Cancel a scan**:
```bash
DELETE /api/scans/{scan_id}/cancel
```

#### Host Operations

**List discovered hosts**:
```bash
GET /api/hosts?status=online&limit=100&offset=0
```

**Get host details**:
```bash
GET /api/hosts/{host_id}
```

**Get host by IP**:
```bash
GET /api/hosts/ip/{ip_address}
```

**Get host services**:
```bash
GET /api/hosts/{host_id}/services
```

**Get network statistics**:
```bash
GET /api/hosts/statistics
```

#### Topology Operations

**Get network topology**:
```bash
GET /api/topology?include_offline=false
```

**Get host neighbors**:
```bash
GET /api/topology/neighbors/{host_id}
```

#### WebSocket

**Connect to WebSocket for real-time updates**:
```javascript
const ws = new WebSocket('ws://localhost:8000/api/ws');

ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  console.log('Received:', message);
};

// Subscribe to scan updates
ws.send(JSON.stringify({
  type: 'subscribe',
  scan_id: 1
}));
```

### Configuration

Edit `.env` file or set environment variables:

```bash
# Database
DATABASE_URL=sqlite:///./network_scanner.db

# Application
APP_NAME=Network Scanner
DEBUG=False

# Scanning
DEFAULT_SCAN_TIMEOUT=3
MAX_CONCURRENT_SCANS=50
ENABLE_NMAP=True

# Network
DEFAULT_NETWORK_RANGE=192.168.1.0/24
SCAN_PRIVATE_NETWORKS_ONLY=True

# API
API_PREFIX=/api
CORS_ORIGINS=["http://localhost:3000"]

# Logging
LOG_LEVEL=INFO
LOG_FILE=logs/network_scanner.log
```

## Scan Types

### Quick Scan
- Scans top 15 most common ports
- Fast execution (~30 seconds for /24 network)
- Suitable for quick network discovery

### Standard Scan
- Scans top 1000 ports
- Balanced speed and coverage (~2-3 minutes)
- Recommended for most use cases

### Deep Scan
- Scans all 65535 ports
- Comprehensive but slow (~15-20 minutes)
- Use for thorough security audits

### Custom Scan
- Specify custom port ranges
- Example: "80,443,8000-8100"
- Flexible for specific needs

## Security Considerations

### Network Scanning Ethics
- **Only scan networks you own or have explicit permission to scan**
- Tool defaults to private network ranges only
- All scanning activity is logged

### Network Impact
- Rate limiting prevents network disruption
- Configurable timeout and concurrency settings
- Respectful scanning practices

### Application Security
- Input validation on all endpoints
- Network range validation (private networks only by default)
- No command injection vulnerabilities
- SQL injection protection via SQLAlchemy

## Development

### Running Tests
```bash
pytest tests/
```

### Code Style
```bash
# Format code
black app/

# Lint
pylint app/
flake8 app/
```

### Database Migrations
```bash
# Create migration
alembic revision --autogenerate -m "description"

# Apply migrations
alembic upgrade head
```

## Troubleshooting

### Port Scanning Issues

**Problem**: No hosts discovered
- Check network range is correct
- Ensure hosts are actually online (try pinging them)
- Firewall might be blocking scans

**Problem**: Slow scanning
- Reduce `MAX_CONCURRENT_SCANS` in config
- Use "quick" scan type instead of "deep"
- Check network latency

### Nmap Issues

**Problem**: Nmap not working
- Install nmap: `sudo apt-get install nmap` (Linux) or `brew install nmap` (macOS)
- Set `use_nmap: false` in scan config to use socket-based scanning
- Check nmap is in PATH

### Database Issues

**Problem**: Database locked
- Only one process can write to SQLite at a time
- Close other connections to the database
- Consider using PostgreSQL for multi-user scenarios

## Performance

### Benchmarks
- Quick scan (/24 network): ~30 seconds
- Standard scan (/24 network): ~2-3 minutes
- Deep scan (single host): ~15-20 minutes

### Optimization Tips
- Use socket-based scanning for speed (no nmap)
- Increase `MAX_CONCURRENT_SCANS` for faster execution
- Reduce `DEFAULT_SCAN_TIMEOUT` for quicker host checks
- Disable service detection for faster scans

## API Examples

### Python Client Example

```python
import requests

# Start a scan
response = requests.post('http://localhost:8000/api/scans/start', json={
    'network_range': '192.168.1.0/24',
    'scan_type': 'quick'
})
scan_id = response.json()['scan_id']

# Check status
status = requests.get(f'http://localhost:8000/api/scans/{scan_id}/status')
print(status.json())

# Get topology
topology = requests.get('http://localhost:8000/api/topology')
print(topology.json())
```

### JavaScript Client Example

```javascript
// Start a scan
const startScan = async () => {
  const response = await fetch('http://localhost:8000/api/scans/start', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      network_range: '192.168.1.0/24',
      scan_type: 'quick'
    })
  });
  const data = await response.json();
  return data.scan_id;
};

// Get hosts
const getHosts = async () => {
  const response = await fetch('http://localhost:8000/api/hosts');
  return await response.json();
};
```

## License

This project is provided as-is for educational and authorized network administration purposes only.

## Support

For issues or questions:
1. Check the troubleshooting section
2. Review API documentation at `/docs`
3. Check application logs in `logs/network_scanner.log`

## Roadmap

### Future Enhancements
- [ ] Vulnerability scanning integration
- [ ] Network change detection and alerting
- [ ] Historical trend analysis
- [ ] Scheduled scanning
- [ ] Export to PDF/PNG
- [ ] Multi-subnet support
- [ ] PostgreSQL support for larger deployments

## Contributing

Contributions are welcome! Please ensure:
- Code follows PEP 8 style guidelines
- All tests pass
- New features include tests
- Documentation is updated

---

**Author**: DevAgent  
**Version**: 1.0.0  
**Last Updated**: December 4, 2025