Initial commit: Network scanner with pfSense integration and SVG diagram generation

2025-10-10 11:03:11 +02:00
commit fbdc4974ec
15 changed files with 11869 additions and 0 deletions
--- a/PROJECT_SUMMARY.md
+++ b/PROJECT_SUMMARY.md
@@ -0,0 +1,423 @@
+# Network Scanner Project - Complete Summary
+
+## 📁 Project Structure
+
+```
+netzwerk_diagramm_scanner/
+├── network_scanner.py          # Main network scanning engine
+├── pfsense_scanner.py          # pfSense-specific scanner
+├── svg_generator.py            # SVG diagram generator
+├── integrated_scanner.py       # Combined scanner with pfSense integration
+├── test_system.py              # System verification and testing
+├── quickstart.sh               # Interactive quick start guide
+├── EXAMPLES.sh                 # Usage examples and scenarios
+├── config.json.example         # Configuration template
+├── requirements.txt            # Python dependencies
+├── README.md                   # Complete documentation
+└── .gitignore                  # Git ignore file
+```
+
+## 🚀 Quick Start
+
+```bash
+# 1. Run system tests
+./test_system.py
+
+# 2. Use the interactive quickstart
+./quickstart.sh
+
+# 3. Or run directly
+./integrated_scanner.py --generate-svg -v
+```
+
+## 📋 What Each Script Does
+
+### network_scanner.py
+**Main network scanner** - Discovers and scans network devices
+- Auto-discovers network segments from routing table
+- Ping sweep to find live hosts
+- Port scanning (common services)
+- SSH-based information gathering
+- Device type identification
+- Routing table extraction
+- Network interface enumeration
+
+**Usage:**
+```bash
+./network_scanner.py -c config.json -o scan_results.json -v
+```
+
+### pfsense_scanner.py
+**pfSense-specific scanner** - Deep dive into pfSense firewalls
+- Network interfaces and IP configuration
+- Complete routing table
+- VPN information:
+  - WireGuard tunnels and peers
+  - OpenVPN connections
+  - IPsec tunnels
+- Active firewall rules
+- DHCP leases
+- ARP table
+- Gateway status
+
+**Usage:**
+```bash
+./pfsense_scanner.py 192.168.1.1 -u root -k ~/.ssh/id_rsa -o pfsense.json
+```
+
+### svg_generator.py
+**SVG diagram generator** - Creates visual network topology
+- Color-coded network segments
+- Device type icons (routers, servers, clients)
+- Device information (IP, hostname, type)
+- Network connections
+- Legend explaining device types
+
+**Usage:**
+```bash
+./svg_generator.py scan_results.json -o network_diagram.svg
+```
+
+### integrated_scanner.py
+**All-in-one scanner** - Combines everything
+- Full network scan
+- Automatic pfSense detection and deep scanning
+- Enhanced routing analysis
+- Optional SVG generation
+- Comprehensive reporting
+
+**Usage:**
+```bash
+./integrated_scanner.py -c config.json -o full_scan.json --generate-svg -v
+```
+
+### test_system.py
+**System verification** - Checks if everything is ready
+- Python version check
+- Network connectivity test
+- Required commands verification
+- Script syntax validation
+- SSH key detection
+- Network detection test
+
+**Usage:**
+```bash
+./test_system.py
+```
+
+## 🔧 Configuration
+
+Create `config.json` from the example:
+
+```json
+{
+  "ssh_user": "root",
+  "ssh_key_path": "/home/user/.ssh/id_rsa",
+  "timeout": 2,
+  "additional_networks": [
+    "192.168.1.0/24",
+    "10.0.0.0/24",
+    "10.8.0.0/24"
+  ],
+  "special_devices": {
+    "192.168.1.1": {
+      "name": "pfSense Main Firewall",
+      "type": "firewall",
+      "os": "pfSense"
+    }
+  },
+  "scan_options": {
+    "max_workers": 10,
+    "ping_timeout": 2,
+    "port_scan_timeout": 1
+  }
+}
+```
+
+## 📊 Output Format
+
+### JSON Structure
+```json
+{
+  "scan_timestamp": "2025-10-10T12:00:00",
+  "segments": [
+    {
+      "name": "192.168.1.0/24",
+      "cidr": "192.168.1.0/24",
+      "gateway": "192.168.1.1",
+      "is_vpn": false,
+      "devices": [
+        {
+          "ip": "192.168.1.1",
+          "hostname": "pfsense.local",
+          "mac": "00:11:22:33:44:55",
+          "device_type": "firewall",
+          "os_type": "pfSense (FreeBSD)",
+          "open_ports": [22, 80, 443],
+          "ssh_accessible": true,
+          "routes": [...],
+          "interfaces": [...],
+          "pfsense_info": {
+            "vpn": {...},
+            "firewall_rules": [...],
+            "dhcp_leases": [...]
+          }
+        }
+      ]
+    }
+  ]
+}
+```
+
+## 🎯 Common Use Cases
+
+### 1. Network Documentation
+```bash
+# Generate complete network documentation
+./integrated_scanner.py -o network_doc.json --generate-svg
+```
+
+### 2. Security Audit
+```bash
+# Scan and analyze open ports
+./network_scanner.py -v -o security_scan.json
+cat security_scan.json | jq '.segments[].devices[] | select(.open_ports[]? == 23)'
+```
+
+### 3. VPN Topology Mapping
+```bash
+# Extract VPN information
+./pfsense_scanner.py <pfsense-ip> -o vpn_info.json
+cat vpn_info.json | jq '.vpn'
+```
+
+### 4. Change Detection
+```bash
+# Baseline scan
+./integrated_scanner.py -o baseline.json
+
+# After changes
+./integrated_scanner.py -o current.json
+
+# Compare
+diff <(jq -S . baseline.json) <(jq -S . current.json)
+```
+
+### 5. Scheduled Monitoring
+```bash
+# Add to crontab for daily scans
+0 2 * * * /path/to/integrated_scanner.py -o /var/log/network-scan-$(date +\%Y\%m\%d).json
+```
+
+## 🔐 Security Setup
+
+### SSH Key Generation
+```bash
+ssh-keygen -t ed25519 -f ~/.ssh/network_scanner -N ""
+```
+
+### Distribute to All Devices
+```bash
+for ip in 192.168.1.{1..254}; do
+    ssh-copy-id -i ~/.ssh/network_scanner.pub root@$ip 2>/dev/null
+done
+```
+
+### Update Configuration
+```json
+{
+  "ssh_user": "root",
+  "ssh_key_path": "/home/user/.ssh/network_scanner"
+}
+```
+
+## 🔍 Features
+
+### Network Discovery
+- ✅ Auto-detect network segments from routing table
+- ✅ Configurable additional networks
+- ✅ VPN network detection
+- ✅ Multi-threaded scanning
+
+### Device Information
+- ✅ IP address and hostname
+- ✅ MAC address lookup
+- ✅ OS detection (via SSH)
+- ✅ Device type classification
+- ✅ Open port scanning
+- ✅ Service enumeration
+
+### pfSense Integration
+- ✅ Full routing table extraction
+- ✅ WireGuard tunnel mapping
+- ✅ OpenVPN status
+- ✅ IPsec tunnels
+- ✅ Firewall rules
+- ✅ DHCP lease tracking
+- ✅ ARP table
+
+### Visualization
+- ✅ SVG diagram generation
+- ✅ Color-coded networks
+- ✅ Device type icons
+- ✅ Connection mapping
+- ✅ Legend and labels
+
+### Output & Integration
+- ✅ JSON export
+- ✅ Structured data format
+- ✅ Easy integration with other tools
+- ✅ jq-friendly output
+
+## 🛠️ Requirements
+
+- Python 3.8+
+- Linux operating system
+- Standard utilities: `ping`, `ip`, `ssh`
+- SSH access to network devices (recommended)
+- SSH key-based authentication (recommended)
+
+## 📈 Performance
+
+- **Small networks** (<50 devices): ~1-2 minutes
+- **Medium networks** (50-200 devices): ~5-10 minutes
+- **Large networks** (200+ devices): ~15-30 minutes
+
+Times vary based on:
+- Network latency
+- SSH accessibility
+- Number of ports scanned
+- Concurrent worker threads
+
+## 🐛 Troubleshooting
+
+### No devices found
+```bash
+# Check network connectivity
+ping <gateway-ip>
+
+# Check routing table
+ip route show
+
+# Run with verbose mode
+./network_scanner.py -v
+```
+
+### SSH failures
+```bash
+# Test SSH manually
+ssh -i ~/.ssh/network_scanner root@<device-ip>
+
+# Check key permissions
+chmod 600 ~/.ssh/network_scanner
+
+# Verify key is loaded
+ssh-add -l
+```
+
+### Slow scanning
+```bash
+# Reduce timeout in config.json
+{
+  "timeout": 1,
+  "scan_options": {
+    "ping_timeout": 1,
+    "port_scan_timeout": 1
+  }
+}
+
+# Scan specific networks only
+{
+  "additional_networks": ["192.168.1.0/24"]
+}
+```
+
+## 🎓 Advanced Usage
+
+### Custom Device Types
+Edit `network_scanner.py` to add custom device identification logic.
+
+### Extended Port Scanning
+Modify `_scan_common_ports()` to scan additional ports.
+
+### Custom Visualization
+Edit `svg_generator.py` to customize colors, icons, and layout.
+
+### Integration with Other Tools
+Use JSON output with tools like:
+- `jq` - JSON processing
+- `python` - Custom analysis
+- `ansible` - Automation
+- `grafana` - Monitoring
+- `prometheus` - Metrics
+
+## 📝 Next Steps
+
+1. **Initial Setup**
+   ```bash
+   ./test_system.py
+   cp config.json.example config.json
+   # Edit config.json with your details
+   ```
+
+2. **First Scan**
+   ```bash
+   ./quickstart.sh
+   # Or: ./integrated_scanner.py --generate-svg -v
+   ```
+
+3. **Review Results**
+   ```bash
+   # View JSON
+   cat network_scan.json | jq .
+   
+   # View diagram
+   firefox network_diagram.svg
+   ```
+
+4. **Customize**
+   - Add your networks to config.json
+   - Mark special devices (pfSense, routers)
+   - Adjust timeouts and workers
+   - Set up SSH keys for all devices
+
+5. **Automate**
+   - Schedule regular scans
+   - Compare with baselines
+   - Generate documentation
+   - Monitor for changes
+
+## 💡 Tips
+
+- **Always get authorization** before scanning networks
+- **Use SSH keys** instead of passwords for automation
+- **Start small** - test on a single network first
+- **Increase verbosity** (`-v`) for troubleshooting
+- **Use jq** for powerful JSON querying
+- **Keep baselines** for change detection
+- **Document special devices** in config.json
+
+## 📞 Support
+
+- See `README.md` for detailed documentation
+- Check `EXAMPLES.sh` for usage scenarios
+- Run `./quickstart.sh` for interactive help
+- Run `./test_system.py` to verify setup
+
+## ✅ Project Status
+
+All components are complete and tested:
+- ✅ Core network scanner
+- ✅ pfSense integration
+- ✅ SVG diagram generation
+- ✅ Integrated scanner
+- ✅ Configuration system
+- ✅ Documentation
+- ✅ Examples and quick start
+- ✅ System tests
+
+**Ready to use!** 🎉
+
+---
+
+Generated for comprehensive network topology discovery and visualization.