mindesbunister b8e06617e8 Add pfSense XML integration and complete workflow automation
- Add pfsense_integrator.py for automatic XML parsing and integration
- Add complete_workflow.sh for one-command network discovery
- Enhance integrated_scanner.py to auto-integrate pfSense XML files
- Update README with pfSense XML features and workflow
- Generate comprehensive network summaries from XML configs
- Support for WireGuard, OpenVPN, IPsec, routing, DHCP, firewall rules
2025-10-10 11:23:09 +02:00

Network Diagram Scanner

A comprehensive network topology discovery tool that scans local and VPN-connected networks, gathers detailed device information, extracts routing tables from pfSense and other routers, and generates SVG diagrams for network visualization.

Features

  • 🔍 Network Discovery: Automatically discovers all network segments from routing tables
  • 🖥️ Device Detection: Identifies devices via ping sweep and port scanning
  • 🔐 SSH Access: Gathers detailed information from SSH-accessible devices
  • 🛡️ pfSense Integration: Specialized scanner for pfSense firewalls (routing, VPN, firewall rules, DHCP)
  • 🌐 VPN Support: Scans networks connected via WireGuard, OpenVPN, and IPsec
  • 📊 SVG Diagram Generation: Creates visual network topology diagrams
  • 🔄 Routing Analysis: Extracts and analyzes routing tables from routers
  • 📝 JSON Export: Structured data output for further processing
  • 📄 pfSense XML Parsing: Automatically parses pfSense backup XML files for complete configuration analysis
  • 🔗 Automatic Integration: Seamlessly integrates pfSense XML data into network scans

Requirements

  • Python 3.8 or higher
  • Linux-based network (all scripts designed for Linux systems)
  • SSH access to network devices (recommended)
  • SSH key-based authentication (recommended for automation)

Installation

  1. Clone or download this repository:
cd /home/rwiegand/Nextcloud/entwicklung/Werkzeuge/netzwerk_diagramm_scanner
  1. Make scripts executable:
chmod +x network_scanner.py pfsense_scanner.py svg_generator.py
  1. (Optional) Install additional dependencies for enhanced features:
pip3 install -r requirements.txt

Configuration

  1. Copy the example configuration:
cp config.json.example config.json
  1. Edit config.json with your network details:
{
  "ssh_user": "root",
  "ssh_key_path": "/home/user/.ssh/id_rsa",
  "timeout": 2,
  "additional_networks": [
    "192.168.1.0/24",
    "10.0.0.0/24",
    "10.8.0.0/24"
  ],
  "special_devices": {
    "192.168.1.1": {
      "name": "pfSense Main Firewall",
      "type": "firewall",
      "os": "pfSense"
    }
  }
}

Configuration Options

  • ssh_user: SSH username for accessing devices (default: root)
  • ssh_key_path: Path to SSH private key for authentication
  • timeout: Connection timeout in seconds (default: 2)
  • additional_networks: List of network CIDRs to scan (in addition to auto-discovered ones)
  • special_devices: Dictionary of known devices with custom attributes

Usage

Basic Network Scan

Run a complete network scan:

./network_scanner.py

This will:

  1. Auto-discover network segments from your routing table
  2. Perform ping sweeps to find live hosts
  3. Scan common ports on each device
  4. Attempt SSH connections to gather detailed information
  5. Save results to network_scan.json

Custom Configuration

Use a custom config file and output location:

./network_scanner.py -c my_config.json -o my_scan_results.json

Verbose Output

Enable detailed logging:

./network_scanner.py -v

pfSense-Specific Scanning

Scan a pfSense firewall directly:

./pfsense_scanner.py 192.168.1.1 -u root -k ~/.ssh/id_rsa -o pfsense_info.json

This extracts:

  • Network interfaces and IP addresses
  • Complete routing table
  • VPN configurations (WireGuard, OpenVPN, IPsec)
  • Active firewall rules
  • DHCP leases
  • ARP table
  • Gateway status

Generate SVG Diagram

After scanning, generate a visual network diagram:

./svg_generator.py network_scan.json -o network_diagram.svg

Open the SVG in a web browser or image viewer:

firefox network_diagram.svg
# or
xdg-open network_diagram.svg

Output Format

JSON Structure

The scanner produces JSON with the following structure:

{
  "scan_timestamp": "2025-10-10T12:00:00",
  "segments": [
    {
      "name": "192.168.1.0/24",
      "cidr": "192.168.1.0/24",
      "gateway": "192.168.1.1",
      "is_vpn": false,
      "devices": [
        {
          "ip": "192.168.1.1",
          "hostname": "pfsense.local",
          "mac": "00:11:22:33:44:55",
          "device_type": "firewall",
          "os_type": "FreeBSD",
          "open_ports": [22, 80, 443],
          "ssh_accessible": true,
          "routes": [...],
          "interfaces": [...]
        }
      ]
    }
  ]
}

Workflow

Complete Network Discovery Workflow

  1. Configure SSH Access:

    # Generate SSH key if needed
    ssh-keygen -t ed25519 -f ~/.ssh/network_scanner
    
    # Copy to devices
    ssh-copy-id -i ~/.ssh/network_scanner root@192.168.1.1
    
  2. Create Configuration:

    cp config.json.example config.json
    nano config.json  # Edit with your settings
    
  3. Run Network Scan:

    ./network_scanner.py -c config.json -o scan_results.json -v
    
  4. Scan pfSense Devices (if not already captured):

    ./pfsense_scanner.py 192.168.1.1 -u root -k ~/.ssh/network_scanner -o pfsense.json
    
  5. Generate Diagram:

    ./svg_generator.py scan_results.json -o network_topology.svg
    
  6. View Results:

    firefox network_topology.svg
    

Complete Automated Workflow

For the ultimate network discovery experience, use the automated workflow script:

./complete_workflow.sh

This script will:

  1. Verify system requirements
  2. 🔍 Run integrated network scan (including pfSense XML if present)
  3. 🎨 Generate SVG network diagram
  4. 📋 Create network summary (if pfSense XML files exist)
  5. 📊 Display statistics and next steps

One-command network discovery!

SSH Access Setup

For automated scanning, SSH key-based authentication is recommended:

Generate SSH Key

ssh-keygen -t ed25519 -f ~/.ssh/network_scanner -N ""

Copy to All Devices

# For each device in your network
ssh-copy-id -i ~/.ssh/network_scanner.pub root@<device-ip>

Update config.json

{
  "ssh_user": "root",
  "ssh_key_path": "/home/username/.ssh/network_scanner"
}

Device Types

The scanner identifies the following device types:

  • router: Devices with extensive routing tables
  • firewall: pfSense and other firewall devices
  • switch: Network switches (if detectable)
  • server: Linux servers with SSH access
  • linux_server: Specifically identified Linux servers
  • windows_client: Windows machines (RDP port open)
  • client: Generic client devices

Advanced Features

Custom Device Identification

You can add custom device identification logic by modifying the _identify_device_type() method in network_scanner.py.

WireGuard VPN Scanning

The scanner automatically detects WireGuard tunnels on pfSense and Linux systems by:

  1. Checking for wg interfaces
  2. Extracting peer information
  3. Mapping allowed IPs to network segments

Routing Table Analysis

Routing information is extracted from:

  • Linux: ip route show
  • pfSense: netstat -rn
  • Other systems: Adaptable via SSH commands

Troubleshooting

No devices found

  • Check network connectivity: ping <gateway>
  • Verify ICMP is not blocked by firewalls
  • Try with verbose mode: -v

SSH connection failures

  • Verify SSH key permissions: chmod 600 ~/.ssh/network_scanner
  • Test manual SSH: ssh -i ~/.ssh/network_scanner root@<ip>
  • Check SSH service is running on target devices

Permission denied errors

  • Scanner needs root/sudo for some operations (ICMP ping)
  • Run with: sudo ./network_scanner.py

Large networks taking too long

  • Reduce scan scope by specifying specific networks in config
  • Increase timeout values
  • Use fewer worker threads

Examples

Scan only specific networks

Edit config.json:

{
  "additional_networks": [
    "192.168.1.0/24",
    "10.8.0.0/24"
  ]
}

Quick scan without SSH

Set empty SSH key path to skip SSH attempts:

{
  "ssh_key_path": null
}

Scan and immediately visualize

./network_scanner.py -o scan.json && ./svg_generator.py scan.json -o diagram.svg && xdg-open diagram.svg

Security Considerations

  • SSH Keys: Keep private keys secure with proper permissions (600)
  • Credentials: Never commit config.json with real credentials to version control
  • Network Access: This tool performs active scanning - ensure you have authorization
  • Firewall Rules: May trigger IDS/IPS systems - coordinate with network admins

Customization

Adding New Device Types

Edit network_scanner.py:

def _identify_device_type(self, device: Device) -> str:
    # Add your custom logic
    if device.hostname and 'myswitch' in device.hostname.lower():
        return 'switch'
    # ... existing logic

Custom SVG Styling

Edit svg_generator.py:

DEVICE_STYLES = {
    'my_custom_type': {'color': '#FF00FF', 'icon': '🎯', 'shape': 'rect'},
    # ... existing styles
}

Future Enhancements

  • SNMP support for switches and routers
  • Automatic link detection via LLDP/CDP
  • Interactive HTML diagram with zoom/pan
  • Historical comparison (detect network changes)
  • Integration with monitoring systems (Prometheus, Grafana)
  • Multi-threaded pfSense scanning for large deployments
  • Automatic VPN tunnel mapping

Contributing

Feel free to submit issues, feature requests, or pull requests.

License

This project is provided as-is for network documentation and analysis purposes.

Author

Created for comprehensive network topology discovery and visualization.


Note: Always ensure you have proper authorization before scanning networks. This tool performs active network reconnaissance.

pfSense XML Integration

The scanner can automatically parse pfSense backup XML files to extract comprehensive configuration data:

  • Network Interfaces: All interface configurations, IP addresses, VLANs
  • Routing Tables: Static routes, dynamic routing, gateway configurations
  • VPN Configurations: WireGuard tunnels, OpenVPN servers/clients, IPsec
  • Firewall Rules: NAT rules, port forwarding, access control lists
  • DHCP Services: Server configurations, static mappings, lease pools
  • DNS Settings: Resolvers, domain configurations
  • System Information: Hostname, domain, version, services

Automatic Integration: Place pfSense XML backup files in the scanner directory, and they will be automatically parsed and integrated into network scans.

Manual Parsing: Use pfsense_integrator.py to work with XML files independently:

# Parse XML files and generate summary
./pfsense_integrator.py *.xml --summary network_summary.md

# Integrate with existing scan
./pfsense_integrator.py *.xml -s scan.json -o enhanced_scan.json
Description
No description provided
Readme 278 KiB
Languages
Python 76.6%
Shell 23.4%