backup_to_external_m.2/lvm-migration-tools

System Backup and LVM Migration Tools

A comprehensive toolset for Linux system backup and LVM migration that provides both GUI and command-line interfaces for backing up to external M.2 SSDs and migrating systems to LVM.

Features

Traditional Backup Features

• GUI Interface: User-friendly graphical interface built with Python Tkinter
• Command Line Interface: Full CLI support for automated and scripted operations
• Smart Drive Detection: Automatically detects internal drive and external M.2 SSDs
• Full System Backup: Complete drive cloning with dd for exact system replication
• Smart Sync Backup: ⚡ Fast incremental backups using rsync for minor changes
• Change Analysis: Analyze filesystem changes to recommend sync vs full backup
• Restore Functionality: Complete system restore from external drive
• Portable Tools: Backup tools survive on external drive and remain accessible after cloning
• Reboot Integration: Optional reboot before backup/restore operations
• Progress Monitoring: Real-time progress display and logging
• Safety Features: Multiple confirmations and drive validation
• Desktop Integration: Create desktop shortcuts for easy access

🆕 LVM Migration Features

• Automated LVM Migration: Complete migration from traditional partitions to LVM

• Size Detection: Automatic source partition size detection and matching

• Data Preservation: Full data integrity with permissions, ownership, and timestamps

• External Drive Support: Optimized for USB 3.0+ M.2 external drives

• GRUB Installation: Automatic EFI bootloader installation and configuration

• System Validation: Built-in verification of migration success

• One-Button Operation: Complete migration with a single command

• GUI Interface: User-friendly graphical interface built with Python Tkinter

• Command Line Interface: Full CLI support for automated and scripted operations

• Smart Drive Detection: Automatically detects internal drive and external M.2 SSDs

• Full System Backup: Complete drive cloning with dd for exact system replication

• Smart Sync Backup: ⚡ NEW! Fast incremental backups using rsync for minor changes

• Change Analysis: Analyze filesystem changes to recommend sync vs full backup

• Restore Functionality: Complete system restore from external drive

• Portable Tools: Backup tools survive on external drive and remain accessible after cloning

• Reboot Integration: Optional reboot before backup/restore operations

• Progress Monitoring: Real-time progress display and logging

• Safety Features: Multiple confirmations and drive validation

• Desktop Integration: Create desktop shortcuts for easy access

Requirements

• Linux system (tested on Ubuntu/Debian)
• Python 3.6+ with tkinter
• External M.2 SSD in USB enclosure
• sudo privileges for drive operations

Installation

Traditional Backup Installation

1. Clone or download this repository:

   git clone <repository-url>
   cd backup_to_external_m.2
   

2. Make scripts executable:

   chmod +x backup_script.sh
   chmod +x backup_manager.py
   

3. Install dependencies (if needed):

   sudo apt update
   sudo apt install python3-tk pv parted
   

LVM Migration Setup

The LVM migration tools require a live USB environment:

1. Boot from Debian/Ubuntu Live USB
2. Clone this repository to the live environment
3. The migration script will automatically install required dependencies
4. Run the migration with a single command:

   sudo ./migrate_to_lvm.sh /dev/sdX
   

LVM Migration Process

The migration is now fully automated with a single command:

1. Preparation: Boot from live USB, run the migration script
2. Detection: Automatically detect source partition sizes and layout
3. LVM Setup: Create LVM physical volumes, volume groups, and logical volumes
4. Data Copy: Efficiently copy all filesystem data using optimized cp -a method
5. Boot Configuration: Automatically update fstab, install GRUB EFI bootloader
6. Validation: Verify all components are properly configured
7. Ready: System is immediately bootable from external drive

One-Button Migration

sudo ./migrate_to_lvm.sh /dev/sdX

The script automatically handles:

• ✅ Partition detection and size calculation
• ✅ LVM volume creation with optimal sizes
• ✅ Complete data transfer (root, home, boot, EFI)
• ✅ fstab configuration with proper UUIDs
• ✅ GRUB EFI bootloader installation
• ✅ initramfs updates for LVM support
• ✅ System validation and readiness check

Post-Migration Benefits

Once migrated to LVM, your system gains:

• Instant Snapshots: Create point-in-time snapshots before system changes
• Dynamic Resizing: Resize partitions without downtime
• Advanced Backups: Consistent snapshots for reliable backups
• Easy Rollback: Revert to previous snapshots if needed

4. Optional: Set up portable tools on external M.2 SSD:

   ./setup_portable_tools.sh
   

   This creates a separate partition on your external drive that preserves backup tools even after cloning.

Usage

GUI Application

Launch the graphical backup manager:

python3 backup_manager.py

Features:

• Automatic drive detection and classification
• Source and target drive selection with smart defaults
• Real-time progress monitoring
• Backup Modes:
  • Smart Sync Backup: ⚡ Fast incremental backup using rsync (requires existing backup)
  • Analyze Changes: Analyze what has changed since last backup
  • Start Backup: Full drive clone (immediate backup while system running)
  • Reboot & Backup: Reboot system then full backup (recommended for first backup)
• Restore Modes:
  • Restore from External: Immediate restore from external to internal
  • Reboot & Restore: Reboot system then restore (recommended)
• Drive Swap Button: Easily swap source and target for restore operations

Command Line Script

For command-line usage:

# List available drives
./backup_script.sh --list

# Analyze changes without performing backup
./backup_script.sh --analyze --target /dev/sdb

# Smart sync backup (fast incremental update)  
sudo ./backup_script.sh --sync --target /dev/sdb

# Perform full backup with specific drives
sudo ./backup_script.sh --source /dev/nvme0n1 --target /dev/sda

# Restore from external to internal (note the restore flag)
sudo ./backup_script.sh --restore --source /dev/sda --target /dev/nvme0n1

# Or more simply in restore mode (source/target are swapped automatically)
sudo ./backup_script.sh --restore --source /dev/nvme0n1 --target /dev/sda

# Create desktop entry
./backup_script.sh --desktop

# Launch GUI from script
./backup_script.sh --gui

Desktop Integration

Create a desktop shortcut:

./backup_script.sh --desktop

This creates a clickable icon on your desktop that launches the backup tool.

Restore Operations

When to Restore

• System Failure: Internal drive crashed or corrupted
• System Migration: Moving to new hardware
• Rollback: Reverting to previous system state
• Testing: Restoring test environment

How to Restore

GUI Method

1. Connect External Drive: Plug in your M.2 SSD with backup
2. Launch GUI: python3 backup_manager.py
3. Check Drive Selection:
   • Source should be external drive (your backup)
   • Target should be internal drive (will be overwritten)
4. Use Swap Button: If needed, click "Swap Source↔Target"
5. Choose Restore Mode:
   • "Restore from External": Immediate restore
   • "Reboot & Restore": Reboot then restore (recommended)

Command Line Method

# Restore with automatic drive detection
sudo ./backup_script.sh --restore

# Restore with specific drives
sudo ./backup_script.sh --restore --source /dev/sda --target /dev/nvme0n1

⚠️ Restore Safety Warnings

• Data Loss: Target drive is completely overwritten
• Irreversible: No undo after restore starts
• Multiple Confirmations: System requires explicit confirmation
• Drive Verification: Double-check source and target drives
• Boot Issues: Ensure external drive contains valid system backup

Portable Tools (Boot from External M.2)

Setup Portable Tools

Run this once to set up backup tools on your external M.2 SSD:

./setup_portable_tools.sh

This will:

• Create a 512MB tools partition on your external drive
• Install backup tools that survive cloning operations
• Set up automatic tool restoration after each backup

Using Portable Tools

When Booted from External M.2 SSD:

1. Access Tools:

   # Easy access helper
   ./access_tools.sh
   
   # Or manually:
   sudo mount LABEL=BACKUP_TOOLS /mnt/tools
   cd /mnt/tools/backup_system
   ./launch_backup_tools.sh
   

2. Restore Internal Drive:

   • Launch backup tools from external drive
   • External drive (source) → Internal drive (target)
   • Click "Reboot & Restore" for safest operation

3. Create Desktop Entry:

   cd /mnt/tools/backup_system
   ./create_desktop_entry.sh
   

Disaster Recovery Workflow

1. Normal Operation: Internal drive fails
2. Boot from External: Use M.2 SSD as boot drive
3. Access Tools: Run ./access_tools.sh
4. Restore System: Use backup tools to restore to new internal drive
5. Back to Normal: Boot from restored internal drive

Safety Features

• Drive Validation: Prevents accidental overwriting of wrong drives
• Size Checking: Ensures target drive is large enough
• Confirmation Prompts: Multiple confirmation steps before destructive operations
• Mount Detection: Automatically unmounts target drives before backup
• Progress Monitoring: Real-time feedback during backup operations

File Structure

backup_to_external_m.2/
├── backup_manager.py      # GUI application
├── backup_script.sh       # Command-line script
├── install.sh            # Installation script
├── systemd/              # Systemd service files
│   └── backup-service.service
└── README.md             # This file

Smart Sync Technology ⚡

The backup system now includes advanced Smart Sync functionality that dramatically reduces backup time for incremental updates:

How Smart Sync Works

1. Analysis Phase: Compares source and target filesystems to determine changes
2. Decision Engine: Recommends sync vs full clone based on amount of changes:
   • < 2GB changes: Smart sync recommended (much faster)
   • 2-10GB changes: Smart sync beneficial
   • > 10GB changes: Full clone may be more appropriate
3. Sync Operation: Uses rsync to transfer only changed files and metadata

Smart Sync Benefits

• Speed: 10-100x faster than full clone for minor changes
• No Downtime: System remains usable during sync operation
• Efficiency: Only transfers changed data, preserving bandwidth and storage wear
• Safety: Preserves backup tools and maintains full system consistency

When to Use Smart Sync vs Full Clone

Use Smart Sync when:

• You have an existing backup on the target drive
• Regular incremental updates (daily/weekly backups)
• Minimal system changes since last backup
• You want faster backup with minimal downtime

Use Full Clone when:

• First-time backup to a new drive
• Major system changes (OS upgrade, large software installations)
• Corrupted or incomplete previous backup
• Maximum compatibility and reliability needed

Smart Sync Usage

GUI Method:

1. Click "Analyze Changes" to see what has changed
2. Review the recommendation and estimated time savings
3. Click "Smart Sync Backup" to perform incremental update

Command Line:

# Analyze changes first
./backup_script.sh --analyze --target /dev/sdb

# Perform smart sync
sudo ./backup_script.sh --sync --target /dev/sdb

Traditional Full Backup

For comprehensive system backup, the system uses proven dd cloning technology:

Backup Process

1. Drive Detection: Automatically scans for available drives
2. Auto-Selection: Internal drive as source, external as target
3. Operation Selection: Choose backup or restore mode
4. Validation: Confirms drives exist and are different
5. Execution: Uses dd command to clone entire drive
6. Progress: Shows real-time progress and logging

Backup vs Restore

• Backup: Internal → External (preserves your system on external drive)
• Restore: External → Internal (overwrites internal with backup data)
• Swap Button: Easily switch source/target for restore operations

Reboot vs Immediate Operations

• Immediate: Faster start, but system is running (potential file locks)
• Reboot Mode: System restarts first, then operates (cleaner, more reliable)
• Recommendation: Use reboot mode for critical operations

Reboot & Backup Mode

1. Script Creation: Creates a backup script for post-reboot execution
2. Reboot Scheduling: Schedules system reboot
3. Auto-Execution: Backup starts automatically after reboot
4. Notification: Shows completion status

Command Line Mode

• Direct dd cloning with progress monitoring
• Comprehensive logging to /var/log/system_backup.log
• Drive size validation and safety checks

Technical Details

Backup Method

• Uses dd command for bit-perfect drive cloning
• Block size optimized at 4MB for performance
• fdatasync ensures all data is written to disk

Drive Detection

• Uses lsblk to enumerate block devices
• Filters for actual disk drives
• Shows drive sizes for easy identification

Safety Mechanisms

• Root privilege verification
• Block device validation
• Source/target drive comparison
• Mount status checking
• Size compatibility verification

Troubleshooting

Common Issues

1. Permission Denied

   # Run with sudo for drive operations
   sudo python3 backup_manager.py
   

2. Drive Not Detected

   # Check if drive is connected and recognized
   lsblk
   dmesg | tail
   

3. Backup Fails

   # Check system logs
   sudo journalctl -f
   tail -f /var/log/system_backup.log
   

Performance Tips

• Use USB 3.0+ connection for external M.2 SSD
• Ensure sufficient power supply for external enclosure
• Close unnecessary applications during backup
• Use SSD for better performance than traditional HDD

Security Considerations

• Data Destruction: Target drive data is completely overwritten
• Root Access: Scripts require elevated privileges
• Verification: Always verify backup integrity after completion
• Testing: Test restore process with non-critical data first

Limitations

• Only works with block devices (entire drives)
• Cannot backup to smaller drives
• Requires manual drive selection for safety
• No incremental backup support (full clone only)

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

License

This project is open source. Use at your own risk.

Disclaimer

WARNING: This tool performs low-level disk operations that can result in data loss. Always:

• Verify drive selections carefully
• Test with non-critical data first
• Maintain separate backups of important data
• Understand the risks involved

The authors are not responsible for any data loss or system damage.