root/backup_to_external_m.2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Complete LVM migration toolset with fixes

Browse Source 
- Fixed partition size calculation bugs in migrate_to_lvm.sh
- Added comprehensive error handling for USB drive stability
- Optimized data copy operations using cp -a for better performance
- Corrected mount point detection for encrypted home partitions
- Enhanced drive detection and exclusion logic
- Added proper size override mechanisms for manual intervention
- Improved filesystem creation and validation processes
- Complete toolset for external M.2 drive migration scenarios

Tested successfully on:
- Debian 13 Trixie Live USB environment
- 476GB external M.2 drives via USB 3.0
- Complex partition layouts (root/home/EFI + encryption)
- Large data transfers (314GB+ encrypted home directories)
This commit is contained in:
Migration Tools
2025-09-25 05:53:12 +00:00
commit 9d25520de9
26 changed files with 7183 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

391
lvm-migration-tools/README.md Normal file
View File

@@ -0,0 +1,391 @@
# System Backup to External M.2 SSD
A comprehensive backup solution for Linux systems that provides both GUI and command-line interfaces for cloning your internal drive to an external M.2 SSD.
## Features
## Features
- **GUI Interface**: User-friendly graphical interface built with Python Tkinter
- **Command Line Interface**: Full CLI support for automated and scripted operations
- **Smart Drive Detection**: Automatically detects internal drive and external M.2 SSDs
- **Full System Backup**: Complete drive cloning with dd for exact system replication
- **Smart Sync Backup**: ⚡ NEW! Fast incremental backups using rsync for minor changes
- **Change Analysis**: Analyze filesystem changes to recommend sync vs full backup
- **Restore Functionality**: Complete system restore from external drive
- **Portable Tools**: Backup tools survive on external drive and remain accessible after cloning
- **Reboot Integration**: Optional reboot before backup/restore operations
- **Progress Monitoring**: Real-time progress display and logging
- **Safety Features**: Multiple confirmations and drive validation
- **Desktop Integration**: Create desktop shortcuts for easy access
## Requirements
- Linux system (tested on Ubuntu/Debian)
- Python 3.6+ with tkinter
- External M.2 SSD in USB enclosure
- sudo privileges for drive operations
## Installation
1. Clone or download this repository:
```bash
git clone <repository-url>
cd backup_to_external_m.2
```
2. Make scripts executable:
```bash
chmod +x backup_script.sh
chmod +x backup_manager.py
```
3. Install dependencies (if needed):
```bash
sudo apt update
sudo apt install python3-tk pv parted
```
4. **Optional**: Set up portable tools on external M.2 SSD:
```bash
./setup_portable_tools.sh
```
This creates a separate partition on your external drive that preserves backup tools even after cloning.
## Usage
### GUI Application
Launch the graphical backup manager:
```bash
python3 backup_manager.py
```
**Features:**
- Automatic drive detection and classification
- Source and target drive selection with smart defaults
- Real-time progress monitoring
- **Backup Modes**:
- **Smart Sync Backup**: ⚡ Fast incremental backup using rsync (requires existing backup)
- **Analyze Changes**: Analyze what has changed since last backup
- **Start Backup**: Full drive clone (immediate backup while system running)
- **Reboot & Backup**: Reboot system then full backup (recommended for first backup)
- **Restore Modes**:
- **Restore from External**: Immediate restore from external to internal
- **Reboot & Restore**: Reboot system then restore (recommended)
- **Drive Swap Button**: Easily swap source and target for restore operations
### Command Line Script
For command-line usage:
```bash
# List available drives
./backup_script.sh --list
# Analyze changes without performing backup
./backup_script.sh --analyze --target /dev/sdb
# Smart sync backup (fast incremental update)
sudo ./backup_script.sh --sync --target /dev/sdb
# Perform full backup with specific drives
sudo ./backup_script.sh --source /dev/nvme0n1 --target /dev/sda
# Restore from external to internal (note the restore flag)
sudo ./backup_script.sh --restore --source /dev/sda --target /dev/nvme0n1
# Or more simply in restore mode (source/target are swapped automatically)
sudo ./backup_script.sh --restore --source /dev/nvme0n1 --target /dev/sda
# Create desktop entry
./backup_script.sh --desktop
# Launch GUI from script
./backup_script.sh --gui
```
### Desktop Integration
Create a desktop shortcut:
```bash
./backup_script.sh --desktop
```
This creates a clickable icon on your desktop that launches the backup tool.
## Restore Operations
### When to Restore
- **System Failure**: Internal drive crashed or corrupted
- **System Migration**: Moving to new hardware
- **Rollback**: Reverting to previous system state
- **Testing**: Restoring test environment
### How to Restore
#### GUI Method
1. **Connect External Drive**: Plug in your M.2 SSD with backup
2. **Launch GUI**: `python3 backup_manager.py`
3. **Check Drive Selection**:
- Source should be external drive (your backup)
- Target should be internal drive (will be overwritten)
4. **Use Swap Button**: If needed, click "Swap Source↔Target"
5. **Choose Restore Mode**:
- **"Restore from External"**: Immediate restore
- **"Reboot & Restore"**: Reboot then restore (recommended)
#### Command Line Method
```bash
# Restore with automatic drive detection
sudo ./backup_script.sh --restore
# Restore with specific drives
sudo ./backup_script.sh --restore --source /dev/sda --target /dev/nvme0n1
```
### ⚠️ Restore Safety Warnings
- **Data Loss**: Target drive is completely overwritten
- **Irreversible**: No undo after restore starts
- **Multiple Confirmations**: System requires explicit confirmation
- **Drive Verification**: Double-check source and target drives
- **Boot Issues**: Ensure external drive contains valid system backup
## Portable Tools (Boot from External M.2)
### Setup Portable Tools
Run this once to set up backup tools on your external M.2 SSD:
```bash
./setup_portable_tools.sh
```
This will:
- Create a 512MB tools partition on your external drive
- Install backup tools that survive cloning operations
- Set up automatic tool restoration after each backup
### Using Portable Tools
#### When Booted from External M.2 SSD:
1. **Access Tools**:
```bash
# Easy access helper
./access_tools.sh
# Or manually:
sudo mount LABEL=BACKUP_TOOLS /mnt/tools
cd /mnt/tools/backup_system
./launch_backup_tools.sh
```
2. **Restore Internal Drive**:
- Launch backup tools from external drive
- External drive (source) → Internal drive (target)
- Click "Reboot & Restore" for safest operation
3. **Create Desktop Entry**:
```bash
cd /mnt/tools/backup_system
./create_desktop_entry.sh
```
### Disaster Recovery Workflow
1. **Normal Operation**: Internal drive fails
2. **Boot from External**: Use M.2 SSD as boot drive
3. **Access Tools**: Run `./access_tools.sh`
4. **Restore System**: Use backup tools to restore to new internal drive
5. **Back to Normal**: Boot from restored internal drive
## Safety Features
- **Drive Validation**: Prevents accidental overwriting of wrong drives
- **Size Checking**: Ensures target drive is large enough
- **Confirmation Prompts**: Multiple confirmation steps before destructive operations
- **Mount Detection**: Automatically unmounts target drives before backup
- **Progress Monitoring**: Real-time feedback during backup operations
## File Structure
```
backup_to_external_m.2/
├── backup_manager.py # GUI application
├── backup_script.sh # Command-line script
├── install.sh # Installation script
├── systemd/ # Systemd service files
│ └── backup-service.service
└── README.md # This file
```
## Smart Sync Technology ⚡
The backup system now includes advanced **Smart Sync** functionality that dramatically reduces backup time for incremental updates:
### How Smart Sync Works
1. **Analysis Phase**: Compares source and target filesystems to determine changes
2. **Decision Engine**: Recommends sync vs full clone based on amount of changes:
- **< 2GB changes**: Smart sync recommended (much faster)
- **2-10GB changes**: Smart sync beneficial
- **> 10GB changes**: Full clone may be more appropriate
3. **Sync Operation**: Uses rsync to transfer only changed files and metadata
### Smart Sync Benefits
- **Speed**: 10-100x faster than full clone for minor changes
- **No Downtime**: System remains usable during sync operation
- **Efficiency**: Only transfers changed data, preserving bandwidth and storage wear
- **Safety**: Preserves backup tools and maintains full system consistency
### When to Use Smart Sync vs Full Clone
**Use Smart Sync when:**
- You have an existing backup on the target drive
- Regular incremental updates (daily/weekly backups)
- Minimal system changes since last backup
- You want faster backup with minimal downtime
**Use Full Clone when:**
- First-time backup to a new drive
- Major system changes (OS upgrade, large software installations)
- Corrupted or incomplete previous backup
- Maximum compatibility and reliability needed
### Smart Sync Usage
**GUI Method:**
1. Click "Analyze Changes" to see what has changed
2. Review the recommendation and estimated time savings
3. Click "Smart Sync Backup" to perform incremental update
**Command Line:**
```bash
# Analyze changes first
./backup_script.sh --analyze --target /dev/sdb
# Perform smart sync
sudo ./backup_script.sh --sync --target /dev/sdb
```
## Traditional Full Backup
For comprehensive system backup, the system uses proven `dd` cloning technology:
### Backup Process
1. **Drive Detection**: Automatically scans for available drives
2. **Auto-Selection**: Internal drive as source, external as target
3. **Operation Selection**: Choose backup or restore mode
4. **Validation**: Confirms drives exist and are different
5. **Execution**: Uses `dd` command to clone entire drive
6. **Progress**: Shows real-time progress and logging
### Backup vs Restore
- **Backup**: Internal → External (preserves your system on external drive)
- **Restore**: External → Internal (overwrites internal with backup data)
- **Swap Button**: Easily switch source/target for restore operations
### Reboot vs Immediate Operations
- **Immediate**: Faster start, but system is running (potential file locks)
- **Reboot Mode**: System restarts first, then operates (cleaner, more reliable)
- **Recommendation**: Use reboot mode for critical operations
### Reboot & Backup Mode
1. **Script Creation**: Creates a backup script for post-reboot execution
2. **Reboot Scheduling**: Schedules system reboot
3. **Auto-Execution**: Backup starts automatically after reboot
4. **Notification**: Shows completion status
### Command Line Mode
- Direct `dd` cloning with progress monitoring
- Comprehensive logging to `/var/log/system_backup.log`
- Drive size validation and safety checks
## Technical Details
### Backup Method
- Uses `dd` command for bit-perfect drive cloning
- Block size optimized at 4MB for performance
- `fdatasync` ensures all data is written to disk
### Drive Detection
- Uses `lsblk` to enumerate block devices
- Filters for actual disk drives
- Shows drive sizes for easy identification
### Safety Mechanisms
- Root privilege verification
- Block device validation
- Source/target drive comparison
- Mount status checking
- Size compatibility verification
## Troubleshooting
### Common Issues
1. **Permission Denied**
```bash
# Run with sudo for drive operations
sudo python3 backup_manager.py
```
2. **Drive Not Detected**
```bash
# Check if drive is connected and recognized
lsblk
dmesg | tail
```
3. **Backup Fails**
```bash
# Check system logs
sudo journalctl -f
tail -f /var/log/system_backup.log
```
### Performance Tips
- Use USB 3.0+ connection for external M.2 SSD
- Ensure sufficient power supply for external enclosure
- Close unnecessary applications during backup
- Use SSD for better performance than traditional HDD
## Security Considerations
- **Data Destruction**: Target drive data is completely overwritten
- **Root Access**: Scripts require elevated privileges
- **Verification**: Always verify backup integrity after completion
- **Testing**: Test restore process with non-critical data first
## Limitations
- Only works with block devices (entire drives)
- Cannot backup to smaller drives
- Requires manual drive selection for safety
- No incremental backup support (full clone only)
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request
## License
This project is open source. Use at your own risk.
## Disclaimer
**WARNING**: This tool performs low-level disk operations that can result in data loss. Always:
- Verify drive selections carefully
- Test with non-critical data first
- Maintain separate backups of important data
- Understand the risks involved
The authors are not responsible for any data loss or system damage.