backup_to_external_m.2/README.md

# System Backup to External M.2 SSD

A comprehensive backup solution for Linux systems that provides both GUI and command-line interfaces for cloning your internal drive to an external M.2 SSD.

## Features

## Features

- **GUI Interface**: User-friendly graphical interface built with Python Tkinter
- **Command Line Interface**: Full CLI support for automated and scripted operations
- **Smart Drive Detection**: Automatically detects internal drive and external M.2 SSDs
- **Full System Backup**: Complete drive cloning with dd for exact system replication
- **Smart Sync Backup**: ⚡ NEW! Fast incremental backups using rsync for minor changes
- **Change Analysis**: Analyze filesystem changes to recommend sync vs full backup
- **Restore Functionality**: Complete system restore from external drive
- **Portable Tools**: Backup tools survive on external drive and remain accessible after cloning
- **Reboot Integration**: Optional reboot before backup/restore operations
- **Progress Monitoring**: Real-time progress display and logging
- **Safety Features**: Multiple confirmations and drive validation
- **Desktop Integration**: Create desktop shortcuts for easy access

## Requirements

- Linux system (tested on Ubuntu/Debian)
- Python 3.6+ with tkinter
- External M.2 SSD in USB enclosure
- sudo privileges for drive operations

## Installation

1. Clone or download this repository:
   ```bash
   git clone <repository-url>
   cd backup_to_external_m.2
   ```

2. Make scripts executable:
   ```bash
   chmod +x backup_script.sh
   chmod +x backup_manager.py
   ```

3. Install dependencies (if needed):
   ```bash
   sudo apt update
   sudo apt install python3-tk pv parted
   ```

4. **Optional**: Set up portable tools on external M.2 SSD:
   ```bash
   ./setup_portable_tools.sh
   ```
   This creates a separate partition on your external drive that preserves backup tools even after cloning.

## Usage

### GUI Application

Launch the graphical backup manager:

```bash
python3 backup_manager.py
```

**Features:**
- Automatic drive detection and classification
- Source and target drive selection with smart defaults
- Real-time progress monitoring
- **Backup Modes**:
  - **Smart Sync Backup**: ⚡ Fast incremental backup using rsync (requires existing backup)
  - **Analyze Changes**: Analyze what has changed since last backup
  - **Start Backup**: Full drive clone (immediate backup while system running)
  - **Reboot & Backup**: Reboot system then full backup (recommended for first backup)
- **Restore Modes**:
  - **Restore from External**: Immediate restore from external to internal
  - **Reboot & Restore**: Reboot system then restore (recommended)
- **Drive Swap Button**: Easily swap source and target for restore operations

### Command Line Script

For command-line usage:

```bash
# List available drives
./backup_script.sh --list

# Analyze changes without performing backup
./backup_script.sh --analyze --target /dev/sdb

# Smart sync backup (fast incremental update)  
sudo ./backup_script.sh --sync --target /dev/sdb

# Perform full backup with specific drives
sudo ./backup_script.sh --source /dev/nvme0n1 --target /dev/sda

# Restore from external to internal (note the restore flag)
sudo ./backup_script.sh --restore --source /dev/sda --target /dev/nvme0n1

# Or more simply in restore mode (source/target are swapped automatically)
sudo ./backup_script.sh --restore --source /dev/nvme0n1 --target /dev/sda

# Create desktop entry
./backup_script.sh --desktop

# Launch GUI from script
./backup_script.sh --gui
```

### Desktop Integration

Create a desktop shortcut:

```bash
./backup_script.sh --desktop
```

This creates a clickable icon on your desktop that launches the backup tool.

## Restore Operations

### When to Restore
- **System Failure**: Internal drive crashed or corrupted
- **System Migration**: Moving to new hardware
- **Rollback**: Reverting to previous system state
- **Testing**: Restoring test environment

### How to Restore

#### GUI Method
1. **Connect External Drive**: Plug in your M.2 SSD with backup
2. **Launch GUI**: `python3 backup_manager.py`
3. **Check Drive Selection**: 
   - Source should be external drive (your backup)
   - Target should be internal drive (will be overwritten)
4. **Use Swap Button**: If needed, click "Swap Source↔Target"
5. **Choose Restore Mode**:
   - **"Restore from External"**: Immediate restore
   - **"Reboot & Restore"**: Reboot then restore (recommended)

#### Command Line Method
```bash
# Restore with automatic drive detection
sudo ./backup_script.sh --restore

# Restore with specific drives
sudo ./backup_script.sh --restore --source /dev/sda --target /dev/nvme0n1
```

### ⚠️ Restore Safety Warnings
- **Data Loss**: Target drive is completely overwritten
- **Irreversible**: No undo after restore starts
- **Multiple Confirmations**: System requires explicit confirmation
- **Drive Verification**: Double-check source and target drives
- **Boot Issues**: Ensure external drive contains valid system backup

## Portable Tools (Boot from External M.2)

### Setup Portable Tools
Run this once to set up backup tools on your external M.2 SSD:

```bash
./setup_portable_tools.sh
```

This will:
- Create a 512MB tools partition on your external drive
- Install backup tools that survive cloning operations
- Set up automatic tool restoration after each backup

### Using Portable Tools

#### When Booted from External M.2 SSD:

1. **Access Tools**:
   ```bash
   # Easy access helper
   ./access_tools.sh
   
   # Or manually:
   sudo mount LABEL=BACKUP_TOOLS /mnt/tools
   cd /mnt/tools/backup_system
   ./launch_backup_tools.sh
   ```

2. **Restore Internal Drive**:
   - Launch backup tools from external drive
   - External drive (source) → Internal drive (target)
   - Click "Reboot & Restore" for safest operation

3. **Create Desktop Entry**:
   ```bash
   cd /mnt/tools/backup_system
   ./create_desktop_entry.sh
   ```

### Disaster Recovery Workflow

1. **Normal Operation**: Internal drive fails
2. **Boot from External**: Use M.2 SSD as boot drive
3. **Access Tools**: Run `./access_tools.sh`
4. **Restore System**: Use backup tools to restore to new internal drive
5. **Back to Normal**: Boot from restored internal drive

## Safety Features

- **Drive Validation**: Prevents accidental overwriting of wrong drives
- **Size Checking**: Ensures target drive is large enough
- **Confirmation Prompts**: Multiple confirmation steps before destructive operations
- **Mount Detection**: Automatically unmounts target drives before backup
- **Progress Monitoring**: Real-time feedback during backup operations

## File Structure

```
backup_to_external_m.2/
├── backup_manager.py      # GUI application
├── backup_script.sh       # Command-line script
├── install.sh            # Installation script
├── systemd/              # Systemd service files
│   └── backup-service.service
└── README.md             # This file
```

## Smart Sync Technology ⚡

The backup system now includes advanced **Smart Sync** functionality that dramatically reduces backup time for incremental updates:

### How Smart Sync Works

1. **Analysis Phase**: Compares source and target filesystems to determine changes
2. **Decision Engine**: Recommends sync vs full clone based on amount of changes:
   - **< 2GB changes**: Smart sync recommended (much faster)
   - **2-10GB changes**: Smart sync beneficial 
   - **> 10GB changes**: Full clone may be more appropriate
3. **Sync Operation**: Uses rsync to transfer only changed files and metadata

### Smart Sync Benefits

- **Speed**: 10-100x faster than full clone for minor changes
- **No Downtime**: System remains usable during sync operation
- **Efficiency**: Only transfers changed data, preserving bandwidth and storage wear
- **Safety**: Preserves backup tools and maintains full system consistency

### When to Use Smart Sync vs Full Clone

**Use Smart Sync when:**
- You have an existing backup on the target drive
- Regular incremental updates (daily/weekly backups)
- Minimal system changes since last backup
- You want faster backup with minimal downtime

**Use Full Clone when:**
- First-time backup to a new drive
- Major system changes (OS upgrade, large software installations)
- Corrupted or incomplete previous backup
- Maximum compatibility and reliability needed

### Smart Sync Usage

**GUI Method:**
1. Click "Analyze Changes" to see what has changed
2. Review the recommendation and estimated time savings
3. Click "Smart Sync Backup" to perform incremental update

**Command Line:**
```bash
# Analyze changes first
./backup_script.sh --analyze --target /dev/sdb

# Perform smart sync
sudo ./backup_script.sh --sync --target /dev/sdb
```

## Traditional Full Backup

For comprehensive system backup, the system uses proven `dd` cloning technology:

### Backup Process
1. **Drive Detection**: Automatically scans for available drives
2. **Auto-Selection**: Internal drive as source, external as target
3. **Operation Selection**: Choose backup or restore mode
4. **Validation**: Confirms drives exist and are different
5. **Execution**: Uses `dd` command to clone entire drive
6. **Progress**: Shows real-time progress and logging

### Backup vs Restore
- **Backup**: Internal → External (preserves your system on external drive)
- **Restore**: External → Internal (overwrites internal with backup data)
- **Swap Button**: Easily switch source/target for restore operations

### Reboot vs Immediate Operations
- **Immediate**: Faster start, but system is running (potential file locks)
- **Reboot Mode**: System restarts first, then operates (cleaner, more reliable)
- **Recommendation**: Use reboot mode for critical operations

### Reboot & Backup Mode
1. **Script Creation**: Creates a backup script for post-reboot execution
2. **Reboot Scheduling**: Schedules system reboot
3. **Auto-Execution**: Backup starts automatically after reboot
4. **Notification**: Shows completion status

### Command Line Mode
- Direct `dd` cloning with progress monitoring
- Comprehensive logging to `/var/log/system_backup.log`
- Drive size validation and safety checks

## Technical Details

### Backup Method
- Uses `dd` command for bit-perfect drive cloning
- Block size optimized at 4MB for performance
- `fdatasync` ensures all data is written to disk

### Drive Detection
- Uses `lsblk` to enumerate block devices
- Filters for actual disk drives
- Shows drive sizes for easy identification

### Safety Mechanisms
- Root privilege verification
- Block device validation
- Source/target drive comparison
- Mount status checking
- Size compatibility verification

## Troubleshooting

### Common Issues

1. **Permission Denied**
   ```bash
   # Run with sudo for drive operations
   sudo python3 backup_manager.py
   ```

2. **Drive Not Detected**
   ```bash
   # Check if drive is connected and recognized
   lsblk
   dmesg | tail
   ```

3. **Backup Fails**
   ```bash
   # Check system logs
   sudo journalctl -f
   tail -f /var/log/system_backup.log
   ```

### Performance Tips

- Use USB 3.0+ connection for external M.2 SSD
- Ensure sufficient power supply for external enclosure
- Close unnecessary applications during backup
- Use SSD for better performance than traditional HDD

## Security Considerations

- **Data Destruction**: Target drive data is completely overwritten
- **Root Access**: Scripts require elevated privileges
- **Verification**: Always verify backup integrity after completion
- **Testing**: Test restore process with non-critical data first

## Limitations

- Only works with block devices (entire drives)
- Cannot backup to smaller drives
- Requires manual drive selection for safety
- No incremental backup support (full clone only)

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

## License

This project is open source. Use at your own risk.

## Disclaimer

**WARNING**: This tool performs low-level disk operations that can result in data loss. Always:
- Verify drive selections carefully
- Test with non-critical data first
- Maintain separate backups of important data
- Understand the risks involved

The authors are not responsible for any data loss or system damage.