backup_to_external_m.2/lvm-migration-tools/LIVE_USB_MIGRATION_GUIDE.md

LVM Migration Guide: Live USB to LVM System

Overview

This guide provides comprehensive instructions for migrating your current non-LVM system to an LVM-based system on an external M.2 SSD. This migration MUST be performed from a live USB system to avoid file system conflicts and ensure data integrity.

Why Migrate to LVM?

Benefits of LVM System:

• Instant Snapshots: Create consistent backups without downtime
• Flexible Storage: Resize volumes dynamically without repartitioning
• Advanced Backups: Snapshot-based backups with rollback capability
• Space Efficiency: Snapshots only store changes, not full copies
• System Recovery: Easy rollback to previous states

Prerequisites

Hardware Requirements

• Live USB System: Boot from any Linux live USB (Ubuntu, Debian, etc.)
• Internal Drive: Your current system (will remain unchanged)
• External M.2 SSD: Target drive for LVM system (will be formatted)
• Sufficient Space: External drive should be ≥ size of used space on internal drive

Software Requirements

• Live USB system with LVM tools (will be installed automatically)
• Network connection for package installation
• Root/sudo access on live system

Before You Begin

1. Create Recovery Environment

# Prepare live USB with migration tools
# Download latest Ubuntu/Debian live ISO
# Flash to USB drive using dd or balenaEtcher

2. Backup Important Data

⚠️ CRITICAL: While the internal drive remains unchanged, create an additional backup of critical data before proceeding.

3. Document Current System

# Boot your current system and document the configuration
lsblk -f > system_layout.txt
df -h > disk_usage.txt
cat /etc/fstab > fstab_backup.txt

Migration Process

Step 1: Boot from Live USB System

1. Shutdown your system completely
2. Insert live USB and connect external M.2 SSD
3. Configure BIOS/UEFI:
   • Set USB as first boot device
   • Ensure both internal and external drives are detected
4. Boot live system:
   • Select "Try Ubuntu" or "Live System" (not "Install")
   • Wait for desktop to load completely
5. Open terminal and gain root access:

   sudo -i
   

Step 2: Prepare Live System

# Download and prepare the migration tools
cd /tmp
git clone <your-repo> backup_tools
cd backup_tools

# Or if you have the tools on external drive already:
mkdir -p /mnt/temp
mount /dev/sda1 /mnt/temp  # Adjust device as needed
cp -r /mnt/temp/migration_tools/* /tmp/
umount /mnt/temp

# Prepare the live system
./prepare_live_system.sh

This script will:

• ✅ Verify you're running from live system
• 📦 Install required packages (lvm2, cryptsetup, rsync, etc.)
• 🔧 Load kernel modules for LVM
• 💽 Detect available drives
• 📁 Create migration workspace

Step 3: Run Migration Script

# Execute the migration (this will take 30-90 minutes)
./migrate_to_lvm.sh

The migration process includes:

1. Drive Detection (Automatic):

   Detecting drives...
   Available drives:
   1. /dev/nvme0n1 - 477GB Samsung SSD 980 (Internal)
   2. /dev/sda - 477GB Samsung T7 (External USB)
   
   Selected drives:
   Internal (source): /dev/nvme0n1
   External (target): /dev/sda
   

2. System Analysis:

   • Automatically detects partition layout
   • Identifies filesystem types
   • Handles encrypted partitions
   • Calculates optimal LVM sizes

3. Confirmation Prompts:

   ⚠️  WARNING: This will DESTROY all data on /dev/sda!
   
   Migration Summary:
   Source: /dev/nvme0n1 (non-LVM system)
   Target: /dev/sda (will become LVM system)
   Root size: 70G
   Home size: 350G
   Swap size: 16G
   Boot size: 2G
   
   Do you want to continue? [y/N]
   

4. LVM Layout Creation:

   • Creates GPT partition table
   • EFI boot partition (512MB)
   • LVM physical volume (remaining space)
   • Creates volume group and logical volumes

5. Data Migration:

   • Mounts source filesystems (handles encryption)
   • Copies all system data with rsync
   • Preserves permissions, links, and attributes
   • Updates system configuration files

6. System Configuration:

   • Updates /etc/fstab for LVM volumes
   • Configures initramfs for LVM support
   • Installs and configures GRUB bootloader
   • Creates LVM snapshot backup tools

Step 4: Validation and Testing

# Validate the migration
./validate_lvm_migration.sh

Validation checks:

• ✅ LVM volumes created correctly
• ✅ Filesystems are healthy
• ✅ Boot configuration is valid
• ✅ GRUB installation successful
• ✅ System files copied completely
• ✅ LVM snapshot capability working

Step 5: First Boot Test

1. Cleanup and shutdown:

   # Clean up and prepare for reboot
   sync
   umount -a
   shutdown -h now
   

2. Configure BIOS/UEFI:

   • Boot into BIOS/UEFI settings
   • Change boot order: External M.2 SSD as first boot device
   • Save and exit

3. Test boot from external M.2:

   • System should boot normally from external drive
   • Login and verify everything works
   • Check that all your files and settings are present

4. Verify LVM system:

   # Check LVM status
   sudo lvs
   sudo vgs
   sudo pvs
   
   # Check filesystem mounts
   df -h
   cat /proc/mounts | grep mapper
   

System Configuration Details

LVM Layout Created

Physical Volume: /dev/sda2
Volume Group: system-vg
Logical Volumes:
├── root (70G) - ext4 - mounted at /
├── home (350G) - ext4 - mounted at /home  
├── boot (2G) - ext4 - mounted at /boot
└── swap (16G) - swap - swap space

Additional:
├── /dev/sda1 (512M) - vfat - EFI boot partition - mounted at /boot/efi
└── Free space (~38G) - available for snapshots and volume expansion

Migration Advantages

Flexibility:

• Resize any volume without repartitioning
• Add new drives to volume group
• Move logical volumes between physical drives

Backup & Recovery:

• Create instant snapshots of any volume
• Rollback changes using snapshots
• Consistent backups without downtime

Space Management:

• Thin provisioning support
• Automatic space allocation
• Easy expansion and shrinking

Using LVM Snapshots

Basic Snapshot Operations

# Create snapshots for backup
sudo ./lvm_snapshot_backup.sh backup

# Snapshots are mounted at:
/mnt/backup/root  # Snapshot of root filesystem
/mnt/backup/home  # Snapshot of home filesystem
/mnt/backup/boot  # Snapshot of boot filesystem

# Perform backup to external storage
rsync -avH /mnt/backup/ /path/to/external/backup/

# Clean up snapshots
sudo ./lvm_snapshot_backup.sh remove

Advanced LVM Operations

# Extend a logical volume (add 10GB to home)
sudo lvextend -L +10G /dev/system-vg/home
sudo resize2fs /dev/system-vg/home

# Create additional logical volume
sudo lvcreate -L 20G -n data system-vg
sudo mkfs.ext4 /dev/system-vg/data
sudo mkdir /data
sudo mount /dev/system-vg/data /data

# Snapshot before system changes
sudo lvcreate -L 5G -s -n root-before-update /dev/system-vg/root

# Rollback if needed
sudo umount /
sudo lvconvert --merge /dev/system-vg/root-before-update
# Reboot to activate rollback

Troubleshooting Guide

Migration Issues

Migration Script Fails

# Check logs for detailed error information
tail -f /var/log/lvm-migration.log

# Common issues and solutions:

Issue	Cause	Solution
"Drive not found"	Drive not connected/detected	Check connections, try different USB port
"Insufficient space"	Target drive too small	Use larger drive or reduce partition sizes
"LVM tools not found"	Missing packages	Run prepare_live_system.sh first
"Permission denied"	Not running as root	Use sudo or sudo -i
"Mount failed"	Filesystem corruption	Check drive with fsck

Encrypted Partition Issues

# If encrypted partition unlock fails:
sudo cryptsetup luksOpen /dev/nvme0n1p3 temp-unlock
# Enter correct password
sudo cryptsetup close temp-unlock

Drive Detection Problems

# Manually check drives
lsblk -dpno NAME,SIZE,MODEL
sudo fdisk -l

# If drives not detected:
sudo partprobe  # Re-read partition tables
sudo udevadm settle  # Wait for device detection

Boot Issues After Migration

System Won't Boot from External Drive

1. Check BIOS/UEFI Settings:

   • Verify external M.2 is detected in BIOS
   • Set correct boot priority
   • Enable UEFI boot mode
   • Disable Secure Boot if necessary

2. Repair GRUB from Live USB:

   # Boot from live USB and mount LVM system
   sudo vgchange -ay system-vg
   sudo mount /dev/system-vg/root /mnt
   sudo mount /dev/system-vg/boot /mnt/boot
   sudo mount /dev/sda1 /mnt/boot/efi
   
   # Reinstall GRUB
   sudo mount --bind /dev /mnt/dev
   sudo mount --bind /proc /mnt/proc
   sudo mount --bind /sys /mnt/sys
   sudo chroot /mnt
   grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=debian
   update-grub
   exit
   
   # Cleanup and reboot
   sudo umount /mnt/dev /mnt/proc /mnt/sys
   sudo umount /mnt/boot/efi /mnt/boot /mnt
   sudo reboot
   

Emergency Recovery

If external system is completely broken:

1. Change BIOS boot order back to internal drive
2. Boot from original system (unchanged)
3. Re-attempt migration or continue with original system

LVM Issues

Volume Group Not Found

# Activate volume group manually
sudo vgchange -ay system-vg

# Scan for volume groups
sudo vgscan
sudo pvscan

Snapshot Issues

# Remove stuck snapshots
sudo umount /mnt/backup/root /mnt/backup/home 2>/dev/null || true
sudo lvremove -f system-vg/root-snapshot
sudo lvremove -f system-vg/home-snapshot

# Check volume group free space
sudo vgs system-vg

File System Corruption

# Check and repair LVM volumes
sudo fsck /dev/system-vg/root
sudo fsck /dev/system-vg/home
sudo fsck /dev/system-vg/boot

Recovery Procedures

Complete Rollback to Original System

If you decide to abandon LVM migration:

1. Boot from internal drive:

   • Change BIOS boot order to internal drive
   • Boot normally from original system

2. Reformat external drive (optional):

   # Wipe LVM configuration
   sudo dd if=/dev/zero of=/dev/sda bs=1M count=100
   # Or use backup tools to restore external drive
   

3. Continue with original system:

   • Everything remains as before migration
   • Use existing backup tools for regular backups

Retry Migration

If you want to attempt migration again:

1. Boot from live USB
2. Run migration script again:

   ./migrate_to_lvm.sh
   

   • Script will destroy existing LVM setup and recreate
   • Source system (internal) remains unchanged

Disaster Recovery

If both systems fail:

1. Boot from live USB

2. Mount internal drive for data recovery:

   mkdir -p /mnt/recovery
   
   # Mount root partition
   sudo mount /dev/nvme0n1p1 /mnt/recovery
   
   # If home is encrypted:
   sudo cryptsetup open /dev/nvme0n1p3 recovery-home
   sudo mkdir -p /mnt/recovery/home
   sudo mount /dev/mapper/recovery-home /mnt/recovery/home
   
   # Copy important data to external storage
   rsync -avH /mnt/recovery/home/username/ /path/to/safe/backup/
   

3. Fresh OS installation if needed:

   • Install fresh OS on any drive
   • Restore personal data from backup

Performance Optimization

LVM Performance Tuning

# Enable read-ahead for better performance
sudo blockdev --setra 2048 /dev/system-vg/root
sudo blockdev --setra 2048 /dev/system-vg/home

# Add to /etc/fstab for persistent read-ahead:
# /dev/system-vg/root / ext4 defaults,noatime 0 1
# /dev/system-vg/home /home ext4 defaults,noatime 0 2

Snapshot Management

# Monitor snapshot usage
sudo lvs -a -o lv_name,lv_size,data_percent system-vg

# Remove old snapshots regularly
sudo lvremove system-vg/old-snapshot-name

# Set up automatic snapshot cleanup (cron job)
echo '0 2 * * * root /usr/local/bin/lvm-snapshot-backup.sh remove' >> /etc/crontab

Best Practices

Regular Maintenance

1. Monitor disk space:

   sudo vgs system-vg  # Check volume group free space
   df -h  # Check filesystem usage
   

2. Regular snapshots:

   # Before system updates
   sudo lvcreate -L 5G -s -n pre-update-$(date +%Y%m%d) /dev/system-vg/root
   
   # Before major changes
   sudo ./lvm_snapshot_backup.sh backup
   

3. Backup strategy:

   • Daily: LVM snapshots to external storage
   • Weekly: Full system backup using existing tools
   • Monthly: Verify backup integrity

Security Considerations

• Encryption: Home data is no longer encrypted in LVM setup

  • Consider full disk encryption if security is critical
  • Use file-level encryption for sensitive data

• Access Control: Secure LVM management commands

  # Restrict LVM command access
  sudo chmod 750 /usr/local/bin/lvm-*
  

Summary

The migration successfully transforms your system from traditional partitions to a flexible LVM-based setup, providing:

✅ Instant snapshots for consistent backups
✅ Dynamic volume resizing without downtime
✅ Advanced backup strategies with rollback capability
✅ Space efficiency with thin provisioning
✅ System recovery options with snapshots

Your original system remains intact as a fallback, making this a low-risk enhancement to your backup and storage capabilities.