Linux USB

These notes describe how I installed Linux on a USB flash drive. The intention was to provide a bootable system that could be used for recovery purposes if something should go wrong with the system on the hard drive. The system on the flash drive is basically a copy of most of the Linux system on the hard drive - doing it this way (rather than a fresh install) means that you automatically get configuration and personal preferences set up.

WARNING: these notes use /dev/sda as the device for the USB flash drive. You should substitute whatever is the correct device for your machine (it might be /dev/sdb or /dev/sdc, for example). Be careful not to overwrite a USB flash drive used for backups.

Configuring the kernel

In order for the kernel to boot from the flash drive, it must contain (compiled-in, not as modules) the components needed to read from the flash drive. These seem to be:

You could create a separate kernel for use on the flash drive, but it's simpler to use the same kernel on both the hard drive and the flash drive.

Partitioning the drive

Use 'fdisk /dev/sda' to partition the flash drive. It's probably already partitioned, with a single FAT partition, but this can be deleted and the partitioning done again from scratch. I used a single partition for the whole Linux system, though I may add a swap partition later. The files I installed total around 2.5GB, so a 4GB partition is more than big enough. I'm using the rest of the drive as a backup partition.

The device name for the flash drive may change each time it is plugged in - it might be /dev/sda on one occasion and /dev/sdb on the next. This means that we can't use device names in /etc/fstab. Instead, we can label the partitions we want to mount, e.g.:

We can then refer to the partition by its label instead of its device name. If you want to use a swap partition on the flash drive, you can label it when you initialize it, e.g.:

Copying files

I decided to install a fairly complete Linux system, with desktop, Firefox, acroread, etc. (but not OpenOffice) to make life easier if I ever have to use it for recovery purposes - imagine having to sort out a tricky filesystem corruption problem (say) without access to the internet. On the other hand, you don't want to copy files that aren't actually needed - they waste space on the flash drive and it takes time to copy them. One consequence of this is that there are a lot of cases where we want to create directories on the flash drive (with correct ownership and permissions), but we don't want to copy the files contained in those directories.

Another issue to be considered in copying files is how to keep the files on the flash drive up-to-date. Using the system for recovery could be much harder if, for example, the kernel or your Firefox bookmarks are old.

The rsync utility seems ideal both for doing the initial copying and for maintaining the copy. Its 'include' and 'exclude' options allow you to specify which files and directories are to be copied, and it's very efficient at updating the copy. Tweaking it to copy what I wanted took a while, but this was the result:

/usr/bin/rsync \
    --verbose \
    --archive \
    --delete \
    --sparse \
    --human-readable --human-readable \
    --log-file=update.log \
    --exclude=.svn \
    --exclude=.reiserfs_priv \
    --include=/bin \
    --exclude="/boot*/boot.*" \
    --exclude="/boot*/map" \
    --include="/boot*" \
    --exclude=/dev/.udev \
    --include=/dev \
    --exclude=/etc/adjtime \
    --exclude="/etc/blkid.tab*" \
    --exclude=/etc/cron.daily \
    --exclude=/etc/cron.hourly \
    --exclude=/etc/cron.monthly \
    --exclude=/etc/cron.weekly \
    --exclude=/etc/cups/certs/ \
    --exclude=/etc/fstab \
    --exclude=/etc/ld.so.cache \
    --exclude=/etc/lilo.conf \
    --exclude=/etc/motd \
    --exclude="/etc/mtab*" \
    --exclude=/etc/ntp/drift \
    --exclude=/etc/printcap \
    --exclude=/etc/random-seed \
    --include=/etc \
    --include=/home/fog/SYSTEM_NOTES \
    --include=/home/fog/bookmarks.html \
    --include=/home/fog/private \
    --include=/home/fog/private_raw \
    --include=/home/fog/sounds \
    --include=/home/fog/.Xresources \
    --include=/home/fog/.bash_profile \
    --include=/home/fog/.bashrc \
    --include=/home/fog/.config \
    --include=/home/fog/.keychain \
    --exclude="/home/fog/.mozilla/*/*/Cache" \
    --include=/home/fog/.mozilla \
    --include=/home/fog/.netrc \
    --include=/home/fog/.ssh \
    --include=/home/fog/.subversion \
    --include=/home/fog/.thunderbird \
    --include=/home/fog/.vimrc \
    --include=/home/fog/.xinitrc \
    --exclude="/home/fog/logs/*" \
    --include=/home/fog/logs \
    --exclude="/home/fog/*" \
    --include=/home/fog \
    --include=/home/wingpath/public_html \
    --exclude="/home/wingpath/*" \
    --include=/home/wingpath \
    --exclude="/home/*" \
    --include=/home \
    --include=/lib \
    --include=/media \
    --exclude="/mnt/*/*" \
    --include=/mnt \
    --exclude="/opt/*" \
    --include=/opt \
    --exclude="/proc/*" \
    --include=/proc \
    --include=/root/.bashrc \
    --include=/root/.bash_profile \
    --include=/root/.ssh \
    --include=/root/.subversion \
    --include=/root/.vimrc \
    --exclude="/root/*" \
    --include=/root \
    --exclude="/sys/*" \
    --include=/sbin \
    --include=/sys \
    --exclude="/tmp/*" \
    --include=/tmp \
    --exclude=/usr/local/Adobe/Reader8 \
    --include=/usr/local/Adobe \
    --include=/usr/local/bin \
    --include=/usr/local/lib \
    --include=/usr/local/sbin \
    --include=/usr/local/share \
    --exclude="/usr/local/*" \
    --exclude=/usr/src/* \
    --exclude=/usr/share/ghostscript \
    --exclude=/usr/share/gimp \
    --exclude=/usr/share/gtk-doc \
    --include=/usr \
    --exclude="/var/cache/*" \
    --exclude=/var/cvs \
    --exclude=/var/www/htdocs/website/logs \
    --exclude=/var/imap \
    --exclude="/var/lock/*" \
    --exclude="/var/log/apache/*" \
    --include=/var/log/apache \
    --exclude="/var/log/*" \
    --exclude="/var/run/named/*" \
    --include=/var/run/named \
    --exclude="/var/run/*" \
    --exclude="/var/spool/cups/tmp/*" \
    --exclude="/var/state/*" \
    --exclude=/var/svn \
    --exclude="/var/tmp/*" \
    --include=/var \
    --include=/www \
    --exclude="/*" \
    / \
    $TMPDIR

The last two arguments say that I am copying from '/' (the root of the hard-disk filesystem) to '$TMPDIR' (which is where I mounted /dev/sda1).

The remaining ('exclude' and 'include') options specify what is to be copied. Note that the final '--exclude="/*"' option ensures that only explicitly named top-level directories will be included. Be careful to put patterns that match the beginning of other patterns AFTER those other patterns, otherwise the longer patterns will never get used. For example, if you put '--exclude="/var/tmp/*"' after '--include=/var', all files in /var/tmp/ would be included because they would match the '--include=/var' pattern.

Installing specific files

The file /etc/fstab has to be different on the flash drive from the file on the hard drive - when booting from the flash drive we want to mount filesystems and swap from the flash drive, not the hard drive. Since this is a recovery system, any mounts of hard disk partitions should be done manually.

LABEL=UsbRoot /                reiserfs    noatime                  1   1
#LABEL=UsbSwap swap             swap        defaults                 0   0
tmpfs            /tmp             tmpfs       mode=1777        0   0
tmpfs            /var/tmp         tmpfs       mode=1777                0   0
/dev/cdrom       /mnt/cdrom       auto        noauto,user,ro           0   0
/dev/dvd         /mnt/dvd         auto        noauto,user,ro           0   0
devpts           /dev/pts         devpts      gid=5,mode=620           0   0
proc             /proc            proc        defaults                 0   0
none             /sys             sysfs       defaults                 0   0

You might want to edit /etc/rc.d/rc.local and/or remove the execute permission from some of the files in /etc/rc.d to prevent some servers starting up. If you do this, you should probably add '--exclude=/etc/rc.d' to the rsync command (after the initial copying) to prevent rsync resetting your changes.

Installing a bootstrap

LILO works fine on a flash drive. Recent versions use the Volume-ID to identify the drive, so if you install the bootstrap when your flash drive is /dev/sda, but it happens to be /dev/sdb when you boot (because you have anothed flash drive plugged in), LILO will handle it OK and tell the kernel the correct device for the root filesystem. In other words, any references to /dev/sda in the lilo.conf file are used only at the time you run the lilo command - they are replaced in the bootstrap itself by references to the Volume-ID.

I install the bootstrap when I am running Linux from the hard disk, with the flash drive mounted on a directory called '$TMPDIR'. This means that any flash-drive filenames that I give to lilo must be prefixed by the mount point. lilo uses default filenames for a couple of the files ('map' and 'boot.NNNN') that it writes - these must be specified explicitly to prevent lilo writing them to the hard disk. The lilo.conf file that I use for the flash drive is:

The "rootdelay=8" kernel parameter provides an 8 second delay before the kernel attempts to mount the root filing system. This was found to be necessary because the kernel takes about 5 seconds to initialize the USB-SCSI interface to the flash drive, and the kernel is multi-threaded... There is another kernel parameter, 'rootwait', which looks from its description as though it would do a more intelligent delay - but it doesn't work (tested on kernel 2.6.28.7).

The lilo.conf man page says that you can use a partition label in the "root=" line. This doesn't work - I think it relies on the use of a program/script in an initrd ("initial RAM disk") to find the partition with the specified label - the standard kernel doesn't seem to support it.

Booting

The BIOS on the laptops has a "boot menu" (press F12 when booting), which you might expect would provide an option for booting from a USB flash drive - it doesn't, so far as I can see.

In order to boot from the flash drive, you have to go into the BIOS setup (press F2 when booting). In the boot setup page, use the down-arrow key to highlight "Hard disks", press Enter, press F5/F6 to make the flash drive the first in the list, and then press F10 to reboot. This setting will persist so long as you leave the flash drive plugged in, but if you boot without the flash drive plugged in and then plug it in again you will have to repeat the setup.

On sage, the boot menu works, although you have to select 'Hard disks' (not one of the USB options) - this brings up another menu, which contains the USB flash drive.

Maintenance

I have put together a shell script (in /home/fog/Development/scripts/linux_usb.sh on sage) that does the first and last of these steps. It can also be used to do the initial filesystem copying and setup of /etc/fstab. Although it works on both nettle and sage, it would need modifying for other Linux installations (in particular, the references to /home/fog are obviously system-specific). By default, it uses 'UsbRoot' as the partition label, but the partition label can be passed as an argument if required. The script determines the device name from the partition label - there are no hard-coded references to /dev.

Booting Linux from a USB flash drive