Experiences setting up FreeBSD 6.2 to run as a router entirely off flash media.
A lot of these instructions assume a single go. Rebooting before the end can cause problems with missing files or incorrect permissions, which can be difficult to get out of. It should also be done in single user mode, to reduce the chance of daemons scribbling during preperation.
Patch rc.initdiskless to mount /conf from things other than nfs.
--- /etc/rc.initdiskless.orig Sun May 7 19:00:00 2006
+++ /etc/rc.initdiskless Mon Feb 12 08:03:03 2007
@@ -229,8 +229,25 @@
if [ `expr "$nfspt" : '\(.\)'` = "/" ]; then
nfspt="${nfsroot}${nfspt}"
fi
- mount_nfs $nfspt /conf
- chkerr $? "mount_nfs $nfspt /conf"
+
+ # <CyberLeo> Hack to allow mounting of any filesystem type, not just NFS (thumbdrive w/glabel *hint hint*)
+ # ufs:/dev/label/conf (or nfs:server:/root -- server:/root works too, if server isn't named ufs, msdos, etc..)
+ eval $(echo ${nfspt} | sed -E 's/^([^:]+):(.*)$/fstype="\1" device="\2"/')
+ case "${fstype}" in
+ ufs) ;;
+ msdosfs) ;;
+ cd9660) ;;
+ nfs) ;;
+ # Backwards compatibility. Hopefully someone didn't name their NFS server 'msdos'...
+ *) device="${fstype}:${device}"; fstype="nfs" ;;
+ esac
+
+ mount -t ${fstype} ${device} /conf
+ chkerr $? "mount -t ${fstype} ${device} /conf"
+
+ # mount_nfs $nfspt /conf
+ # chkerr $? "mount_nfs $nfspt /conf"
+
+ # </CyberLeo>
to_umount="/conf"
fi
Get things ready to be read-only. This includes moving all configuration directories together, and moving tmp into a writable space. This is best done in single-user mode.
# Move /tmp into to-be-writable space mv /tmp/* /tmp/.??* /var/tmp/ rmdir /tmp ln -svf /var/tmp /tmp # Move 'local' configuration into /etc mv /usr/local/etc/rc.d /etc/rc.local # An arbitrary name for the local init scripts directory mv /usr/local/etc/* /etc/ rmdir /usr/local/etc ln -svf /etc /usr/local/etc echo 'local_startup="/etc/rc.local"' >> /etc/rc.conf # Tell rc where to find the new location of the local init scripts
Enable diskless mode (rc.initdiskless)
touch /etc/diskless
Indicate where the config partition resides. Around here, it's also useful to modify /etc/fstab to set / as read-only, so that the changes are propagated properly.
mkdir /conf glabel label -v conf /dev/ad4 newfs -U /dev/label/conf echo "ufs:/dev/label/conf" > /conf/diskless_remount echo "/dev/label/conf /conf ufs ro,noauto,noatime,sync 2 2" > /etc/fstab
Set up the configuration. Ideally, this should be done on a 'clean' system with as many baseline enhancements as you want to keep. I.e. useful packages and ports, barebones passwd and group files, etc.. Additional configuration will be saved elsewhere, allowing a quick method of 'starting over' by simply removing the changed configuration.
mount -w /conf
mkdir -p /conf/{base,default}/{etc,var}
echo "10240" > /conf/base/etc/md_size # 5MB
echo "131072" > /conf/base/var/md_size # 64MB
There are two methods to handle the files. If you have ample space on your flash media, the file method can save accesses and writes to precious flash cells. If you are short on space, the archive method is tighter, but requires proper timestamping, and it writes all changed files each save.
When staging the base archives or filesets, it's useful to remove files that are not necessary across reboots, or are autogenerated if absent. These include SSH keys from /etc/ssh/ssh_host_* (They will be autogenerated, then saved in /conf/default when saveconfig is run, so that nonconflicting copies can be made without too much trouble), /var/run/sudo/* (most likely to expire across reboots anyways) and so on.
File Method
cp -vpR /etc /conf/base/etc
cp -vpR /var /conf/base/var
for I in /conf/base/var/log; do cat /dev/null > ${I}; done # Zero out logfiles to save space
rm -Rf /conf/base/var/tmp/* /conf/base/var/tmp/.??* # Remove stale temp files. Everything in here gets recreated on reboot anyways.
I haven't written a full saveconfig script for the file method, as I am using the archive method myself. However, the following rsync command line (requires rsync, from packages or ports) does the job, provided /conf is mounted. (It won't be left mounted after booting)
# Copy all changed files, times, permissions, ownership from /etc and /var to /conf/default/{etc,var},
# and consider /conf/base/{etc,var} as additional comparison files. This will keep all the files in /conf/base from being
# copied to /conf/default if they haven't been changed.
rsync --archive --delete --hard-links --exclude='*/tmp/*' --compare-dest=/conf/base /etc /var /conf/default/
Archive Method
cd /tmp # Temporary staging area
cp -vpR /var /tmp/
for I in var/log; do cat /dev/null > ${I}; done # Zero out logfiles to save space
rm -Rf /conf/base/var/tmp/* /conf/base/var/tmp/.??* # Remove stale temp files. Everything in here gets recreated on reboot anyways.
find var -print0 | xargs -0 touch -ht 200605080000.00 # Set the timestamps for all the unchanged files, so we can find the changed ones easier.
tar zcpvf /conf/base/var.cpio.gz var # Structure must be relative to / for extraction to work properly.
rm -Rf /var
cp -vpR /etc /tmp/
find etc -print0 | xargs -0 touch -ht 200605080000.00
touch -ht 200605080000.01 etc/diskless # This file is our baseline, as it will (hopefully) never change.
tar zcpvf /conf/base/etc.cpio.gz etc
rm -Rf /etc
The saveconfig script (700 root:wheel in /sbin)
#!/bin/sh
if [ ! -f /etc/diskless ]
then
echo "No flash setup detected."
exit 1
fi
# Mount /conf read-write, and remount if it already is.
if [ $(grep -c "/conf" /etc/fstab) -gt 0 ]
then
mount -w /conf
if [ $? -ne 1 ]
then
umount /conf
mount -w /conf
fi
fi
# Five oldest backups method:
# Remove the oldest backup
#[ -d /conf/backup/5 ] && rm -Rf /conf/backup/5
# Rotate the backups to make room
#for I in 4 3 2 1
#do
# dest=$(expr "${I}" + 1)
# [ -d "/conf/backup/${I}" ] && mv "/conf/backup/${I}" "/conf/backup/${dest}"
#done
# Back up previous config and create space for the new one
#mv /conf/default /conf/backup/1
# Unlimited history method: (make sure you have provisions for removing the old backups, or this can get HUGE!
dest=$(expr "$(ls -1 /conf/backup | tail -n 1)" + 1)
mv /conf/default "/conf/backup/${dest}"
mkdir /conf/default
# Copy changed config files to /conf/default (anything younger than /etc/diskless)
find /etc /var -type f -not -regex '.*/tmp/.*' -newer /etc/diskless -print0 | xargs -0 tar cpf - | tar xpCf /conf/default/ -
# This can be changed to allow compressed configuration here as well:
# cd / ; find /etc -type f -not -regex '.*/tmp/.*' -newer /etc/diskless -print0 | xargs -0 tar cpf /conf/default/etc.cpio.gz
# cd / ; find /var -type f -not -regex '.*/tmp/.*' -newer /etc/diskless -print0 | xargs -0 tar cpf /conf/default/var.cpio.gz
# Umount /conf afterwards
if [ "$(mount |grep -c "/conf")" -gt 0 ]
then
umount /conf
fi