8,288
edits
Changes
snapshot-merge procedure
The Fedora LiveOS uses the [[wikipedia:Device mapper|Device-mapper]] service of the Linux kernel to manage the file stores on the device. This is the same service that is used by [[wikipedia:Logical Volume Manager (Linux)|Logical Volume Manager]] to provide disc partition services.
The Fedora LiveOS uses the [[wikipedia:Device mapper|Device-mapper]] service of the Linux kernel to manage the file stores on the device. This is the same service that is used by [[wikipedia:Logical Volume Manager (Linux)|Logical Volume Manager]] to provide disc partition services.
One limitation, mentioned above, is that the LiveOS persistent overlay is a write-once file space. This is related to its use of ''device mapper'' snapshots to merge a read-only file system image (copied from the compressed SquashFS.img on the read-only LiveCD or installation .iso file) with a [[wikipedia:Copy-on-write|Copy-on-write]] service that tracks only changed blocks of data in the snapshot (overlay) file and then re-referencing file pointers to the updated blocks.<ref>http://people.gnome.org/~markmc/code/merge-dm-snapshot.c</ref><ref>http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob;f=drivers/md/dm-snap-persistent.c;hb=HEAD</ref> Any changes to the operating system files are stored as differences from the base. As such, "deletions" of files are saved as additional difference references, and the originals are hidden. With this mechanism, physical storage space is consumed in the write-once file space rather than recovered.
One limitation, mentioned above, is that the LiveOS persistent overlay is a write-once file space. This is related to its use of ''device mapper'' snapshots to combine a read-only file system image (copied from the compressed SquashFS.img on the read-only LiveCD or installation .iso file) with a [[wikipedia:Copy-on-write|Copy-on-write]] service that tracks only changed blocks of data in the snapshot (overlay) file and then re-referencing file pointers to the updated blocks.<ref>http://people.gnome.org/~markmc/code/merge-dm-snapshot.c</ref><ref>http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob;f=drivers/md/dm-snap-persistent.c;hb=HEAD</ref> Any changes to the operating system files are stored as differences from the base. As such, "deletions" of files are saved as additional difference references, and the originals are hidden. With this mechanism, physical storage space is consumed in the write-once file space rather than recovered.
Consumption of the space allocated for persistent storage in the snapshot overlay file may be tracked with the ''device mapper'' {{Code|dmsetup status}} report. '''Sugar Cellar''' is a small, utility script which uses that service to allow for Learner testing and discovery. It will help Learners manage their storage resources and learn ways to economize limited resources.
Consumption of the space allocated for persistent storage in the snapshot overlay file may be tracked with the ''device mapper'' {{Code|dmsetup status}} report. '''Sugar Cellar''' is a small, utility script which uses that service to allow for Learner testing and discovery. It will help Learners manage their storage resources and learn ways to economize limited resources.
** there may be other good candidates...
** there may be other good candidates...
===Provide image recompression and overlay refreshment===
===Image recompression and overlay refreshment===
The Device-mapper snapshot-merge target allows one to merge changes in a persistent snapshot into the original filesystem (See notes from [[{{TALKPAGENAMEE}}|Andrew Gilmore]]). In a LiveOS image the original filesystem is compressed within a read-only SquashFS image. If one has sufficient free disk space available (for example, 4.3 GiB of free space for Fedora 17 Live Desktop—typically on an attached hard drive—plus about 660 MiB or more, depending on additions of software or root filesystem files, of free space on the LiveOS device filesystem), one can copy (and uncompress) the original filesystem to a working folder, invoke the snapshot-merge Device-mapper target, delete the old SquashFS image, replace it with a recompressed version of the merged LiveOS snapshot, and reset the original snapshot overlay before rebooting the LiveOS device.
The following Bash script demonstrates the process where {{Code|$1}} is the working folder and {{Code|$2}} is the LiveOS device (such as {{Code|/run/initramfs/livedev}}, for the currently booted LiveOS image):
{{Note/warning|Test and practice the following script!|This procedure will delete the LiveOS filesystem before replacing it with a recompressed merged version. Be sure to verify that you will have sufficient space for the replacement before running the full script. (You could comment out the deletion, and squash into a directory on a large working drive to determine the new filesystem size.)}}
<nowiki>#!/bin/bash
# 2012-08-20 00:27:28
# Prepare working directory.
TMPDIR=$1
mkdir $TMPDIR/LiveOS 2>/dev/null
# Mount, copy, unmount the original root filesystem.
SRCMNT=$(mktemp -d /media/XXXXXX)
mount $2 $SRCMNT
SQUASHMNT=$(mktemp -d /media/XXXXXX)
mount $SRCMNT/LiveOS/squashfs.img $SQUASHMNT
cp $SQUASHMNT/LiveOS/ext3fs.img $TMPDIR/LiveOS
umount $SQUASHMNT
# Prepare temporary devices for Device-mapper snapshot-merge.
FSDEV=$(losetup -f --show $TMPDIR/LiveOS/ext3fs.img)
OVDEV=$(losetup -f --show $SRCMNT/LiveOS/overlay-*)
FS_SIZE=$(blockdev -q --getsz $FSDEV)
# Invoke snapshot-merge target device.
dmsetup create merged <<< "0 $FS_SIZE snapshot-merge $FSDEV $OVDEV P 8"
# Wait for merge completion.
while state=$(dmsetup status merged)
state=${state#*snapshot-merge }
meta=${state#* }
alloc=${state%/*}
[[ $alloc != $meta ]]; do
printf '|'
sleep 0.5
printf '\b/'
sleep 0.5
printf '\b-'
sleep 0.5
printf '\b\'
sleep 0.5
done
# Clean up from merge.
dmsetup remove merged
losetup -d $OVDEV $FSDEV
# Replace SquashFS image
rm $SRCMNT/LiveOS/squashfs.img
mksquashfs $TMPDIR/LiveOS $SRCMNT/LiveOS/squashfs.img -comp xz -keep-as-directory
# Reset overlay.
dd if=/dev/zero of=$SRCMNT/LiveOS/overlay-* bs=64k count=1 conv=notrunc,fsync
# Clean up.
umount $SRCMNT
rmdir $SQUASHMNT $SRCMNT</nowiki>
[[Sugar on a Stick/Sugar Clone]] provides a method to build a customized LiveOS image installation file through the [[Sugar_on_a_Stick/Sugar_Clone#SoaS-remix | SoaS-remix]] bundle. The SoaS-remix.iso installation file can be used to redistribute the customized image. A new installation from this .iso will have the old image recompressed and a fresh overlay. With the current version, the home.img file must be manually copied to a new installation. One's aging SoaS image can be refreshed by an over-installation (see the instructions on the referenced page).
[[Sugar on a Stick/Sugar Clone]] provides a method to build a customized LiveOS image installation file through the [[Sugar_on_a_Stick/Sugar_Clone#SoaS-remix | SoaS-remix]] bundle. The SoaS-remix.iso installation file can be used to redistribute the customized image. A new installation from this .iso will have the old image recompressed and a fresh overlay. With the current version, the home.img file must be manually copied to a new installation. One's aging SoaS image can be refreshed by an over-installation (see the instructions on the referenced page).
where overlay_size and size_increase are now 512 byte units.
where overlay_size and size_increase are now 512 byte units.
Once you have a working filesystem, you should proceed to merge your overlay into the original filesystem as describe [[#Image recompression and overlay refreshment|above]].
==References==
==References==
<references />
<references />
:: See this [http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/14644 dm-devel thread].
:: See this [http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/14644 dm-devel thread].