Difference between revisions of "LiveOS image"

From Sugar Labs
Jump to navigation Jump to search
m (overlay is not compressed)
(correct Sugar Activity Storage locations)
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
<noinclude>
+
<noinclude>[[Category:Live USB]]
−
[[Category:Live USB]]
 
 
</noinclude>
 
</noinclude>
 +
''This page has been moved to the [[fedora:Fedora_Project_Wiki|Fedora Project Wiki]].''
  
−
==Introduction==
+
Additional information for [[Sugar on a Stick]] is available [[#Sugar Activity Storage|below the frame]].
−
Fedora has developed [[wikipedia:Live CD|Live CD]] [[wikipedia:Live USB|USB]] DVD images for their GNU/Linux operating system. Since the image file systems are stored in the /LiveOS folder of the image, this is the name we'll use to reference their product.
 
  
−
This page shares some information about the LiveOS design that helps [[Sugar on a Stick]] Learners make better use of their disc resources.
+
The frame below is an active view of the [[fedora:LiveOS image|LiveOS image]] page on the Fedora Project Wiki:
  
−
==References==
+
{{#widget:Iframe
 +
|url=https://fedoraproject.org/wiki/LiveOS_image
 +
|width=893
 +
|height=900
 +
|border=1
 +
}}
  
−
* [http://git.fedorahosted.org/git/?p=hosted/livecd;a=blob;f=tools/livecd-iso-to-disk.sh;hb=HEAD livecd-iso-to-disk] (Usage instructions are on the first pages.)
+
==Sugar Activity Storage==
−
* [[fedora:FedoraLiveCD|FedoraLiveCD]]
+
Activity bundles that come pre-installed with the Sugar on a Stick (SoaS) are saved in the /usr/share/sugar/activities folder of the root filesystem. When Activities are updated from the Sugar Control Panel, the new Activity bundles are saved in the /home/liveuser/Activities folder. When an Activity is run, Activity instance data is saved in the Sugar Journal in /home/liveuser/.sugar/default/<activity bundle_id> folder and /home/liveuser/.sugar/default/datastore/<hashed folder names>
−
* [[fedora:How to create and use a Live CD|How to create and use a Live CD]]
 
−
* [[fedora:How to create and use Live USB|How to create and use Live USB]]
 
  
−
==Storage==
+
When additional Activity bundles are downloaded, they are saved in the /home/liveuser/Activities folder (but not the /usr/share/sugar/activities folder).
−
When a Live CD or Live DVD (a LiveOS image on read-only disc media) is booted, temporary storage is prepared for the system in RAM on each boot by [http://git.kernel.org/?p=boot/dracut/dracut.git;a=blob;f=modules.d/90dmsquash-live/dmsquash-live-root;hb=HEAD /sbin/dmsquash-live-root] in initrd0, the initial ram disk filesystem. An in-memory, copy-on-write, system overlay is used (see [[#File Systems|File Systems]] below).
+
Thus, when SoaS is installed with the --home-size-mb NNN option of livecd-iso-to-disk, storage for the Activities one wants to try and, perhaps later, throw away, is made available without consuming the write-once root filesystem overlay, which can be consumed very quickly (and overlay file space is not directly reusable).
  
−
The Fedora LiveOS system allows for persistent storage in 3 locations:
+
==Device filesystem==
  
−
# An all-purpose, persistent overlay - a write-once, fixed-size file space that will save updates and changes to the LiveOS image (Activities, operating system changes, anything written in the LiveOS file space).
+
The filesystem of the USB/SD flash storage device before SoaS was loaded is available through the /run/initramfs/live mount point (/mnt/live before Fedora 17 or SoaS Quandong).  It would be helpful if the Sugar Journal code were adjusted to recognize this partition structure and offer this space as an attached disk to facilitate file sharing.)
−
# Persistent home folder - a re-writable, re-sizable (with difficulty), uncompressed, optionally-encryptable, file space for anything that goes in the Learner's /home/ folder (all the Sugar Activities, logs, and good stuff).  A persistent home folder is an option that may be selected at the time of installation of the LiveOS image (although with some effort, one could manually create a home.img filesystem in /LiveOS/ and move the /home/ folder to it). This installation option is only available through the script, '[[fedora:livecd-iso-to-disk|livecd-iso-to-disk]]'. The Windows and Fedora 'Live USB Creator' installers do not provide this option as of July 2011.
 
−
# Host device file space - this is the USB stick or SD card file system that is outside of the LiveOS file tree, but which is accessible through the /mnt/live folder mount point of a booted LiveOS installation.  There, one finds the boot configuration files and anything else one had on the device before loading the SoaS image.  One may save files here without consuming the other, limited file spaces. (This file space is limited by the device capacity).
 
  
−
The all-purpose, persistent overlay is needed for operating system changes and updates.
+
==Installation options==
 +
[[Sugar on a Stick]] may be installed on a 1 GB USB device using the following options with '''[[fedora:livecd-iso-to-disk|livecd-iso-to-disk]]''' (on a single, Terminal Activity or console command line, even though the wiki may wrap the following text in accord with your browser window size):
  
−
The file systems are prepared on each boot by the [http://git.fedorahosted.org/git/?p=spin-kickstarts.git;a=blob;f=fedora-live-base.ks;h=0b14feccfd99bd60d88dc55ebc32d48bc7251242;hb=HEAD#l63 /etc/rc.d/init.d/livesys script].
+
:{{Code|./livecd-iso-to-disk --reset-mbr --overlay-size-mb '''200''' --home-size-mb '''150''' --delete-home --unencrypted-home /path/to/source/iso/or/device /dev/sd'''?'''1}}
  
−
===Home filesystem===
+
:: where '{{Code|'''?'''}}' in the final parameter represents the target bootable device node, such as {{Code|sd'''b'''1}} or {{Code|sd'''c'''1}}, etc.
−
One may find many advantages to installing Sugar on a Stick, or any LiveOS, with a persistent, home folder (using the --home-size-mb NN --delete-home options), which will hold all the Activities one wants to try and, perhaps later, throw away&mdash;all without consuming the write-once overlay, which can be consumed very quickly (and overlay file space is not normally reusable).
 
−
 
 
−
===Device filesystem===
 
−
Additionally, keeping some storage space on the device disc outside of the LiveOS system will let you copy, carry, and delete large resource files, such as image.iso files, or anything you might want to use or share.  (We should adjust the Journal code to show this root mount to facilitate file sharing.)
 
−
 
 
−
===Installation options===
 
−
[[Sugar on a Stick]] may be installed on a 1-GB USB device using the following options with '''[[fedora:livecd-iso-to-disk|livecd-iso-to-disk]]''' (on a single, Terminal Activity or console command line, even though the wiki may wrap the following text in accord with your browser window size):
 
−
 
 
−
:{{Code|./livecd-iso-to-disk --reset-mbr --overlay-size-mb 300 --home-size-mb 200 --delete-home --unencrypted-home /path/to/source/iso/or/device /dev/sd'''?'''1}}
 
−
 
 
−
:: where '{{Code|'''?'''}}' in the final parameter represents the target bootable device node, such as {{Code|sdb1}} or {{Code|sdc1}}, etc.
 
−
:: (You may use '\' line-continuation symbols followed by a newline [enter or return keypress] to break a long line visually on the terminal, but not logically to the script processing software.)
 
  
 
The above configuration would allow space for the home folder, the operating system, and a little on the device root.
 
The above configuration would allow space for the home folder, the operating system, and a little on the device root.
Line 47: Line 36:
  
 
==File Systems==
 
==File Systems==
−
 
−
The Fedora LiveOS uses the [[wikipedia:Device mapper|Device-mapper]] service of the Linux kernel to manage the file stores on the device.  This is the same service that is used by [[wikipedia:Logical Volume Manager (Linux)|Logical Volume Manager]] to provide disc partition services.
 
−
 
−
One limitation, mentioned above, is that the LiveOS persistent overlay is a write-once file space.  This is related to its use of ''device mapper'' snapshots to merge a read-only file system image (copied from the compressed SquashFS.img on the read-only LiveCD or installation .iso file) with a [[wikipedia:Copy-on-write|Copy-on-write]] service that tracks only changed bits of data in the snapshot (overlay) file and then re-referencing file pointers to the updated bits.<ref>http://people.gnome.org/~markmc/code/merge-dm-snapshot.c</ref><ref>http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=drivers/md/dm-snap-persistent.c;hb=HEAD</ref>  Any changes to the operating system files are stored as differences from the base. As such, "deletions" of files are saved as additional difference references, and the originals are hidden.  With this mechanism, physical storage space is consumed in the write-once file space rather than recovered.
 
  
 
Consumption of the space allocated for persistent storage in the snapshot overlay file may be tracked with the ''device mapper'' {{Code|dmsetup status}} report.  '''Sugar Cellar''' is a small, utility script which uses that service to allow for Learner testing and discovery. It will help Learners manage their storage resources and learn ways to economize limited resources.
 
Consumption of the space allocated for persistent storage in the snapshot overlay file may be tracked with the ''device mapper'' {{Code|dmsetup status}} report.  '''Sugar Cellar''' is a small, utility script which uses that service to allow for Learner testing and discovery. It will help Learners manage their storage resources and learn ways to economize limited resources.
 
   
 
   
 
* ''Sugar Cellar'' is available at http://people.sugarlabs.org/fgrose/SugarCellar and as a component of [[Sugar on a Stick/Sugar Clone]].
 
* ''Sugar Cellar'' is available at http://people.sugarlabs.org/fgrose/SugarCellar and as a component of [[Sugar on a Stick/Sugar Clone]].
−
 
  
 
Several developments in the Sugar on a Stick design may be considered to maximize the endurance of the LiveOS image.
 
Several developments in the Sugar on a Stick design may be considered to maximize the endurance of the LiveOS image.
  
−
===Avoid persistent overlay consumption===
+
==Image customization, recompression, and overlay refreshment==
−
 
 
−
* Use a persistent /home folder (discussed above).
 
−
*: This is a very effective method to avoid consumption and will also make the Journal files more available for sharing and backup purposes.
 
−
* Move the root user's home to /home/root in a persistent home.img
 
−
*: This would avoid some consumption, but would compromise the root user account if there was a boot problem and the /home folder was not mounted.
 
−
* Mount more folders onto temporary, in-memory filesystems (like /var/cache/yum, /var/tmp, & /tmp are now).
 
−
** /var/lib/NetworkManager (holds a timestamps file that is deleted and refreshed quite often, every 5 minutes)
 
−
** /var/log/audit (holds the SELinux audit.log that records a great many file accesses.)
 
−
** /var/spool/abrt (holds often large, error reports and core dumps).  Learners could be advised to act on any abrt reports in the current session or copy the reports to other permanent storage, such as in /home/ or external storage.
 
−
** there may be other good candidates...
 
−
 
 
−
===Provide image recompression and overlay refreshment===
 
  
 
[[Sugar on a Stick/Sugar Clone]] provides a method to build a customized LiveOS image installation file through the [[Sugar_on_a_Stick/Sugar_Clone#SoaS-remix | SoaS-remix]] bundle.  The SoaS-remix.iso installation file can be used to redistribute the customized image.  A new installation from this .iso will have the old image recompressed and a fresh overlay. With the current version, the home.img file must be manually copied to a new installation.  One's aging SoaS image can be refreshed by an over-installation (see the instructions on the referenced page).
 
[[Sugar on a Stick/Sugar Clone]] provides a method to build a customized LiveOS image installation file through the [[Sugar_on_a_Stick/Sugar_Clone#SoaS-remix | SoaS-remix]] bundle.  The SoaS-remix.iso installation file can be used to redistribute the customized image.  A new installation from this .iso will have the old image recompressed and a fresh overlay. With the current version, the home.img file must be manually copied to a new installation.  One's aging SoaS image can be refreshed by an over-installation (see the instructions on the referenced page).
Line 78: Line 50:
  
 
The remix building of a LiveOS image is also a good way to maintain and distribute system updates. Many software security, enhancement, and bug fixes are distributed through PackageKit (Software Update) or Yum update, which are turned off by default on LiveOS installations like SoaS.  After manually running an update, the image can be remixed to incorporate the changes into a new Updated-SoaS.iso file.  This process could be migrated to the Sugar BuildBot service to provide the Community with an up-to-date Sugar on a Stick image.
 
The remix building of a LiveOS image is also a good way to maintain and distribute system updates. Many software security, enhancement, and bug fixes are distributed through PackageKit (Software Update) or Yum update, which are turned off by default on LiveOS installations like SoaS.  After manually running an update, the image can be remixed to incorporate the changes into a new Updated-SoaS.iso file.  This process could be migrated to the Sugar BuildBot service to provide the Community with an up-to-date Sugar on a Stick image.
−
 
−
===Overlay recovery===
 
−
{{Note/warning|Test and practice the following|This procedure is not thoroughly validated, and may destroy your data.}}
 
−
If one 'exhausts' the limited storage capacity of a LiveOS overlay, Device-mapper will mark the filesystem as 'Invalid', as shown by the {{Code|dmsetup status}} command executed in Terminal or a console (if you haven't crashed):
 
−
# dmsetup status
 
−
live-osimg-min: 0 8388608 snapshot 2464/2464 24
 
−
live-rw: 0 8388608 snapshot Invalid
 
−
The invalid bit is 00 at byte 5 of the overlay.  You might try to recover the overlay by switching it to 01 with the following command line:
 
−
# echo $'\x01' | dd of=/path/to/overlay-file bs=1 count=1 seek=4 conv=notrunc
 
−
where /path/to/overlay may be /mnt/live/LiveOS/overlay-devicename-discUUID or<br>/media/devicename/LiveOS/overlay-devicename-discUUID for an attached device.
 
−
 
−
Follow this by registering the LiveOS image with Device-mapper:
 
−
# mount /media/devicename/LiveOS/squashfs.img /mnt/some_mountpoint
 
−
# losetup /dev/loop1 /mnt/some_mountpoint/LiveOS/ext3fs.img -r
 
−
# losetup /dev/loop2 /media/devicename/LiveOS/overlay-devicename-discUUID
 
−
# dmsetup create devicename --table "0 8388608 snapshot 7:1 7:2 P 8"
 
−
* devicename in the last line may be any string.
 
−
* loop1 and loop2 (and the corresponding 7:1 7:2) may be any free loop devices; just substitute the appropriate numbers.
 
−
* 8388608 is the size of the ext3fs.img file in 512 byte units.  This can be read by the following command:
 
−
# blockdev -q --getsz /dev/loop1
 
−
Execute
 
−
# dmsetup status
 
−
to check the the virtual filesystem has been configured.
 
−
 
−
Then try to repair any damage with the following command:
 
−
# e2fsck -f -y /dev/mapper/devicename
 
−
where devicename is the string you used in the dmsetup create command.
 
−
 
−
Run a second check,
 
−
# e2fsck -p /dev/mapper/devicename
 
−
to verify if the filesystem could be repaired.
 
−
At this point you will want to enlarge the overlay, or backup or rebuild the image.
 
−
 
−
Remove the virtual filesystem registration,
 
−
# dmsetup remove devicename
 
−
Determine the size of the overlay file:
 
−
# blockdev -q --getsz /dev/loop2
 
−
# losetup -d /dev/loop2
 
−
# losetup -d /dev/loop1
 
−
# dd if=/dev/zero of=/path/to/overlay-file seek=overlay_size count=size_increase
 
−
where overlay_size and size_increase are now 512 byte units.
 
−
 
−
==References==
 
−
<references />
 
−
:: See this [http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/14644 dm-devel thread].
 

Latest revision as of 08:34, 22 August 2012


This page has been moved to the Fedora Project Wiki.

Additional information for Sugar on a Stick is available below the frame.

The frame below is an active view of the LiveOS image page on the Fedora Project Wiki:

Sugar Activity Storage

Activity bundles that come pre-installed with the Sugar on a Stick (SoaS) are saved in the /usr/share/sugar/activities folder of the root filesystem. When Activities are updated from the Sugar Control Panel, the new Activity bundles are saved in the /home/liveuser/Activities folder. When an Activity is run, Activity instance data is saved in the Sugar Journal in /home/liveuser/.sugar/default/<activity bundle_id> folder and /home/liveuser/.sugar/default/datastore/<hashed folder names>

When additional Activity bundles are downloaded, they are saved in the /home/liveuser/Activities folder (but not the /usr/share/sugar/activities folder). Thus, when SoaS is installed with the --home-size-mb NNN option of livecd-iso-to-disk, storage for the Activities one wants to try and, perhaps later, throw away, is made available without consuming the write-once root filesystem overlay, which can be consumed very quickly (and overlay file space is not directly reusable).

Device filesystem

The filesystem of the USB/SD flash storage device before SoaS was loaded is available through the /run/initramfs/live mount point (/mnt/live before Fedora 17 or SoaS Quandong). It would be helpful if the Sugar Journal code were adjusted to recognize this partition structure and offer this space as an attached disk to facilitate file sharing.)

Installation options

Sugar on a Stick may be installed on a 1 GB USB device using the following options with livecd-iso-to-disk (on a single, Terminal Activity or console command line, even though the wiki may wrap the following text in accord with your browser window size):

./livecd-iso-to-disk --reset-mbr --overlay-size-mb 200 --home-size-mb 150 --delete-home --unencrypted-home /path/to/source/iso/or/device /dev/sd?1
where '?' in the final parameter represents the target bootable device node, such as sdb1 or sdc1, etc.

The above configuration would allow space for the home folder, the operating system, and a little on the device root.

But with a larger capacity device, one may allocate the resources to suit the anticipated use, as described above.

File Systems

Consumption of the space allocated for persistent storage in the snapshot overlay file may be tracked with the device mapper dmsetup status report. Sugar Cellar is a small, utility script which uses that service to allow for Learner testing and discovery. It will help Learners manage their storage resources and learn ways to economize limited resources.

Several developments in the Sugar on a Stick design may be considered to maximize the endurance of the LiveOS image.

Image customization, recompression, and overlay refreshment

Sugar on a Stick/Sugar Clone provides a method to build a customized LiveOS image installation file through the SoaS-remix bundle. The SoaS-remix.iso installation file can be used to redistribute the customized image. A new installation from this .iso will have the old image recompressed and a fresh overlay. With the current version, the home.img file must be manually copied to a new installation. One's aging SoaS image can be refreshed by an over-installation (see the instructions on the referenced page).

Development is proceeding on a new version of Sugar Clone, which uses the Device-mapper mirror facility to copy and merge the root filesystem and overlay, and then replaces the compressed SquashFS images on the USB or SD card with fresh, recompressed versions along with a refreshed overlay. This procedure does not require re-running the livecd-iso-to-disk script to effect the refreshment. An updated version of livecd-iso-to-disk will provide --copy-home, --reset-overlay, & --include <path s> options to support installation of the new, remixed builds.

The remix building of a LiveOS image is also a good way to maintain and distribute system updates. Many software security, enhancement, and bug fixes are distributed through PackageKit (Software Update) or Yum update, which are turned off by default on LiveOS installations like SoaS. After manually running an update, the image can be remixed to incorporate the changes into a new Updated-SoaS.iso file. This process could be migrated to the Sugar BuildBot service to provide the Community with an up-to-date Sugar on a Stick image.

Retrieved from "https://wiki.sugarlabs.org/index.php?title=LiveOS_image&oldid=82406"