8,288
edits
Changes
Jump to navigation
Jump to search
Line 5:
Line 5: − * '''Latest news''': An alpha version of a python script that allows one to rebuild a custom iso image from a running image by 1.) staging the build on an attached hard drive and then 2.) copy only the content already on the current image. See http://www.mail-archive.com/soas@lists.sugarlabs.org/msg02044.html '''28 January 2011'''+ + + + + + + + + + + + + + + + Line 89:
Line 104: − + + + + Line 96:
Line 114: − + Line 102:
Line 120: − + Line 122:
Line 140: − + − + + + Line 132:
Line 152: − + − + − + + + − + − + Line 147:
Line 169: − + − + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + − + − + + + + + + − + − + − + − #:* -n names the remix, -o designates the output file location, -t designates the temporary directory to use, #:* -i designates files from the outer USB device's filesystem to include in the build.+ − #:*: (Multiple files may be included with this syntax: {{Code|-i "/f1, /d/f2, /d[/d.../f3]"}})+ − #:* -r designates releasefile names in the image that may need adjustment, --builder names the builder, --clone flags the cloning feature to be invoked by the script on the {{Code|/dev/live}} partition (the currently running image).+ − # The script will launch and spew the files being copied to the screen and the progress of the rebuilding process.+ − # After about 20 minutes (on my system) the product iso file will be ready in the file, <code>/mnt/ext4/SoaSremix-x86_64-YYYYMMDD.HH.iso</code>+ − #* This iso holds, in the <code>/LiveOS/</code> directory, the {{Code|home.img}} from the booted Live USB Stick, and fresh {{Code|squashfs.img osmin.img}} files.+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + − + − + − +
Sugar on a Stick/Sugar Clone (view source)
Revision as of 18:16, 6 August 2013
, 18:16, 6 August 2013no edit summary
{{Message|'''Latest news''':
'''23 September 2012''': I've updated editliveos.py with bug fixes and simplifications. Run it in from a folder with these files:
https://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/trees/master/editliveos
There is also a new Bash script, LiveOS-merge, for merging a LiveOS snapshot and refreshing an installation with the new image. (A new
osmin.img is not created, but it is only used during installation of a traditional image from a LiveOS image.)
'''17 June 2012''': I've posted my working files for a new version of edit-livecd, which I named editliveos.py in this commit,
http://git.sugarlabs.org/soas/sugar-clone-extensions/commit/8c4e03323fc9a7e630e6c62213bb842909fbaa00
See my comments in this mailing list post,
http://www.mail-archive.com/livecd@lists.fedoraproject.org/msg01506.html
It includes several new options for adjusting the image filesystem size as well as home.img and overlay file sizes. Expand the [[#editliveos.help]] listing below.
It includes editliveos.py, liveimage-mount, and temporary versions of fs.py, live.py, creator.py, util.py (pending incremental patches for livecd-tools acceptance). The livecd-iso-to-disk version is not up-to-date with all the recent changes and improvements.}}
This method of backup has the advantage that it copies Learner changes to the core operating system as well as the Journal.
This method of backup has the advantage that it copies Learner changes to the core operating system as well as the Journal.
A LiveOS image using a separate persistent home folder could be partially cloned with either the operating system overlay or home folder without the other should that be desired.
A LiveOS image using a separate persistent home folder could be partially cloned with either the operating system overlay or home folder without the other, should that be desired.
====Ubiquitous Sugar====
An overarching vision is to be able to use and share Learners' Sugar environment wherever there are computing devices (see [[Sugar_on_a_Stick/Resources|Project Essence and Core Challenges]].) Sugar then becomes a platform for continuous learning. Working out the technical and social challenges of near-instantly resumable and portable learning with software is our challenge. The separation of the system and Learner's Journal and Activities opens more opportunities for simpler solutions.
====See also====
====See also====
===Notes===
===Notes===
These scripts have been tested with SoaS-Mirabelle (available at http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/) but should work with other versions of SoaS.
These scripts have been tested with SoaS-Mirabelle (available at http://download.sugarlabs.org/soas/releases/soas-3-mirabelle.iso) but should work with other versions of SoaS.
The current version of the script has the /bin/bash shell specified without the -x (xtrace) option. Set this option, #!/bin/bash -x in the first line of the script to aid testing. When set, the script will show a lot of output on the screen as it runs.
The current version of the script has the /bin/bash shell specified without the -x (xtrace) option. Set this option, #!/bin/bash -x in the first line of the script to aid testing. When set, the script will show a lot of output on the screen as it runs.
The script is copied to any new Sugar Clone devices prepared with this script so that Sugar Clone images can easily propagate themselves. This also applies to all new SoaS installations that are prepared with these scripts.
The script is copied to any new Sugar Clone devices prepared with this script so that Sugar Clone images can easily propagate themselves. This also applies to all new SoaS installations that are prepared with these scripts.
==Needed:==
==Needs and future:==
# More review and testing.
# More review and testing.
# A way to automatically identify live media images on secondary devices and partitions that could be used as sources for a Sugar Clone.
# A way to automatically identify live media images on secondary devices and partitions that could be used as sources for a Sugar Clone.
# <s>Bundling of the scripts for convenience until they are included in SoaS.</s> (available 23 April 2010)
# <s>Bundling of the scripts for convenience until they are included in SoaS.</s> (available 23 April 2010)
# Extend the capability to write iso image files to refresh the SoaS installation with an empty system overlay.
# Write custom iso image files to refresh the SoaS installation with an empty system overlay and permit redistribution of the customized system iso. (available 28 January 2011)
#* See this comment https://bugzilla.redhat.com/show_bug.cgi?id=448030#c27
#* See this comment https://bugzilla.redhat.com/show_bug.cgi?id=448030#c27
# Write custom iso images for inexpensive distribution of CD-ROMs of the custom curriculum.
#* Add the ability to refresh the installation without restarting or re-installing.
# Extend the facility to any Sugar image.
# Extend the facility to any Sugar image.
# Allow Learner account/stick swapping without rebooting.
# Allow multi-architecture distributions for larger capacity devices.
These all should help us better prepare a SoaS customization kit.
These all should help us better prepare a SoaS customization kit.
===Sugar Clone===
===Sugar Clone===
The script is available at http://people.sugarlabs.org/fgrose/SugarClone and is displayed from there below.<br>
The script is available at http://people.sugarlabs.org/fgrose/SugarClone and is displayed from there below.<br>
Embedded help, which describes the script and its usage in greater detail, is available in the script below (after the stock headers, click 3 to 5 times in the vertical scroll bar and look for 'shortusage()' and then the more complete 'usage()' functions).
Embedded help, which describes the script and its usage in greater detail, is available in the script below (There is a 'shortusage()' section followed by a more complete 'usage()' section).
{{Iframe|collapsed=|http://people.sugarlabs.org/fgrose/SugarClone|http://people.sugarlabs.org/fgrose/SugarClone|800|342|1}}
{{Iframe|http://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/sugar-clone/newSugarStick#line52|http://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/sugar-clone/newSugarStick#line52|800|600|1}}
===Sugar Cellar===
===Sugar Cellar===
{{Iframe|collapsed=|http://people.sugarlabs.org/fgrose/SugarCellar|http://people.sugarlabs.org/fgrose/SugarCellar|800|200|1}}
A small, utility script, available in the bundle, or separately. It is used to report on the storage space available on the persistent overlay, home directory, and other file spaces attached to the running image.
{{Iframe|http://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/sugar-clone/SugarCellar|http://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/sugar-clone/SugarCellar#line4|800|600|1}}
===Bundle===
===Bundle===
{{Iframe|collapsed=|http://people.sugarlabs.org/fgrose/bundle|http://people.sugarlabs.org/fgrose/bundle|800|200|1}}
{{Iframe|http://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/sugar-clone/bundle|http://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/sugar-clone/bundle#line4|800|600|1}}
===Launcher===
===Launcher===
{{Iframe|collapsed=|http://people.sugarlabs.org/fgrose/launcher|http://people.sugarlabs.org/fgrose/launcher|800|200|1}}
{{Iframe|http://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/sugar-clone/launcher|http://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/sugar-clone/launcher#line4|800|600|1}}
===edit-liveos.py===
===edit-liveos.py===
* The edit-liveos script can be thought of as a "liveos-disk-to-iso" script. It takes the currently installed system and converts it back into an iso9660 image file—but with all the current customizations to the system and Learner files.
* The edit-liveos script can be thought of as a "liveos-disk-to-iso" script. It takes the currently installed system and converts it back into an iso9660 image file—but with all the current customizations to the system and Learner files.
** ''livecd-iso-to-disk'' uses the --home-size-mb NNN options to specify the home.img installation.
** ''livecd-iso-to-disk'' uses the --home-size-mb NNN options to specify the home.img installation.
* With edit-liveos (disk-to-iso), a nearly-consumed overlay can be refreshed by rebuilding the system into a new, SquashFS image that re[[wikipedia:Sparse_file|sparse]]s the system files and repackages them into an iso installation file for reuse or distribution.
* With edit-liveos (disk-to-iso), a nearly-consumed overlay can be refreshed by rebuilding the system into a new, SquashFS image that re[[wikipedia:Sparse_file|sparse]]s the system files and repackages them into an iso installation file for reuse or distribution.
====Usage====
====editliveos.help====
This is my recipe:
Below is the output of editliveos --help in a working version from 25 January 2013.
{{Show|'''editliveos.py --help'''|<pre>Usage:
editliveos.py: [options] <LiveOS_source>
Edit a LiveOS image to insert files or to clone a customized
instance, rebuild the image into a new, .iso image installation
file, and refresh the source's persistent filesystem overlay.
options: [-n, --name <name>]
[-o, --output <output directory>]
[-k, --kickstart <kickstart-file>]
[-S, --force-SELinux]
[-s, --script <script.sh>]
[-N, --noshell]
[-t, --tmpdir <tmpdir>]
[-T, --leave-tmpfiles]
[-y, --yumcache <cachedir>]
[-e, --exclude <exclude, s>]
[-i, --include <include, s>]
[-u, --seclude <seclude, s>]
[-r, --releasefile <releasefile, s>]
[-b, --builder <builder>]
[--clone]
[--refresh-only]
[--skip-refresh]
[-c, --compress-type <compression type>]
[--compress]
[--skip-compression]
[--refresh-uncompressed]
[--skip-minimize]
[--skip-reboot]
[--expand-image-gb [+[<size>]]
[--shift-home-mb [+|-]<size>]
[--adjust-overlay-mb [+|-]<size>]
[-a, --extra-kernel-args <arg s>]
[--extra-space-mb <size>]
USAGE HELP editliveos.py --help
LiveOS_source must be entered as "live" to edit or clone the
currently running LiveOS image. An attached LiveOS installed
device may be edited or cloned through its node id, such as
/dev/sdd1, or, if it is mounted, through its mount point path.
An .iso image file is addressed through its pathname, such as
/path/to/build.iso, or, if mounted, through the mount point
directory path. A mounted Live CD-ROM image is addressed through
its device node, such as /dev/sr0. Even a directory containing a
LiveOS and iso/syslinux folders with the appropriate files can be
edited or cloned through that parent directory path.
The --clone option copies the home.img or persistent home folder
to the new build .iso file, allowing user customizations to be
replicated. A new version of livecd-iso-to-disk with the
--copy-home installation option may be used to propagate clones.
By default, other files and folders in the source device's outer
file system are included in the new build. Options are provided
to --include, --exclude, and --seclude files or folders for the
new build. The new builds are branded to distinguish them with
the --name, --builder, and --releasefile options.
Space requirements for stageing the build files are estimated by
the script and compared to the space available in a -t <TMPDIR>
option.
Invoke the --help option to learn about other optional features.
Options:
-h, --help show this help message and exit
-n NAME, --name=NAME Name for the new LiveOS image (don't include .iso, it
will be added.) Unless the name is prefixed with a
colon, :, the build will be branded with a date-
builder-Remix-releasename string, and the .iso will be
named as NAME-arch-Ymd.HM
-o OUTPUT, --output=OUTPUT
Specify directory location for the new .iso file.
-k KSCFG, --kickstart=KSCFG
Path or URL to kickstart config file.
-S, --force-SELinux Force setting SELinux attributes on install root.
-s SCRIPT, --script=SCRIPT
Specify a script to run chrooted in the LiveOS
filesystem hierarchy.
-N, --noshell Specify no breaking to shell after edit.
-t TMPDIR, --tmpdir=TMPDIR
Temporary directory to use for staging the build.
(default: /var/tmp)
-T, --leave-tmpfiles Skip deletion of temporary files and directories and
unmounting of the edited image.
-y CACHEDIR, --yumcache=CACHEDIR
Directory to use for for the yum cache. default: None
(A temporary filesystem cache will be used.)
-e EXCLUDES, --exclude=EXCLUDES
A string of filename patterns to exclude from the copy
of the outer device filesystem. See _copy_src_root().
Denote multiple patterns as "pattern1, pattern2, ...".
To exclude all device content except for the
iso/syslinux and LiveOS directories, enter "all".
-i INCLUDES, --include=INCLUDES
A string of file or directory paths to copy to the
.iso file in _copy_src_root(). Denote multiple files
as "path1, path2, ..." (The paths are referenced
relative to the source mount directory, either
/mnt/live/ or /run/initramfs/live for the running
LiveOS, or /TMPDIR/editliveos-<random>/<srcmnt>/ for
an attached or .iso file. So ../../../<mount
point>/INCLUDE or ../../../../<mount point>/INCLUDE
may be used, respectively, to include files from other
branches of the active hierarchy.)
-u SECLUDES, --seclude=SECLUDES
A string of file or directory paths in the LiveOS
filesystem to seclude from the final build
configuration. The user directory may be denoted with
~/. Denote multiple files as "path1, path2, ..."
-r RELEASEFILE, --releasefile=RELEASEFILE
Specify release file/s for branding. Denote multiple
files as "path1, path2, ..."
-b BUILDER, --builder=BUILDER
Specify the builder of a Remix.
--clone Specify copying of the home.img filesystem or the
/home folder contents to the new .iso file.
--expand-image-gb=EXPAND_IMAGE_SIZE
Specifies a new size of NN GiB for the image (or
changing the size by a difference of +NN GiB, if a +
sign is prefixed). This is useful when a larger image
is needed for a script, or during package updates in
the chroot shell.
--shift-home-mb=SHIFT_HOME_MB
Specify copying of the /home folder contents into a
home.img filesystem of size NN MiB, or changing the
size of an existing home.img filesystem to NN MiB (or
by a difference of +NN or -NN MiB, if a sign is
prefixed). If NN = 0 (or nets to <= 0), the home.img
filesystem contents will be shifted to the the /home
folder, which is normally compressed; however, it
subsequently will not have access to the --encrypted-
home installation option of livecd-iso-to-disk. If
the selected or calculated size is insufficient for
the current home contents, the edit will be to a size
10% larger than the current home size.
--adjust-overlay-mb=ADJUST_OVERLAY_MB
Specifies a new size of NN MiB for the overlay (or
changing the size by a difference of +NN or -NN MiB,
if a size is prefixed). If NN = 0 (or nets to <= 0),
no change will be made.
--refresh-only Specify replacing the squashfs.img or rootfs_img of
the source LiveOS installation instance with such
files from the new build, and resetting any overlay.
No new .iso installation file will be produced.
--skip-refresh Specify no refreshening of source filesystems.
-c COMPRESS_TYPE, --compress-type=COMPRESS_TYPE
Specify the compression type for SquashFS. Will
override the current compression or lack thereof.
--compress Specify compression for the filesystem imageUsed when
overriding an uncompressed source.
--skip-compression Specify building the .iso with an uncompressed root
file system.
--refresh-uncompressed
Specify refreshing the source with an uncompressed
root file system.
--skip-minimize Specify no osmin.img minimal snapshot.
-a KERNELARGS, --extra-kernel-args=KERNELARGS
Specify extra kernel arguments to include in the new
.iso file boot configuration. Multiple arguments
should be specified in one string, i.e., --extra-
kernel-args "arg1 arg2 ..."
--extra-space-mb=EXTRA_SPACE_MB
Specify extra space in MiB to reserve for unexpected
staging area needs.
Debugging options:
These options control the output of logging information during image
creation
-d, --debug Output debugging information
-v, --verbose Output verbose progress information
-q, --quiet Supress stdout
--logfile=FILE Save debug information to FILE</pre>}}
===SoaS-remix===
'''SoaS-remix''' is a bundle of edit-liveos.py and supporting scripts to make testing and use easier.
The bundle will change with development. As of 10 February 2011 it contains,
* edit-liveos.py - the primary remix builder
* liveos-disk-to-iso - a Bash launcher script for edit-liveos.py
* fs2.py, live2.py, creator2.py - temporary modules from livecd-tools with pending squashfs-compression_type() functionality
Get the '''SoaS-remix''' bundle:
:{{Code|wget http://people.sugarlabs.org/fgrose/SoaS-remix}}
====Recipe====
# Having used the ''livecd-iso-to-disk'' with --home-size-mb NNN to install my SoaS iso onto a Live USB device, I proceed to customize my working Stick with new Activites, content, or system settings as described [[#Introduction|above]] in step 2.
# Having used the ''livecd-iso-to-disk'' with --home-size-mb NNN to install my SoaS iso onto a Live USB device, I proceed to customize my working Stick with new Activites, content, or system settings as described [[#Introduction|above]] in step 2.
# I copy ''edit-liveos.py'' onto the USB device bearing SoaS (to the /LiveOS/ folder on the device's filesystem).
# Before copying ''SoaS-remix'' to the Live USB, it may be easiest to open it in an editor and adjust the {{Code|set}} command statement at line 86 to something appropriate for your system.<br>
# Before running the ''edit-liveos'' script, I run these 2 system updates:
#:{{Code|# Example command line. Edit and uncomment the set statement below to suit.}}
#:{{Code|# set -- -v -n SoaSremix -o /media/WD-ext4 -t /media/WD-ext4 -i /GPL \}}
#:{{Code|# -r /boot/olpc_build --builder fgrose --clone /dev/live}}
# I then copy ''SoaS-remix'' onto the USB device bearing a SoaS installation (to the /LiveOS/ folder on the device's filesystem).
# Before running the ''SoaS-remix'' script bundle, I run these 2 system updates:
#* <code>yum install rsync</code>
#* <code>yum install rsync</code>
#* <code>yum update livecd-tools</code>
#* <code>yum update livecd-tools</code>
#*: (Together, these operations consume about 8 MiB of overlay capacity, so be sure to have that available.<br>You can use [[#Sugar Cellar]] to check.)
# I boot my Sugar Stick on a system with a hard disc bearing an ext4-formatted partition with well over 5 GiB of free space.
# I boot my Sugar Stick on a system with a hard disc bearing an ext4-formatted partition with well over 5 GiB of free space.
# Once booted, I mount the partition from a Terminal activity with the root user account:
# Once booted, I mount the partition from a Terminal activity with the root user account:
#* <code>su -</code>
#* <code>su -</code>
#* <code>mount /dev/sdb2 /mnt/ext4</code>
#* <code>mount /dev/sdb2 /media/ext4</code>
#*:(The device node and mountpoint names will depend on your resource names. You might find the device node by issuing <code>df -Th</code> or <code>cat /proc/partitions</code> and knowing something about your disc resources.)
#*:(The device node and mountpoint names will depend on your resource names. You might find the device node by issuing <code>df -Th</code> or <code>cat /proc/partitions</code> and knowing something about your disc resources.)
# I invoked ''edit-liveos'' (saved as edit-liveos.py) with this command line:
# If you didn't edit the {{Code|set}} command before copying, you can use the vi editor or just invoke ''SoaS-remix'' with suitable parameters, such as,
#: <code>/mnt/live/LiveOS/edit-liveos.py -n SoaSremix -o /mnt/ext4 -t /mnt/ext4 -i /GPL -r /boot/olpc-build --builder fgrose --clone /dev/live</code>
#: {{Code|/mnt/live/LiveOS/SoaS-remix -v -n SoaSremix -o /media/ext4 -t /media/ext4 -i /GPL -r /boot/olpc-build --builder fgrose --clone /dev/live}}
# Otherwise, with a proper {{Code|set}} command statement, ''SoaS-remix'' may be invoked simply with this command line:
#: {{Code|/mnt/live/LiveOS/SoaS-remix}}
'''Common options''':
<nowiki> -v requests a slightly more verbose output,
-n names the remix,
-o designates the output file location,
-t designates the temporary directory to use,
-i designates files from the outer USB device's filesystem to include in the build.
Multiple files may be included with this syntax: -i "/f1, /d/f2, /d[/d.../f3]"
-r designates releasefile names in the image that may need adjustment,
--builder names the builder,
--clone flags the cloning feature to be invoked by the script on the
/dev/live device partition (the currently running image).
-h (alone) will show you some more options</nowiki>
The script will launch and provide limited process reporting. (I open another Terminal tab and list the contents of my temporary directory to see the changes occurring.)
After about 10 minutes (on my system) the product iso file will be ready in the file, {{Code|/media/ext4/SoaSremix-x86_64-YYYYMMDD.HHMM.iso}}
* This iso holds, in the {{Code|/LiveOS/}} directory, the {{Code|home.img}} from the booted Live USB Stick, and the fresh {{Code|squashfs.img osmin.img}} files.
The ''livecd-iso-to-disk'' script should be used to install the new build, but it currently doesn't propagate a home.img file from the iso, so you will have to manually move the home.img file to the /LiveOS/ folder of a new USB device.
: (With a GNU/Linux system, the iso image file can be auto-mounted in Nautilus and one can see the file to copy in the /LiveOS/ folder under the iso-image device.)
If one only wants to refresh the source Live USB stick, then the home folder does not need to be transferred. The following commands will refresh the overlay and system image:
* Boot into another image (such as from another SoaS device) with the target SoaS device inserted into a second USB connection port.
* From a root user Terminal activity or console session, issue this installation command:
*: {{Code|/path/to/livecd-iso-to-disk --overlay-size-mb NNN /path/to/refreshed/image.iso /dev/sd?1}}
* One needs to determine the proper paths on the working systems.
*: For example, on a booted Mango Lassi, the path is {{Code|/LiveOS/livecd-iso-to-disk}}.
If the source Live USB was installed without a separate {{Code|home.img}}, the above commands will also refresh your image, but it will consume the overlay more rapidly as all Activity storage is in the write-once overlay storage. Even deleting Activities will consume more of the storage! (See [[LiveOS image]].) Because the home folder (Learner's Journal and Activities) are mixed into the system SquashFS image file in this configuration, that portfolio as a whole will be less easily swapped or shared from one device or machine to another.
Testing of any of the above would be appreciated.
The script code is below:
The script code is below:
{{Iframe|collapsed=
{{Iframe
|http://people.sugarlabs.org/fgrose/edit-liveos
|https://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/editliveos/editliveos.py
|http://people.sugarlabs.org/fgrose/edit-liveos|800|200|1}}
|https://git.sugarlabs.org/~FGrose/soas/sugar-clone-extensions/blobs/master/editliveos/editliveos.py#line4|800|600|1}}