Difference between revisions of "Dextrose/Building"

From Sugar Labs
Jump to navigation Jump to search
m (update link to *.ini)
 
(56 intermediate revisions by 9 users not shown)
Line 1: Line 1:
== Build system ==
+
<noinclude>{{TeamHeader|Dextrose|home=Dextrose|xbgColor=ffe792|join_label=Get Involved}}</noinclude>
  
We use [http://wiki.laptop.org/go/OS_Builder olpc-os-builder], a tool
+
{{Obsolete|Moved and died at https://sugardextrose.org/projects/dextrose/wiki/Build_system_set-up}}
used by OLPC to create official and customized system images. Our version contains local customizations specific to Dextrose and some patches that should be upstreamed.
+
 
 +
== Dextrose build system ==
 +
 
 +
Dextrose uses [http://wiki.laptop.org/go/OS_Builder olpc-os-builder], a tool create by OLPC to build official and customized system images. The Dextrose git repository contains, olpc-os-builder, local customizations specific to Dextrose, and fixes and enhancements waiting to be pushed upstream.
  
 
== Build host requirements ==
 
== Build host requirements ==
  
The olpc-os-builder machinery has been used successfully with systems running Fedora 11 i386 through Fedora 13 x86_64.
+
The Dextrose build system is known to work on '''Fedora 14 x86_64''' and '''Fedora 16 x86_64'''. The OS on the host system ''shouldn't'' matter because the build runs in a chroot environment. olpc-os-builder does not yet run on Ubuntu due to a missing dependency (bitfrost).
 +
 
 +
The initial build requires about '''700MB''' of rpm packages and Sugar activity bundles. The packages and bundles are cached for futures builds.
 +
 
 +
== Initial setup ==
 +
 
 +
* If sudo isn't configured to work with your account, add yourself to the '''wheel''' group. Alternatively, you can become root with <code>su -</code>.
 +
 
 +
* Install build dependencies (you need to be root to do this)
 +
 
 +
  yum upgrade
 +
  yum install libtomcrypt-devel bitfrost crcimg make gcc mtd-utils python-imgcreate zip unzip zlib-devel lzma netpbm-progs git wget
 +
 
 +
* Install the olpc-bootanim-tools package:
 +
 
 +
  rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/x86_64/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.x86_64.rpm
 +
 
 +
If you have a 32bit machine:
  
What the host system runs shouldn't matter much, because all the work is being done in a chroot environment, but olpc-os-builder is known to fail on Ubuntu due to a missing dependency.
+
  rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/i386/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.i686.rpm
  
 +
Alternatively, you could rebuild the olpc-bootanim package from source:
  
== How to create a build ==
+
  yum install rpm-build netpbm-progs
 +
  wget http://download.sugarlabs.org/dextrose/testing/dx3/rpms/source/olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 +
  rpmbuild --rebuild olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 +
  rpm -U ~/rpmbuild/RPMS/*/olpc-bootanim-tools-*.rpm
  
* Checkout our local tree:
+
* Checkout the Dextrose build system
  
   git clone git://git.sugarlabs.org/dextrose/mainline.git
+
   git clone git://git.sugarlabs.org/dextrose/mainline.git dextrose
  
* One time preparation
+
* Compile build helper programs
  
   yum upgrade
+
   cd dextrose
  yum install libtomcrypt-devel bitfrost make gcc mtd-utils
 
 
   make
 
   make
  
* Build:
+
* Generate initial sequence number for the builds (42 is just an example)
 +
 
 +
  echo 42 >buildnr-dx
 +
 
 +
== Creating a build ==
  
  time sudo ./osbuilder.py examples/f11-0.88-xo1-py.ini
+
* Become root and run the build system:
  
* Wait 15 minutes
+
  time sudo ./osbuilder.py config/dextrose3-xo1-nognome.ini
  
* Serve hot
+
The initial build may take several hours, depending on available bandwidth.
 +
Subsequent builds will take about 15-20 minutes on average desktop machines.
  
== Signing ==
+
Upon completion, the new image will be available in <tt>build/output</tt>.
 +
Now copy it on a USB stick and test it on your target.
  
* Put the 3 signing keys somewhere:
+
=== Available variants ===
  
bernie@robbie:~$ ll src/olpc/keys/
+
There's a set of configuration files for each variant of the images you can build. You only need to specify the top-level file (i.e. none of the <code>*-common.ini</code> files) to build an image:
-rw-------. 1 bernie bernie 1,2K Feb  5  2009 pyo1.private
 
-rw-------. 1 bernie bernie  270 Feb  5  2009 pyo1.public
 
-rw-------. 1 bernie bernie 1,2K Feb  5  2009 pys1.private
 
-rw-------. 1 bernie bernie  270 Feb  5  2009 pys1.public
 
-rw-------. 1 bernie bernie 1,2K Feb  5  2009 pyw1.private
 
-rw-------. 1 bernie bernie  270 Feb  5  2009 pyw1.public
 
  
* Make sure the keys are '''NOT''' world-readable
+
;dextrose3-xo1-gnome.ini:
 +
:Works on [[olpc:XO-1|XO-1]], includes Gnome. Not recommended unless you're running from an external SD card as the free space available on the internal NAND is too limited with Gnome installed.
 +
;dextrose3-xo1-nognome.ini:
 +
:Works on [[olpc:XO-1|XO-1]], does not include Gnome. This is the recommended image for XO-1s.
 +
;dextrose3-xo1.5-gnome.ini:
 +
:Works on [[olpc:XO-1.5|XO-1.5]] (including XO-1.5 [[olpc:OLPC_English_Non-membrane_Keyboard|HS]]), includes Gnome. Some teachers and older students seem to like Gnome, so you can trade off some of the free space that could be used by the Journal for the additional desktop environment.
 +
;dextrose3-xo1.5-nognome.ini:
 +
:Works on [[olpc:XO-1.5|XO-1.5]] (including XO-1.5 [[olpc:OLPC_English_Non-membrane_Keyboard|HS]]), doesn't include Gnome. More space for the Journal, so use this one if your users don't ask for Gnome and you haven't paid for a larger (i.e. > 4GB) SD card.
 +
;dextrose3-xo1.75-gnome.ini:
 +
:Experimental [[olpc:XO-1.75|XO-1.75]] image with Gnome.
 +
;dextrose3-xo1.75-nognome.ini:
 +
:Experimental [[olpc:XO-1.75|XO-1.75]] image without Gnome.
  
* Edit the paths in the <tt>[signing]</tt> section of your ini file (e.g. <tt>examples/f11-0.88-xo1-py.ini</tt>
+
== Debugging ==
  
 +
Because several build tools (olpc-os-builder, imgcreate and yum) run chrooted
 +
and nested into each other, error output is often obscure or misleading.
 +
Before digging into the code, try asking on IRC in case someone has already seen
 +
the same problem.
  
 
== Publishing the images ==
 
== Publishing the images ==
  
Signed builds should be published only if they correctly implement the [http://wiki.laptop.org/go/Antitheft_HowTo OLPC anti-theft system]). Signing does not have anything to do with quality or endorsement (i.e. signed builds are not necessarily bug-free or supported).
+
Please do not publish signed builds unless they implement the [http://wiki.laptop.org/go/Antitheft_HowTo OLPC anti-theft system].
 
 
  
== Customizing the build ==
+
== Customizing your build ==
  
The topic of  is too vast to discuss in detail here. Start by reading
+
olpc-os-builder is documented at
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/doc/README README]
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/doc/README README]
in the olpc-os-builder tree. Then, as needed, read the various <tt>README</tt> files contained in the
+
in the olpc-os-builder tree. The modules are documented in <tt>README</tt> files in the
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/modules modules] directory.
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/modules modules] directory.
  
Being a Build Master involves knowledge of many workflows, including:
+
=== Upstream Code ===
  
* Building Sugar in [http://wiki.sugarlabs.org/go/Development_Team/Jhbuild sugar-jhbuild].
+
Dextrose is based on [http://dev.laptop.org/git/projects/olpc-os-builder/ olpc-os-builder] by Daniel Drake. You can fetch the latest source code into your Dextrose repository:
  
* Dealing with yum and [http://www.rpm.org/wiki/Docs rpm].
+
git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
 +
git fetch olpc
 +
git log olpc/master
  
* Understanding the [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora packaging] workflow and conventions.
+
=== Customizing the core RPMs ===
  
* Creating yum package repositories with <tt>[http://createrepo.baseurl.org/]</tt>.
+
* Go to the <tt>rpms/PACKAGENAME</tt> in the dextrose tree
  
* Uploading activities to [http://activities.sugarlabs.org].
+
* Edit the spec file. Typically this involves adding a new patch, which is done by adding a <tt>PatchNN:</tt> tag and a corresponding <tt>%patchNN</tt> rule in <tt>%setup</tt>. Some of the intricacies of RPM spec files are unveiled in the [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora Packaging Guidelines].
  
* General understanding of the Linux system plumbing infrastructure: [http://www.kernel.org/ kernel],
+
* Increase the release number after each change.
[http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html udev], [http://www.freedesktop.org/wiki/Software/dbus dbus], [http://www.freedesktop.org/wiki/Software/DeviceKit DeviceKit], [http://www.freedesktop.org/wiki/NetworkManager NetworkManager],
+
 
[http://www.x.org/wiki/ Xorg]...
+
=== building the Custom RPM ===
 +
 
 +
If you have a Fedora 11 i386 system, just type:
 +
 
 +
  make i586
  
* Flashing laptops and debugging any problems
+
Otherwise, you need to build in a [http://fedoraproject.org/wiki/Projects/Mock mock] chroot:
  
* Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.
+
  make srpm
 +
  sudo mock -r fedora-11-i386 --resultdir=. --rebuild mypackage.src.rpm
  
== Upstream ==
+
=== Uploading the Custom RPM ===
  
Our upstream code comes from dev.laptop.org:
+
* Upload rpm to a yum repository and add it to the ini file used by the OLPC OS builder.
  
git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
+
* To automate this step, we have a rule <tt>make uploadrpm</tt> which points at the Dextrose repository. Edit <tt>rpms/common/Makefile.common</tt> or <tt>rpms/common/uploadrpm</tt> to retarget this rule to your repository.
git fetch olpc
 
git log olpc/master
 
  
== Building custom OLPC kernels ==
+
=== Building custom kernels ===
  
 
  $ git clone http://dev.laptop.org/git/olpc-2.6/
 
  $ git clone http://dev.laptop.org/git/olpc-2.6/
 
  $ cd olpc-2.6
 
  $ cd olpc-2.6
 
  $ setarch i386 make ARCH=i386 xo_1-kernel-rpm
 
  $ setarch i386 make ARCH=i386 xo_1-kernel-rpm
 +
 +
== Resources ==
 +
 +
* [[Machine/dextrose|Dextrose VM]] hosted by [[Machine/treehouse|treehouse]]
 +
 +
* [[Development_Team/Jhbuild|Sugar-jhbuild]] - The Sugar build system
 +
 +
* [http://www.rpm.org/wiki/Docs RPM] - The Red Hat package manager
 +
 +
* [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora packaging] - Fedora packaging workflow and conventions
 +
 +
* [http://createrepo.baseurl.org/ createrepo] - the yum package repositories builder
 +
 +
* [http://activities.sugarlabs.org ASLO] - The Sugar Labs Activities Library
 +
 +
* General understanding of the Linux system plumbing infrastructure: [http://www.kernel.org/ kernel],
 +
[http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html udev], [http://www.freedesktop.org/wiki/Software/dbus dbus], [http://www.freedesktop.org/wiki/Software/DeviceKit DeviceKit], [http://www.freedesktop.org/wiki/NetworkManager NetworkManager],
 +
[http://www.x.org/wiki/ Xorg]...
 +
 +
* Flashing laptops and debugging any problems
 +
 +
* Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.

Latest revision as of 00:35, 17 May 2016

Dextrose   ·   Get Involved   ·   Contacts   ·   Resources   ·   FAQ   ·   Roadmap   ·   To Do   ·   Meetings

Stop hand.png NOTE:
The content of this page is considered
DEPRECATED and OBSOLETE
It is preserved for historical research, along with its talk page.

Moved and died at https://sugardextrose.org/projects/dextrose/wiki/Build_system_set-up


Dextrose build system

Dextrose uses olpc-os-builder, a tool create by OLPC to build official and customized system images. The Dextrose git repository contains, olpc-os-builder, local customizations specific to Dextrose, and fixes and enhancements waiting to be pushed upstream.

Build host requirements

The Dextrose build system is known to work on Fedora 14 x86_64 and Fedora 16 x86_64. The OS on the host system shouldn't matter because the build runs in a chroot environment. olpc-os-builder does not yet run on Ubuntu due to a missing dependency (bitfrost).

The initial build requires about 700MB of rpm packages and Sugar activity bundles. The packages and bundles are cached for futures builds.

Initial setup

  • If sudo isn't configured to work with your account, add yourself to the wheel group. Alternatively, you can become root with su -.
  • Install build dependencies (you need to be root to do this)
 yum upgrade
 yum install libtomcrypt-devel bitfrost crcimg make gcc mtd-utils python-imgcreate zip unzip zlib-devel lzma netpbm-progs git wget
  • Install the olpc-bootanim-tools package:
 rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/x86_64/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.x86_64.rpm

If you have a 32bit machine:

 rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/i386/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.i686.rpm

Alternatively, you could rebuild the olpc-bootanim package from source:

 yum install rpm-build netpbm-progs
 wget http://download.sugarlabs.org/dextrose/testing/dx3/rpms/source/olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 rpmbuild --rebuild olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 rpm -U ~/rpmbuild/RPMS/*/olpc-bootanim-tools-*.rpm
  • Checkout the Dextrose build system
 git clone git://git.sugarlabs.org/dextrose/mainline.git dextrose
  • Compile build helper programs
 cd dextrose
 make
  • Generate initial sequence number for the builds (42 is just an example)
 echo 42 >buildnr-dx

Creating a build

  • Become root and run the build system:
 time sudo ./osbuilder.py config/dextrose3-xo1-nognome.ini

The initial build may take several hours, depending on available bandwidth. Subsequent builds will take about 15-20 minutes on average desktop machines.

Upon completion, the new image will be available in build/output. Now copy it on a USB stick and test it on your target.

Available variants

There's a set of configuration files for each variant of the images you can build. You only need to specify the top-level file (i.e. none of the *-common.ini files) to build an image:

dextrose3-xo1-gnome.ini
Works on XO-1, includes Gnome. Not recommended unless you're running from an external SD card as the free space available on the internal NAND is too limited with Gnome installed.
dextrose3-xo1-nognome.ini
Works on XO-1, does not include Gnome. This is the recommended image for XO-1s.
dextrose3-xo1.5-gnome.ini
Works on XO-1.5 (including XO-1.5 HS), includes Gnome. Some teachers and older students seem to like Gnome, so you can trade off some of the free space that could be used by the Journal for the additional desktop environment.
dextrose3-xo1.5-nognome.ini
Works on XO-1.5 (including XO-1.5 HS), doesn't include Gnome. More space for the Journal, so use this one if your users don't ask for Gnome and you haven't paid for a larger (i.e. > 4GB) SD card.
dextrose3-xo1.75-gnome.ini
Experimental XO-1.75 image with Gnome.
dextrose3-xo1.75-nognome.ini
Experimental XO-1.75 image without Gnome.

Debugging

Because several build tools (olpc-os-builder, imgcreate and yum) run chrooted and nested into each other, error output is often obscure or misleading. Before digging into the code, try asking on IRC in case someone has already seen the same problem.

Publishing the images

Please do not publish signed builds unless they implement the OLPC anti-theft system.

Customizing your build

olpc-os-builder is documented at README in the olpc-os-builder tree. The modules are documented in README files in the modules directory.

Upstream Code

Dextrose is based on olpc-os-builder by Daniel Drake. You can fetch the latest source code into your Dextrose repository:

git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
git fetch olpc
git log olpc/master

Customizing the core RPMs

  • Go to the rpms/PACKAGENAME in the dextrose tree
  • Edit the spec file. Typically this involves adding a new patch, which is done by adding a PatchNN: tag and a corresponding %patchNN rule in %setup. Some of the intricacies of RPM spec files are unveiled in the Fedora Packaging Guidelines.
  • Increase the release number after each change.

building the Custom RPM

If you have a Fedora 11 i386 system, just type:

 make i586

Otherwise, you need to build in a mock chroot:

 make srpm
 sudo mock -r fedora-11-i386 --resultdir=. --rebuild mypackage.src.rpm

Uploading the Custom RPM

  • Upload rpm to a yum repository and add it to the ini file used by the OLPC OS builder.
  • To automate this step, we have a rule make uploadrpm which points at the Dextrose repository. Edit rpms/common/Makefile.common or rpms/common/uploadrpm to retarget this rule to your repository.

Building custom kernels

$ git clone http://dev.laptop.org/git/olpc-2.6/
$ cd olpc-2.6
$ setarch i386 make ARCH=i386 xo_1-kernel-rpm

Resources

  • RPM - The Red Hat package manager
  • ASLO - The Sugar Labs Activities Library
  • General understanding of the Linux system plumbing infrastructure: kernel,

udev, dbus, DeviceKit, NetworkManager, Xorg...

  • Flashing laptops and debugging any problems
  • Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.