Difference between revisions of "Dextrose/Building"

m (edit for readability)
 
(37 intermediate revisions by 9 users not shown)
Line 1: Line 1:
== Build system ==
+
<noinclude>{{TeamHeader|Dextrose|home=Dextrose|xbgColor=ffe792|join_label=Get Involved}}</noinclude>
 +
 
 +
{{Obsolete|Moved and died at https://sugardextrose.org/projects/dextrose/wiki/Build_system_set-up}}
 +
 
 +
== Dextrose build system ==
  
 
Dextrose uses [http://wiki.laptop.org/go/OS_Builder olpc-os-builder], a tool create by OLPC to build official and customized system images. The Dextrose git repository contains, olpc-os-builder, local customizations specific to Dextrose, and fixes and enhancements waiting to be pushed upstream.
 
Dextrose uses [http://wiki.laptop.org/go/OS_Builder olpc-os-builder], a tool create by OLPC to build official and customized system images. The Dextrose git repository contains, olpc-os-builder, local customizations specific to Dextrose, and fixes and enhancements waiting to be pushed upstream.
Line 5: Line 9:
 
== Build host requirements ==
 
== Build host requirements ==
  
The Dextrose build system runs on '''Fedora 11 i386''' through '''Fedora 13 x86_64'''. The host system runs _shouldn't_ matter, because the build runs in a chroot environment. olpc-os-builder does not yet run on Ubuntu due to a missing dependency.
+
The Dextrose build system is known to work on '''Fedora 14 x86_64''' and '''Fedora 16 x86_64'''. The OS on the host system ''shouldn't'' matter because the build runs in a chroot environment. olpc-os-builder does not yet run on Ubuntu due to a missing dependency (bitfrost).
  
 
The initial build requires about '''700MB''' of rpm packages and Sugar activity bundles. The packages and bundles are cached for futures builds.
 
The initial build requires about '''700MB''' of rpm packages and Sugar activity bundles. The packages and bundles are cached for futures builds.
Line 11: Line 15:
 
== Initial setup ==
 
== Initial setup ==
  
* Checkout Dextrose build system
+
* If sudo isn't configured to work with your account, add yourself to the '''wheel''' group. Alternatively, you can become root with <code>su -</code>.
 +
 
 +
* Install build dependencies (you need to be root to do this)
  
   git clone git://git.sugarlabs.org/dextrose/mainline.git dextrose
+
   yum upgrade
 +
  yum install libtomcrypt-devel bitfrost crcimg make gcc mtd-utils python-imgcreate zip unzip zlib-devel lzma netpbm-progs git wget
 +
 
 +
* Install the olpc-bootanim-tools package:
 +
 
 +
  rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/x86_64/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.x86_64.rpm
 +
 
 +
If you have a 32bit machine:
 +
 
 +
  rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/i386/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.i686.rpm
 +
 
 +
Alternatively, you could rebuild the olpc-bootanim package from source:
 +
 
 +
  yum install rpm-build netpbm-progs
 +
  wget http://download.sugarlabs.org/dextrose/testing/dx3/rpms/source/olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 +
  rpmbuild --rebuild olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 +
  rpm -U ~/rpmbuild/RPMS/*/olpc-bootanim-tools-*.rpm
  
* Install dependencies
+
* Checkout the Dextrose build system
  
   yum upgrade
+
   git clone git://git.sugarlabs.org/dextrose/mainline.git dextrose
  yum install libtomcrypt-devel bitfrost make gcc mtd-utils python-imgcreate zip unzip
 
  
 
* Compile build helper programs
 
* Compile build helper programs
  
 +
  cd dextrose
 
   make
 
   make
  
* Generate initial sequence number for the builds
+
* Generate initial sequence number for the builds (42 is just an example)
  
 
   echo 42 >buildnr-dx
 
   echo 42 >buildnr-dx
Line 30: Line 52:
 
== Creating a build ==
 
== Creating a build ==
  
* Run the Build
+
* Become root and run the build system:
  
   time sudo ./osbuilder.py examples/dextrose2-xo1.ini
+
   time sudo ./osbuilder.py config/dextrose3-xo1-nognome.ini
  
The initial build may take several hours depending on available bandwith. Subsequent builds will take about 15-20 minutes
+
The initial build may take several hours, depending on available bandwidth.
 +
Subsequent builds will take about 15-20 minutes on average desktop machines.
  
Upon completion the new image is available at <tt>build/output</tt>.
+
Upon completion, the new image will be available in <tt>build/output</tt>.
 +
Now copy it on a USB stick and test it on your target.
  
Because several build tools, olpc-os-builder, imgcreate and yum, are nested, error output could be misleading or obscure. Before digging into code, you please ask on IRC in case someone has already seen the same problem.
+
=== Available variants ===
 +
 
 +
There's a set of configuration files for each variant of the images you can build. You only need to specify the top-level file (i.e. none of the <code>*-common.ini</code> files) to build an image:
 +
 
 +
;dextrose3-xo1-gnome.ini:
 +
:Works on [[olpc:XO-1|XO-1]], includes Gnome. Not recommended unless you're running from an external SD card as the free space available on the internal NAND is too limited with Gnome installed.
 +
;dextrose3-xo1-nognome.ini:
 +
:Works on [[olpc:XO-1|XO-1]], does not include Gnome. This is the recommended image for XO-1s.
 +
;dextrose3-xo1.5-gnome.ini:
 +
:Works on [[olpc:XO-1.5|XO-1.5]] (including XO-1.5 [[olpc:OLPC_English_Non-membrane_Keyboard|HS]]), includes Gnome. Some teachers and older students seem to like Gnome, so you can trade off some of the free space that could be used by the Journal for the additional desktop environment.
 +
;dextrose3-xo1.5-nognome.ini:
 +
:Works on [[olpc:XO-1.5|XO-1.5]] (including XO-1.5 [[olpc:OLPC_English_Non-membrane_Keyboard|HS]]), doesn't include Gnome. More space for the Journal, so use this one if your users don't ask for Gnome and you haven't paid for a larger (i.e. > 4GB) SD card.
 +
;dextrose3-xo1.75-gnome.ini:
 +
:Experimental [[olpc:XO-1.75|XO-1.75]] image with Gnome.
 +
;dextrose3-xo1.75-nognome.ini:
 +
:Experimental [[olpc:XO-1.75|XO-1.75]] image without Gnome.
 +
 
 +
== Debugging ==
 +
 
 +
Because several build tools (olpc-os-builder, imgcreate and yum) run chrooted
 +
and nested into each other, error output is often obscure or misleading.
 +
Before digging into the code, try asking on IRC in case someone has already seen
 +
the same problem.
  
 
== Publishing the images ==
 
== Publishing the images ==
Line 44: Line 90:
 
Please do not publish signed builds unless they implement the [http://wiki.laptop.org/go/Antitheft_HowTo OLPC anti-theft system].
 
Please do not publish signed builds unless they implement the [http://wiki.laptop.org/go/Antitheft_HowTo OLPC anti-theft system].
  
== Customizing Your build ==
+
== Customizing your build ==
  
The topic of  is too vast to discuss in detail here. Start by reading
+
olpc-os-builder is documented at
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/doc/README README]
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/doc/README README]
in the olpc-os-builder tree. Then, as needed, read the various <tt>README</tt> files contained in the
+
in the olpc-os-builder tree. The modules are documented in <tt>README</tt> files in the
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/modules modules] directory.
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/modules modules] directory.
  
Being a Build Master involves knowledge of many workflows, including:
+
=== Upstream Code ===
 +
 
 +
Dextrose is based on [http://dev.laptop.org/git/projects/olpc-os-builder/ olpc-os-builder] by Daniel Drake. You can fetch the latest source code into your Dextrose repository:
 +
 
 +
git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
 +
git fetch olpc
 +
git log olpc/master
 +
 
 +
=== Customizing the core RPMs ===
  
* Building Sugar in [http://wiki.sugarlabs.org/go/Development_Team/Jhbuild sugar-jhbuild].
+
* Go to the <tt>rpms/PACKAGENAME</tt> in the dextrose tree
  
* Dealing with yum and [http://www.rpm.org/wiki/Docs rpm].
+
* Edit the spec file. Typically this involves adding a new patch, which is done by adding a <tt>PatchNN:</tt> tag and a corresponding <tt>%patchNN</tt> rule in <tt>%setup</tt>. Some of the intricacies of RPM spec files are unveiled in the [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora Packaging Guidelines].
  
* Understanding the [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora packaging] workflow and conventions.
+
* Increase the release number after each change.
  
* Creating yum package repositories with <tt>[http://createrepo.baseurl.org/]</tt>.
+
=== building the Custom RPM ===
  
* Uploading activities to [http://activities.sugarlabs.org].
+
If you have a Fedora 11 i386 system, just type:
  
* General understanding of the Linux system plumbing infrastructure: [http://www.kernel.org/ kernel],
+
  make i586
[http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html udev], [http://www.freedesktop.org/wiki/Software/dbus dbus], [http://www.freedesktop.org/wiki/Software/DeviceKit DeviceKit], [http://www.freedesktop.org/wiki/NetworkManager NetworkManager],
+
 
[http://www.x.org/wiki/ Xorg]...
+
Otherwise, you need to build in a [http://fedoraproject.org/wiki/Projects/Mock mock] chroot:
  
* Flashing laptops and debugging any problems
+
  make srpm
 +
  sudo mock -r fedora-11-i386 --resultdir=. --rebuild mypackage.src.rpm
  
* Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.
+
=== Uploading the Custom RPM ===
  
== Upstream ==
+
* Upload rpm to a yum repository and add it to the ini file used by the OLPC OS builder.
  
Upstream code comes from dev.laptop.org:
+
* To automate this step, we have a rule <tt>make uploadrpm</tt> which points at the Dextrose repository. Edit <tt>rpms/common/Makefile.common</tt> or <tt>rpms/common/uploadrpm</tt> to retarget this rule to your repository.
  
git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
+
=== Building custom kernels ===
git fetch olpc
 
git log olpc/master
 
  
== Building custom RPMs ==
+
$ git clone http://dev.laptop.org/git/olpc-2.6/
 +
$ cd olpc-2.6
 +
$ setarch i386 make ARCH=i386 xo_1-kernel-rpm
  
* Go to the <tt>rpms/PACKAGENAME</tt> in the dextrose tree
+
== Resources ==
  
* Edit the spec file. Typically this involves adding a new patch, which is done by adding a <tt>PatchNN:</tt> tag and a corresponding <tt>%patchNN</tt> rule in <tt>%setup</tt>. Some of the intricacies of RPM spec files are unveiled in the [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora Packaging Guidelines].
+
* [[Machine/dextrose|Dextrose VM]] hosted by [[Machine/treehouse|treehouse]]
  
* Remember to bump the revision after each change or your package won't be seen as an update relative to what is in the cache!
+
* [[Development_Team/Jhbuild|Sugar-jhbuild]] - The Sugar build system
  
* <tt>make i586</tt>
+
* [http://www.rpm.org/wiki/Docs RPM] - The Red Hat package manager
  
* Wait for rpmbuild to do its job
+
* [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora packaging] - Fedora packaging workflow and conventions
  
* Now upload this rpm to a yum repository and add it to the ini file used by the OLPC OS builder.
+
* [http://createrepo.baseurl.org/ createrepo] - the yum package repositories builder
  
* To automate this step, we have a rule <tt>make uploadrpm</tt> which points at the Dextrose repository. Edit <tt>rpms/common/Makefile.common</tt> or <tt>rpms/common/uploadrpm</tt> to retarget this rule to your repository.
+
* [http://activities.sugarlabs.org ASLO] - The Sugar Labs Activities Library
  
 +
* General understanding of the Linux system plumbing infrastructure: [http://www.kernel.org/ kernel],
 +
[http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html udev], [http://www.freedesktop.org/wiki/Software/dbus dbus], [http://www.freedesktop.org/wiki/Software/DeviceKit DeviceKit], [http://www.freedesktop.org/wiki/NetworkManager NetworkManager],
 +
[http://www.x.org/wiki/ Xorg]...
  
== Building custom OLPC kernels ==
+
* Flashing laptops and debugging any problems
  
$ git clone http://dev.laptop.org/git/olpc-2.6/
+
* Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.
$ cd olpc-2.6
 
$ setarch i386 make ARCH=i386 xo_1-kernel-rpm
 

Latest revision as of 00:35, 17 May 2016

Dextrose   ·   Get Involved   ·   Contacts   ·   Resources   ·   FAQ   ·   Roadmap   ·   To Do   ·   Meetings

Stop hand.png NOTE:
The content of this page is considered
DEPRECATED and OBSOLETE
It is preserved for historical research, along with its talk page.

Moved and died at https://sugardextrose.org/projects/dextrose/wiki/Build_system_set-up


Dextrose build system

Dextrose uses olpc-os-builder, a tool create by OLPC to build official and customized system images. The Dextrose git repository contains, olpc-os-builder, local customizations specific to Dextrose, and fixes and enhancements waiting to be pushed upstream.

Build host requirements

The Dextrose build system is known to work on Fedora 14 x86_64 and Fedora 16 x86_64. The OS on the host system shouldn't matter because the build runs in a chroot environment. olpc-os-builder does not yet run on Ubuntu due to a missing dependency (bitfrost).

The initial build requires about 700MB of rpm packages and Sugar activity bundles. The packages and bundles are cached for futures builds.

Initial setup

  • If sudo isn't configured to work with your account, add yourself to the wheel group. Alternatively, you can become root with su -.
  • Install build dependencies (you need to be root to do this)
 yum upgrade
 yum install libtomcrypt-devel bitfrost crcimg make gcc mtd-utils python-imgcreate zip unzip zlib-devel lzma netpbm-progs git wget
  • Install the olpc-bootanim-tools package:
 rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/x86_64/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.x86_64.rpm

If you have a 32bit machine:

 rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/i386/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.i686.rpm

Alternatively, you could rebuild the olpc-bootanim package from source:

 yum install rpm-build netpbm-progs
 wget http://download.sugarlabs.org/dextrose/testing/dx3/rpms/source/olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 rpmbuild --rebuild olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 rpm -U ~/rpmbuild/RPMS/*/olpc-bootanim-tools-*.rpm
  • Checkout the Dextrose build system
 git clone git://git.sugarlabs.org/dextrose/mainline.git dextrose
  • Compile build helper programs
 cd dextrose
 make
  • Generate initial sequence number for the builds (42 is just an example)
 echo 42 >buildnr-dx

Creating a build

  • Become root and run the build system:
 time sudo ./osbuilder.py config/dextrose3-xo1-nognome.ini

The initial build may take several hours, depending on available bandwidth. Subsequent builds will take about 15-20 minutes on average desktop machines.

Upon completion, the new image will be available in build/output. Now copy it on a USB stick and test it on your target.

Available variants

There's a set of configuration files for each variant of the images you can build. You only need to specify the top-level file (i.e. none of the *-common.ini files) to build an image:

dextrose3-xo1-gnome.ini
Works on XO-1, includes Gnome. Not recommended unless you're running from an external SD card as the free space available on the internal NAND is too limited with Gnome installed.
dextrose3-xo1-nognome.ini
Works on XO-1, does not include Gnome. This is the recommended image for XO-1s.
dextrose3-xo1.5-gnome.ini
Works on XO-1.5 (including XO-1.5 HS), includes Gnome. Some teachers and older students seem to like Gnome, so you can trade off some of the free space that could be used by the Journal for the additional desktop environment.
dextrose3-xo1.5-nognome.ini
Works on XO-1.5 (including XO-1.5 HS), doesn't include Gnome. More space for the Journal, so use this one if your users don't ask for Gnome and you haven't paid for a larger (i.e. > 4GB) SD card.
dextrose3-xo1.75-gnome.ini
Experimental XO-1.75 image with Gnome.
dextrose3-xo1.75-nognome.ini
Experimental XO-1.75 image without Gnome.

Debugging

Because several build tools (olpc-os-builder, imgcreate and yum) run chrooted and nested into each other, error output is often obscure or misleading. Before digging into the code, try asking on IRC in case someone has already seen the same problem.

Publishing the images

Please do not publish signed builds unless they implement the OLPC anti-theft system.

Customizing your build

olpc-os-builder is documented at README in the olpc-os-builder tree. The modules are documented in README files in the modules directory.

Upstream Code

Dextrose is based on olpc-os-builder by Daniel Drake. You can fetch the latest source code into your Dextrose repository:

git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
git fetch olpc
git log olpc/master

Customizing the core RPMs

  • Go to the rpms/PACKAGENAME in the dextrose tree
  • Edit the spec file. Typically this involves adding a new patch, which is done by adding a PatchNN: tag and a corresponding %patchNN rule in %setup. Some of the intricacies of RPM spec files are unveiled in the Fedora Packaging Guidelines.
  • Increase the release number after each change.

building the Custom RPM

If you have a Fedora 11 i386 system, just type:

 make i586

Otherwise, you need to build in a mock chroot:

 make srpm
 sudo mock -r fedora-11-i386 --resultdir=. --rebuild mypackage.src.rpm

Uploading the Custom RPM

  • Upload rpm to a yum repository and add it to the ini file used by the OLPC OS builder.
  • To automate this step, we have a rule make uploadrpm which points at the Dextrose repository. Edit rpms/common/Makefile.common or rpms/common/uploadrpm to retarget this rule to your repository.

Building custom kernels

$ git clone http://dev.laptop.org/git/olpc-2.6/
$ cd olpc-2.6
$ setarch i386 make ARCH=i386 xo_1-kernel-rpm

Resources

  • RPM - The Red Hat package manager
  • ASLO - The Sugar Labs Activities Library
  • General understanding of the Linux system plumbing infrastructure: kernel,

udev, dbus, DeviceKit, NetworkManager, Xorg...

  • Flashing laptops and debugging any problems
  • Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.