Difference between revisions of "Dextrose/Building"

From Sugar Labs
Jump to navigation Jump to search
m (edit for readability)
 
(42 intermediate revisions by 9 users not shown)
Line 1: Line 1:
== Build system ==
+
<noinclude>{{TeamHeader|Dextrose|home=Dextrose|xbgColor=ffe792|join_label=Get Involved}}</noinclude>
 +
 
 +
{{Obsolete|Moved and died at https://sugardextrose.org/projects/dextrose/wiki/Build_system_set-up}}
 +
 
 +
== Dextrose build system ==
  
 
Dextrose uses [http://wiki.laptop.org/go/OS_Builder olpc-os-builder], a tool create by OLPC to build official and customized system images. The Dextrose git repository contains, olpc-os-builder, local customizations specific to Dextrose, and fixes and enhancements waiting to be pushed upstream.
 
Dextrose uses [http://wiki.laptop.org/go/OS_Builder olpc-os-builder], a tool create by OLPC to build official and customized system images. The Dextrose git repository contains, olpc-os-builder, local customizations specific to Dextrose, and fixes and enhancements waiting to be pushed upstream.
Line 5: Line 9:
 
== Build host requirements ==
 
== Build host requirements ==
  
The Dextrose build system runs on '''Fedora 11 i386''' through '''Fedora 13 x86_64'''. The host system runs _shouldn't_ matter, because the build runs in a chroot environment. olpc-os-builder does not yet run on Ubuntu due to a missing dependency.
+
The Dextrose build system is known to work on '''Fedora 14 x86_64''' and '''Fedora 16 x86_64'''. The OS on the host system ''shouldn't'' matter because the build runs in a chroot environment. olpc-os-builder does not yet run on Ubuntu due to a missing dependency (bitfrost).
  
 
The initial build requires about '''700MB''' of rpm packages and Sugar activity bundles. The packages and bundles are cached for futures builds.
 
The initial build requires about '''700MB''' of rpm packages and Sugar activity bundles. The packages and bundles are cached for futures builds.
Line 11: Line 15:
 
== Initial setup ==
 
== Initial setup ==
  
* Checkout Dextrose build system
+
* If sudo isn't configured to work with your account, add yourself to the '''wheel''' group. Alternatively, you can become root with <code>su -</code>.
 
 
  git clone git://git.sugarlabs.org/dextrose/mainline.git dextrose
 
  
* Install dependencies
+
* Install build dependencies (you need to be root to do this)
  
 
   yum upgrade
 
   yum upgrade
   yum install libtomcrypt-devel bitfrost make gcc mtd-utils python-imgcreate zip unzip
+
   yum install libtomcrypt-devel bitfrost crcimg make gcc mtd-utils python-imgcreate zip unzip zlib-devel lzma netpbm-progs git wget
  
* Compile build helper programs
+
* Install the olpc-bootanim-tools package:
  
   make
+
   rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/x86_64/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.x86_64.rpm
  
* Generate initial sequence number for the builds
+
If you have a 32bit machine:
  
   echo 42 >buildnr-dx
+
   rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/i386/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.i686.rpm
  
== Creating a build ==
+
Alternatively, you could rebuild the olpc-bootanim package from source:
  
* Build (have a look at the available configuration files):
+
  yum install rpm-build netpbm-progs
 +
  wget http://download.sugarlabs.org/dextrose/testing/dx3/rpms/source/olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 +
  rpmbuild --rebuild olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 +
  rpm -U ~/rpmbuild/RPMS/*/olpc-bootanim-tools-*.rpm
  
  time sudo ./osbuilder.py examples/dextrose2-xo1.ini
+
* Checkout the Dextrose build system
  
* Wait 15-20 minutes
+
  git clone git://git.sugarlabs.org/dextrose/mainline.git dextrose
  
* If everything goes well, your new build can be found in <tt>build/output</tt>. Serve hot.
+
* Compile build helper programs
  
Of course, a number of things will go wrong the first time. Error output could be misleading and
+
  cd dextrose
obscure due to nesting of several tools: olpc-os-builder, imgcreate and yum. Before you dig into
+
  make
code, you may want to ask on IRC in case someone has already seen the same problem.
 
  
 +
* Generate initial sequence number for the builds (42 is just an example)
  
== Publishing the images ==
+
  echo 42 >buildnr-dx
 +
 
 +
== Creating a build ==
  
Signed builds should be published only if they correctly implement the [http://wiki.laptop.org/go/Antitheft_HowTo OLPC anti-theft system]). Signing does not have anything to do with quality or endorsement (i.e. signed builds are not necessarily bug-free or supported).
+
* Become root and run the build system:
  
 +
  time sudo ./osbuilder.py config/dextrose3-xo1-nognome.ini
  
== Customizing the build ==
+
The initial build may take several hours, depending on available bandwidth.
 +
Subsequent builds will take about 15-20 minutes on average desktop machines.
  
The topic of  is too vast to discuss in detail here. Start by reading
+
Upon completion, the new image will be available in <tt>build/output</tt>.
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/doc/README README]
+
Now copy it on a USB stick and test it on your target.
in the olpc-os-builder tree. Then, as needed, read the various <tt>README</tt> files contained in the
 
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/modules modules] directory.
 
  
Being a Build Master involves knowledge of many workflows, including:
+
=== Available variants ===
  
* Building Sugar in [http://wiki.sugarlabs.org/go/Development_Team/Jhbuild sugar-jhbuild].
+
There's a set of configuration files for each variant of the images you can build. You only need to specify the top-level file (i.e. none of the <code>*-common.ini</code> files) to build an image:
  
* Dealing with yum and [http://www.rpm.org/wiki/Docs rpm].
+
;dextrose3-xo1-gnome.ini:
 +
:Works on [[olpc:XO-1|XO-1]], includes Gnome. Not recommended unless you're running from an external SD card as the free space available on the internal NAND is too limited with Gnome installed.
 +
;dextrose3-xo1-nognome.ini:
 +
:Works on [[olpc:XO-1|XO-1]], does not include Gnome. This is the recommended image for XO-1s.
 +
;dextrose3-xo1.5-gnome.ini:
 +
:Works on [[olpc:XO-1.5|XO-1.5]] (including XO-1.5 [[olpc:OLPC_English_Non-membrane_Keyboard|HS]]), includes Gnome. Some teachers and older students seem to like Gnome, so you can trade off some of the free space that could be used by the Journal for the additional desktop environment.
 +
;dextrose3-xo1.5-nognome.ini:
 +
:Works on [[olpc:XO-1.5|XO-1.5]] (including XO-1.5 [[olpc:OLPC_English_Non-membrane_Keyboard|HS]]), doesn't include Gnome. More space for the Journal, so use this one if your users don't ask for Gnome and you haven't paid for a larger (i.e. > 4GB) SD card.
 +
;dextrose3-xo1.75-gnome.ini:
 +
:Experimental [[olpc:XO-1.75|XO-1.75]] image with Gnome.
 +
;dextrose3-xo1.75-nognome.ini:
 +
:Experimental [[olpc:XO-1.75|XO-1.75]] image without Gnome.
  
* Understanding the [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora packaging] workflow and conventions.
+
== Debugging ==
  
* Creating yum package repositories with <tt>[http://createrepo.baseurl.org/]</tt>.
+
Because several build tools (olpc-os-builder, imgcreate and yum) run chrooted
 +
and nested into each other, error output is often obscure or misleading.
 +
Before digging into the code, try asking on IRC in case someone has already seen
 +
the same problem.
  
* Uploading activities to [http://activities.sugarlabs.org].
+
== Publishing the images ==
  
* General understanding of the Linux system plumbing infrastructure: [http://www.kernel.org/ kernel],
+
Please do not publish signed builds unless they implement the [http://wiki.laptop.org/go/Antitheft_HowTo OLPC anti-theft system].
[http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html udev], [http://www.freedesktop.org/wiki/Software/dbus dbus], [http://www.freedesktop.org/wiki/Software/DeviceKit DeviceKit], [http://www.freedesktop.org/wiki/NetworkManager NetworkManager],
 
[http://www.x.org/wiki/ Xorg]...
 
  
* Flashing laptops and debugging any problems
+
== Customizing your build ==
  
* Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.
+
olpc-os-builder is documented at
 +
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/doc/README README]
 +
in the olpc-os-builder tree. The modules are documented in <tt>README</tt> files in the
 +
[http://git.sugarlabs.org/projects/dextrose/repos/mainline/trees/master/modules modules] directory.
  
== Upstream ==
+
=== Upstream Code ===
  
Our upstream code comes from dev.laptop.org:
+
Dextrose is based on [http://dev.laptop.org/git/projects/olpc-os-builder/ olpc-os-builder] by Daniel Drake. You can fetch the latest source code into your Dextrose repository:
  
 
  git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
 
  git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
Line 83: Line 105:
 
  git log olpc/master
 
  git log olpc/master
  
== Building custom RPMs ==
+
=== Customizing the core RPMs ===
  
 
* Go to the <tt>rpms/PACKAGENAME</tt> in the dextrose tree
 
* Go to the <tt>rpms/PACKAGENAME</tt> in the dextrose tree
Line 89: Line 111:
 
* Edit the spec file. Typically this involves adding a new patch, which is done by adding a <tt>PatchNN:</tt> tag and a corresponding <tt>%patchNN</tt> rule in <tt>%setup</tt>. Some of the intricacies of RPM spec files are unveiled in the [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora Packaging Guidelines].
 
* Edit the spec file. Typically this involves adding a new patch, which is done by adding a <tt>PatchNN:</tt> tag and a corresponding <tt>%patchNN</tt> rule in <tt>%setup</tt>. Some of the intricacies of RPM spec files are unveiled in the [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora Packaging Guidelines].
  
* Remember to bump the revision after each change or your package won't be seen as an update relative to what is in the cache!
+
* Increase the release number after each change.
 +
 
 +
=== building the Custom RPM ===
 +
 
 +
If you have a Fedora 11 i386 system, just type:
  
* <tt>make i586</tt>
+
  make i586
  
* Wait for rpmbuild to do its job
+
Otherwise, you need to build in a [http://fedoraproject.org/wiki/Projects/Mock mock] chroot:
  
* Now upload this rpm to a yum repository and add it to the ini file used by the OLPC OS builder.
+
  make srpm
 +
  sudo mock -r fedora-11-i386 --resultdir=. --rebuild mypackage.src.rpm
 +
 
 +
=== Uploading the Custom RPM ===
 +
 
 +
* Upload rpm to a yum repository and add it to the ini file used by the OLPC OS builder.
  
 
* To automate this step, we have a rule <tt>make uploadrpm</tt> which points at the Dextrose repository. Edit <tt>rpms/common/Makefile.common</tt> or <tt>rpms/common/uploadrpm</tt> to retarget this rule to your repository.
 
* To automate this step, we have a rule <tt>make uploadrpm</tt> which points at the Dextrose repository. Edit <tt>rpms/common/Makefile.common</tt> or <tt>rpms/common/uploadrpm</tt> to retarget this rule to your repository.
  
 
+
=== Building custom kernels ===
== Building custom OLPC kernels ==
 
  
 
  $ git clone http://dev.laptop.org/git/olpc-2.6/
 
  $ git clone http://dev.laptop.org/git/olpc-2.6/
 
  $ cd olpc-2.6
 
  $ cd olpc-2.6
 
  $ setarch i386 make ARCH=i386 xo_1-kernel-rpm
 
  $ setarch i386 make ARCH=i386 xo_1-kernel-rpm
 +
 +
== Resources ==
 +
 +
* [[Machine/dextrose|Dextrose VM]] hosted by [[Machine/treehouse|treehouse]]
 +
 +
* [[Development_Team/Jhbuild|Sugar-jhbuild]] - The Sugar build system
 +
 +
* [http://www.rpm.org/wiki/Docs RPM] - The Red Hat package manager
 +
 +
* [http://fedoraproject.org/wiki/Packaging/Guidelines Fedora packaging] - Fedora packaging workflow and conventions
 +
 +
* [http://createrepo.baseurl.org/ createrepo] - the yum package repositories builder
 +
 +
* [http://activities.sugarlabs.org ASLO] - The Sugar Labs Activities Library
 +
 +
* General understanding of the Linux system plumbing infrastructure: [http://www.kernel.org/ kernel],
 +
[http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html udev], [http://www.freedesktop.org/wiki/Software/dbus dbus], [http://www.freedesktop.org/wiki/Software/DeviceKit DeviceKit], [http://www.freedesktop.org/wiki/NetworkManager NetworkManager],
 +
[http://www.x.org/wiki/ Xorg]...
 +
 +
* Flashing laptops and debugging any problems
 +
 +
* Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.

Latest revision as of 00:35, 17 May 2016

Dextrose   ·   Get Involved   ·   Contacts   ·   Resources   ·   FAQ   ·   Roadmap   ·   To Do   ·   Meetings

Stop hand.png NOTE:
The content of this page is considered
DEPRECATED and OBSOLETE
It is preserved for historical research, along with its talk page.

Moved and died at https://sugardextrose.org/projects/dextrose/wiki/Build_system_set-up


Dextrose build system

Dextrose uses olpc-os-builder, a tool create by OLPC to build official and customized system images. The Dextrose git repository contains, olpc-os-builder, local customizations specific to Dextrose, and fixes and enhancements waiting to be pushed upstream.

Build host requirements

The Dextrose build system is known to work on Fedora 14 x86_64 and Fedora 16 x86_64. The OS on the host system shouldn't matter because the build runs in a chroot environment. olpc-os-builder does not yet run on Ubuntu due to a missing dependency (bitfrost).

The initial build requires about 700MB of rpm packages and Sugar activity bundles. The packages and bundles are cached for futures builds.

Initial setup

  • If sudo isn't configured to work with your account, add yourself to the wheel group. Alternatively, you can become root with su -.
  • Install build dependencies (you need to be root to do this)
 yum upgrade
 yum install libtomcrypt-devel bitfrost crcimg make gcc mtd-utils python-imgcreate zip unzip zlib-devel lzma netpbm-progs git wget
  • Install the olpc-bootanim-tools package:
 rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/x86_64/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.x86_64.rpm

If you have a 32bit machine:

 rpm -i http://download.sugarlabs.org/dextrose/testing/dx3/rpms/i386/os/olpc-bootanim-tools-2.14-1.bernie1.fc16.i686.rpm

Alternatively, you could rebuild the olpc-bootanim package from source:

 yum install rpm-build netpbm-progs
 wget http://download.sugarlabs.org/dextrose/testing/dx3/rpms/source/olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 rpmbuild --rebuild olpc-bootanim-2.14-1.bernie1.fc16.src.rpm
 rpm -U ~/rpmbuild/RPMS/*/olpc-bootanim-tools-*.rpm
  • Checkout the Dextrose build system
 git clone git://git.sugarlabs.org/dextrose/mainline.git dextrose
  • Compile build helper programs
 cd dextrose
 make
  • Generate initial sequence number for the builds (42 is just an example)
 echo 42 >buildnr-dx

Creating a build

  • Become root and run the build system:
 time sudo ./osbuilder.py config/dextrose3-xo1-nognome.ini

The initial build may take several hours, depending on available bandwidth. Subsequent builds will take about 15-20 minutes on average desktop machines.

Upon completion, the new image will be available in build/output. Now copy it on a USB stick and test it on your target.

Available variants

There's a set of configuration files for each variant of the images you can build. You only need to specify the top-level file (i.e. none of the *-common.ini files) to build an image:

dextrose3-xo1-gnome.ini
Works on XO-1, includes Gnome. Not recommended unless you're running from an external SD card as the free space available on the internal NAND is too limited with Gnome installed.
dextrose3-xo1-nognome.ini
Works on XO-1, does not include Gnome. This is the recommended image for XO-1s.
dextrose3-xo1.5-gnome.ini
Works on XO-1.5 (including XO-1.5 HS), includes Gnome. Some teachers and older students seem to like Gnome, so you can trade off some of the free space that could be used by the Journal for the additional desktop environment.
dextrose3-xo1.5-nognome.ini
Works on XO-1.5 (including XO-1.5 HS), doesn't include Gnome. More space for the Journal, so use this one if your users don't ask for Gnome and you haven't paid for a larger (i.e. > 4GB) SD card.
dextrose3-xo1.75-gnome.ini
Experimental XO-1.75 image with Gnome.
dextrose3-xo1.75-nognome.ini
Experimental XO-1.75 image without Gnome.

Debugging

Because several build tools (olpc-os-builder, imgcreate and yum) run chrooted and nested into each other, error output is often obscure or misleading. Before digging into the code, try asking on IRC in case someone has already seen the same problem.

Publishing the images

Please do not publish signed builds unless they implement the OLPC anti-theft system.

Customizing your build

olpc-os-builder is documented at README in the olpc-os-builder tree. The modules are documented in README files in the modules directory.

Upstream Code

Dextrose is based on olpc-os-builder by Daniel Drake. You can fetch the latest source code into your Dextrose repository:

git remote add olpc git://dev.laptop.org/projects/olpc-os-builder
git fetch olpc
git log olpc/master

Customizing the core RPMs

  • Go to the rpms/PACKAGENAME in the dextrose tree
  • Edit the spec file. Typically this involves adding a new patch, which is done by adding a PatchNN: tag and a corresponding %patchNN rule in %setup. Some of the intricacies of RPM spec files are unveiled in the Fedora Packaging Guidelines.
  • Increase the release number after each change.

building the Custom RPM

If you have a Fedora 11 i386 system, just type:

 make i586

Otherwise, you need to build in a mock chroot:

 make srpm
 sudo mock -r fedora-11-i386 --resultdir=. --rebuild mypackage.src.rpm

Uploading the Custom RPM

  • Upload rpm to a yum repository and add it to the ini file used by the OLPC OS builder.
  • To automate this step, we have a rule make uploadrpm which points at the Dextrose repository. Edit rpms/common/Makefile.common or rpms/common/uploadrpm to retarget this rule to your repository.

Building custom kernels

$ git clone http://dev.laptop.org/git/olpc-2.6/
$ cd olpc-2.6
$ setarch i386 make ARCH=i386 xo_1-kernel-rpm

Resources

  • RPM - The Red Hat package manager
  • ASLO - The Sugar Labs Activities Library
  • General understanding of the Linux system plumbing infrastructure: kernel,

udev, dbus, DeviceKit, NetworkManager, Xorg...

  • Flashing laptops and debugging any problems
  • Interaction with the Sugar and OLPC community to solve issues and minimize our divergence from the official builds.