Platform Team/Guide/Sweets Packaging

Introduction

The purpose of this Guide is describing how to get benefits of Sugar Services for your library/application, simplify support fork flows and let your users(developers) use your code in simple and convenient way.

This guide is about how to package your code to services, if you are looking for method how to install some software from native packages, see Native packages usage guide.

If service is not wrapper for native package and could be used by other developers, please consider possibility to create service page using template.

Detailed description

The result for service development process will be regular 0install files that are placed to subdirectory on http://download.sugarlabs.org/services/ with name which is identity for the service in the rest of 0sugar infrastructure. Service's subdirectory will contain:

runtime.xml, regular 0install feed file which will be used as a runtime dependency
buildtime.xml, regular 0install feed file which will be used as a buildtime dependency
bunch of tarballs with sources and binaries, feed files contain links to these files

Feed files, in addition to several information fields, contain tags that describe implementations of the service(i.e. releases). At the same time, feed file contains implementations for various service versions and the same version could be represented by several implementations:

source implementation in <implementation> tag with attribute arch == *-src, with links to source tarball and instructions how to build it
no arch implementation in <implementation> tag with attribute arch == *-*, which can be used as-is on different platforms e.g. pure python services
one or several binary implementations in <implementation> tags, that contain binaries of the same source implementation but for various architectures(attribute arch)
package implementation in <package-implementation> tag, to reuse native packages, see Native packages usage for more info.

service.info file

Service publishing work flow is similar to activities. There is an info file, here service/service.info, which describes current status(not history) of development process. All these fields, finally, go to feed file while publishing new version.

Service section

Service.info file should contain at least [Service] section:

name, the identity of service, this field defines name of feed's root directory on the server http://download.sugarlabs.org/services/ as well
summary, short descriptive line
description, long descriptive line, to wrap long text, all lines after second, should start with spaces
could be omited to reuse summary field
license, in 0install, typically a Trove category, as used on freshmeat.net, but could be fedora names, since the rest of sugar uses them
homepage
version, service version number
plain number are welcome, but complex versions are also supported
for libraries, if age was used, it should be plain number
binding, what environment variables, 0sugar should export to session which uses this service

binding = [prepend|append|replace] <variable-name> [<insert-text-to-prepand-variable-value>] [; ...]

main, for applications, path to exec file from service root directory
build-command, for services that have compilation stage, command how to build binaries
Its value is a shell command, executed inside the build directory $BUILDDIR. It must compile the source in $SRCDIR, putting the final result (ready for distribution) in $DISTDIR. If this command starts to get complicated, you should move it to a script (either inside the main source archive or in a separate dependency) and just set this attribute to the command to run the script.
NOTE: This command will be executed not only in service developer environment but also on user side if proper binary wasn't found, so do not use here any development related dependencies like autogen.sh
For example, followed command builds regular autotools based project

"$SRCDIR"/configure --prefix="$DISTDIR" && make install

requires, for any arch and binary implementations, list of runtime services that should exist before using service

requires = <service-name> [<not-before-version>[-<before-version>]] [; ...]

build-requires, what services should be installed before building service from sources

build-requires = <service-name> [<not-before-version>[-<before-version>]] [; ...]

auto-documentation, if set, epydoc documentation project will be created on <http://api.sugarlabs.org/services/>

Buildtime section

Service.info file could have optional Buildtime section which describe additional parameters that should be taken into account during other service(which uses this one) building e.g. C headers files. Section can contain:

requires, additional dependencies that will be required for building of other service

Library section

Adds additional parameters if service is library. Libraries should have plain version number and with age field, they will be use in libtool like style e.g. in autotools based projects.

Section adds:

age libtool like age of version

Support several ABIs of service dependencies

Service.info file could contain other sections for various binary implementations. These sections are intended to describe particular dependencies environment and should contain only:

requires that overwrite Service's section requires

For example if service developer is going to add binary implementations for f9/f11, and the same service dependency in these distributions has different ABIs, activity developer has to build two additional binaries, for f9 and f11. In that case developer creates two sections F9 and F11, put exact dependency versions(to separate particular section from others) and invoke 0sugar dist_bin <section-name> in proper f9/f11 environment.

0sugar tool

To start developing services, install injector tool, and 0sugar - main tool to maintain services infrastructure.

0alias 0sugar http://download.sugarlabs.org/services/0sugar/runtime.xml

While working, 0sugar creates dist/ directory in project's root and uses it as a rsynced copy of service's directory on the server. So, you need ssh access to sunjammer server, create ticket for shell.sugarlabs.org category on http://bugs.sugarlabs.org/.

0sugar commands are:

init, create service/ directory and default service.info file
clone, rsync entirely dist/ from the server
dist, create source or any arch tarball/implementation in dist/ directory and tweak dist/feed.xml correspondingly
If there is second option-less argument, it will be treated as a path of tarball to release(e.g. you can create such tarball by make distcheck command for autotools based projects), otherwise 0sugar will tar all project files(and try to avoid temporary files as much as possible)
dist will create
- any arch archive, if build-command field was omitted in service.info
- source archive, if service.info has build-command field
dist_bin, create binary tarball/implementation for current arch in dist/ directory and tweak dist/feed.xml correspondingly
if second argument was passed, it will be treated as a name of service.info section with additional requires fields
push, rsync dist/ to the server
lint, check feed file on the server
will run FeedLint application for http://download.sugarlabs.org/services/ feed

Feed files in dist/ directory could be changed manually but keep in mind that dist* commands will rewrite entries for current service version.

Development workflows

How to build services.

Any arch services

Work flow for services that don't require compilation stage thus could be just packaged and used as is on server side:

create local initial service.info by invoking 0sugar init, tweak newly created service/service.info file
increase/change version field in service.info file
0sugar dist to change local feed in dist/ and place there tarball
0sugar push stable to rsync changed files from dist/ to the server
see #Release workflows for other release scenarios

Binary services

Work flow for services that require compilation stage:

create local initial service.info by invoking 0sugar init, tweak newly created service/service.info file
provide relevant value for build-command field in service.info file
if you want to support binaries for different dependencies environments, add additional sections to service.info file with requires fields that describe dependencies versions of particular environment
increase/change version field in service.info file
0sugar dist to add sources tarball to dist/ and change local feed correspondingly
on every platform, you are trying to support binaries for, run 0sugar dist_bin to build, and add to local field, binaries for this particular platform
if you added additional sections to service.info, run 0sugar dist_bin <section-name> in every environment, services.info has additional sections for
0sugar push stable to rsync changed files from dist/ to the server
see #Release workflows for other release scenarios

Versioning scheme

Except wrappers for native packages(in that case, service version is version of particular native package), services use followed scheme:

<compatibility-version>.<version>-<revision>

the major version, compatibility-version service.info field
exposes the fact of API/ABI/data-format backward compatibility breakage
only digits are allowed
the minor version, version service.info field
reflect on adding new features that don't break backward compatibility
only digits are allowed
the micro version is a revision number
auto incremented number

Micro version is beeing changed by 0sugar push command. It has through numbers(like Subversion revision).

Commands dist and dist-bin rewrite releases with the same compatibility-version.version versions and don't remove other releases.

NOTE: Be careful, some services and activities could be stuck to particular compatibility-version.version versions of your service, so do not delete stable releases manually from dist/*.xml feed files and mark them as buggy/insecure if they don't work in declared sugar environments.

Stability status

One of core differences from Sucrose development process is having not stable "releases" (here, release is a result of ``0sugar push`` command). So, services need stability status. Internally each service release will have one of 0install stability statuses:

stable, final stable release
testing, let interested in testing people, try last changes that are not yet stable
developer, just more extreme version of testing
buggy, already released versions(of any status) could be marked as buggy after a while to force people upgrade to new bugfix version
insecure, extreme version of buggy

0sugar push w/o arguments will push local changes as is. If you want to push with particular stability status, use 0sugar push <status>.

By default all users will use only stable versions and won't upgrade to new versions even if there are stable ones. To force people to upgrade from buggy versions, see bugfix releases workflow.

Release workflows

Useful scenarios for service release process.

Having only stable releases

All releases of your services will have stable stability status.

when your code is ready to release
0sugar dist/0sugar dist_bin to make local releases
0sugar push stable to rsync local releases to the server
if it was wrong time to release and you want to re-release it, repeat previous steps to rewrite wrong release

Bugfix releases

Services assume simple versioning scheme, if you found bugs in stable version:

either 0sugar push stable release once more(users will not mess this new copy of release with previous ones)
or 0sugar push stable release with incremented version field
if bugs are critical, mark previous release as buggy or insecure to force users upgrade their old releases. If you re-pushed release w/o incrementing version number, you should use revision number

0sugar push {buggy|insecure} {version[-revision]}

Development releases

If you want to test your new version among interested people:

do all steps that are mentioned in previous workflows
but use either testing or developer status for push command
users who interested in testing(enabled testing mode) will get your new development version
you don't have to change compatibility-version and version all time for micro releases, 0sugar will implicitly change revision of new pushes and user's saccharin will download last changes

Development tools support

Useful addons to popular development tools. These are optional features that simplify service development process.

Autotools

Add service.m4 macros file to your project. It provides:

SERVICE_INIT macro to initialize services support, use it after other (AC|AM)_INIT macros like in [1]
It takes optional string to define prefix value for DIST_ variables, in most cases it should be just sugar
Macro exports follows variables:
- SERVICE_NAME, name of service in service.info file
- SERVICE_VERSION, version of service in service.info file
- SERVICE_REVISION, revision number from dist/revision file
- DIST_NAME, combines prefix macro argument with SERVICE_NAME to use it as a distribution name(see example below)
- DIST_VERSION, combines SERVICE_VERSION and SERVICE_REVISION to use it as a distribution version(see example below)
- SERVICE_SUMMARY, summary from service.info file
- macro also changes PACKAGE and VERSION variables with values from proper DIST_, so it rewrites AC_INIT arguments(you have to use fake values for AC_INIT macro)
- SERVICE_AGE, age from service.info file (make sense only for libraries)
  If value is not empty, SERVICE_INIT adds:
  - checks if SERVICE_VERSION is a plain number
  - exports SERVICE_LIBRARY_VERSION which is libtool's -version-info parameter
  - if service.pc.in presents in root directory, rename it to DIST_NAME.pc.in

SERVICE_ECHO convenient macro to print various variables

These macros could be used like in Toolkit serivce.

Tips

Via binding's variables, service's root directory will be accessible for service users, so for python services(for example), better to use subdirectory for service's import modules like Toolkit does - there is toolkit subdirectory in Toolkit's root.

Be careful with machine architecture, especially in making binaries in VMs e.g. XO-1 arch is i586 but built in XO VM, binaries could have i686 thus won't start(0install won't let) on XO-1. Use --arch 0sugar's argument to set targeted architecture explicitly.