Platform Team/Guide/Sweets Packaging

From Sugar Labs
Jump to navigation Jump to search


Introduction

The purpose of this Guide is describing how to get benefits of Sugar Services for your library/application, simplify support fork flows and let your users(developers) use your code in simple and convenient way.

This guide is about how to package your code to services, if you are looking for method how to install some software from native packages, see Native packages usage guide.

If service is not wrapper for native package and could be used by other developers, please consider possibility to create service page using template.

Detailed description

The result for service development process will be regular 0install files that are placed to subdirectory on http://download.sugarlabs.org/services/ with name which is identity for the service in the rest of 0sugar infrastructure. Service's subdirectory will contain:

  • runtime.xml, regular 0install feed file which will be used as a runtime dependency
  • buildtime.xml, regular 0install feed file which will be used as a buildtime dependency
  • bunch of tarballs with sources and binaries, feed files contain links to these files

Feed files, in addition to several information fields, contain tags that describe implementations of the service(i.e. releases). At the same time, feed file contains implementations for various service versions and the same version could be represented by several implementations:

  • source implementation in <implementation> tag with attribute arch == *-src, with links to source tarball and instructions how to build it
  • no arch implementation in <implementation> tag with attribute arch == *-*, which can be used as-is on different platforms e.g. pure python services
  • one or several binary implementations in <implementation> tags, that contain binaries of the same source implementation but for various architectures(attribute arch)
  • package implementation in <package-implementation> tag, to reuse native packages, see Native packages usage for more info.

service.info file

Service publishing workflow is similar to activities. There is an info file, here service/service.info, which describes current status(not history) of development process. All these fields, finally, go to feed file while publishing new version.

[Service] section

File service.info' should contain at least [Service] section:

  • name, the identity of service, this field defines name of feed's root directory on the server http://download.sugarlabs.org/services/ as well
  • summary, short descriptive line
  • description, long descriptive line, to wrap long text, all lines after second, should start with spaces
    could be omited to reuse summary field
  • license, in 0install, typically a Trove category, as used on freshmeat.net, but could be fedora names, since the rest of sugar uses them
  • homepage web link to project home page

[Runtime] section

This section describes parameters that should be taken into account only if service has 0install implementations(including source implementations) in contrast with pure wrappers around native packages.

Section contains:

  • version, used only for services with native versioning scheme
  • binding, what environment variables, 0install should export to session which uses this service
binding = [prepend|append|replace] <variable-name> [<insert-text-to-prepand-variable-value>] [; ...]
  • main, for applications, path to exec file from service root directory
  • requires, for any arch and binary implementations, list of runtime services that should exist before using service
requires = <service-name> [<not-before-version>[-<before-version>]] [; ...]
  • exec, command how to make bundle for dist command
    Shell command, executed from service root directory. If this command starts to get complicated, you should move it to a script and just set this attribute to the command to run the script.
    By default 0sugar just bundle entirely service directory excluding temporary files.
exec = make distcheck

[Library] section

Adds additional parameters if service is a library. Libraries should have plain version value and with age field, it will be used in libtool like style e.g. in autotools based projects.

Section contains:

  • age libtool like age of version

[Buildtime] section

File service.info could have optional [Buildtime] section which describes additional parameters that should be taken into account during other service(which uses this one) building i.e. it is an equivalent for *-dev(el) packages in GNU/Linux distributions.

Section contains:

  • requires, additional dependencies that will be required during building other service which depends on this one
  • exec, command how to make buildtime bundle for dist command
    Shell command, executed from service root directory. If this command starts to get complicated, you should move it to a script and just set this attribute to the command to run the script.
    By default 0sugar dist uses the same bundle as for runtime feed
  • generate-documentation, if set, documentation project will be created on [1], for now only epydoc value is supported

[Build] section

If service contains binary implementations, this section should present to describe building process.

Section contains:

  • requires, what services should be installed before building this one from sources
requires = <service-name> [<not-before-version>[-<before-version>]] [; ...]
  • exec, command how to build binaries
    Its value is a shell command, executed inside the build directory $BUILDDIR. It must compile the source in $SRCDIR, putting the final result (ready for distribution) in $DISTDIR. If this command starts to get complicated, you should move it to a script (either inside the main source archive or in a separate dependency) and just set this attribute to the command to run the script.
    NOTE: This command will be executed not only in service developer environment but also on user side if proper binary wasn't found, so do not use here any development related commands like autogen.sh
    For example, followed command builds regular autotools based project
"$SRCDIR"/configure --prefix="$DISTDIR" && make install

Support several ABIs of service dependencies

File service.info could contain other sections for various binary implementations. These sections are intended to describe particular dependencies environment and should contain only:

  • requires that overwrite Service's section requires

For example if service developer is going to add binary implementations for f9/f11, and the same service dependency in these distributions has different ABIs, activity developer has to build two additional binaries, for f9 and f11. In that case developer creates two sections F9 and F11, put exact dependency versions(to separate particular section from others) and invoke 0sugar dist_bin <section-name> in proper f9/f11 environment.

0sugar tool

To start developing services, install injector tool, and 0sugar - main tool to maintain services infrastructure. 

0alias 0sugar http://download.sugarlabs.org/services/0sugar/runtime.xml

While working, 0sugar creates dist/ directory in project's root and uses it as a rsynced copy of service's directory on the server. So, you need ssh access to sunjammer server, create ticket for shell.sugarlabs.org category on http://bugs.sugarlabs.org/.

0sugar commands are:

  • init, create service/ directory and default service.info file
  • clone, rsync entirely dist/ from the server
  • dist, create source or any arch tarball/implementation in dist/ directory and tweak dist/feed.xml correspondingly
    If there is second option-less argument, it will be treated as a path of tarball to release(e.g. you can create such tarball by make distcheck command for autotools based projects), otherwise 0sugar will tar all project files(and try to avoid temporary files as much as possible)
    dist will create
    • any arch archive, if build-command field was omitted in service.info
    • source archive, if service.info has build-command field
  • dist_bin, create binary tarball/implementation for current arch in dist/ directory and tweak dist/feed.xml correspondingly
    if second argument was passed, it will be treated as a name of service.info section with additional requires fields
  • push, rsync dist/ to the server
  • lint, check feed file on the server
    will run FeedLint application for http://download.sugarlabs.org/services/ feed

Feed files in dist/ directory could be changed manually but keep in mind that dist* commands will rewrite entries for current service version.

Development workflows

How to build services.

Any arch services

Work flow for services that don't require compilation stage thus could be just packaged and used as is on server side:

  • create local initial service.info by invoking 0sugar init, tweak newly created service/service.info file
  • increase/change version field in service.info file
  • 0sugar dist to change local feed in dist/ and place there tarball
  • 0sugar push stable to rsync changed files from dist/ to the server
    see #Release workflows for other release scenarios

Binary services

Work flow for services that require compilation stage:

  • create local initial service.info by invoking 0sugar init, tweak newly created service/service.info file
  • provide relevant value for build-command field in service.info file
  • if you want to support binaries for different dependencies environments, add additional sections to service.info file with requires fields that describe dependencies versions of particular environment
  • increase/change version field in service.info file
  • 0sugar dist to add sources tarball to dist/ and change local feed correspondingly
  • on every platform, you are trying to support binaries for, run 0sugar dist_bin to build, and add to local field, binaries for this particular platform
  • if you added additional sections to service.info, run 0sugar dist_bin <section-name> in every environment, services.info has additional sections for
  • 0sugar push stable to rsync changed files from dist/ to the server
    see #Release workflows for other release scenarios

Versioning scheme

There could be two different versioning scheme for services.

Upstream versions

If service is just a wrapper around existed project with its own versioning scheme then there is even no need in any mentioning of versions in service.info file. Particular version will be chosen on every 0sugar dist* command invoking or will be fetched from native packaging systems(if service doesn't provide binaries and only contains information about already well packaged in various GNU/Linux distributions).

Native versions

The rest of services will use scheme: 

<version>-<revision>
  • <version> is a version field from service.info file which is a plain number
  • <revision> is an auto incremented number which is beeing changed by 0sugar push command (like Subversion revision)

Commands dist and dist_bin rewrite releases with the same version versions and don't remove other releases.

Libraries have also age parameter which is libtool like age version part. age won't be exposed in version string but will be present in feed files and 0sugar will check it while deciding what dependency version should be chosen.

Stability status

One of core differences from Sucrose development process is having not stable "releases" (here, release is a result of ``0sugar push`` command). So, services need stability status. Internally each service release will have one of 0install stability statuses:

  • stable, final stable release
  • testing, let interested in testing people, try last changes that are not yet stable
  • developer, just more extreme version of testing
  • buggy, already released versions(of any status) could be marked as buggy after a while to force people upgrade to new bugfix version
  • insecure, extreme version of buggy

0sugar push w/o arguments will push local changes as is. If you want to push with particular stability status, use 0sugar push <status>.

By default all users will use only stable versions and won't upgrade to new versions even if there are stable ones. To force people to upgrade from buggy versions, see bugfix releases workflow.

Release workflows

Useful scenarios for service release process.

Having only stable releases

All releases of your services will have stable stability status.

  • when your code is ready to release
  • 0sugar dist/0sugar dist_bin to make local releases
  • 0sugar push stable to rsync local releases to the server
  • if it was wrong time to release and you want to re-release it, repeat previous steps to rewrite wrong release

Bugfix releases

Services assume simple versioning scheme, if you found bugs in stable version:

  • either 0sugar push stable release once more(users will not mess this new copy of release with previous ones)
  • or 0sugar push stable release with incremented version field
  • if bugs are critical, mark previous release as buggy or insecure to force users upgrade their old releases. If you re-pushed release w/o incrementing version number, you should use revision number
0sugar push {buggy|insecure} {version[-revision]}

Development releases

If you want to test your new version among interested people:

  • do all steps that are mentioned in previous workflows
  • but use either testing or developer status for push command
  • users who interested in testing(enabled testing mode) will get your new development version
  • you don't have to change compatibility-version and version all time for micro releases, 0sugar will implicitly change revision of new pushes and user's saccharin will download last changes

Development tools support

Useful addons to popular development tools. These are optional features that simplify service development process.

Autotools

Add service.m4 macros file to your project. It provides:

  • SERVICE_INIT macro to initialize services support, use it after other (AC|AM)_INIT macros like in [2]
    It takes optional string to define prefix value for DIST_ variables, in most cases it should be just sugar
    Macro exports follows variables:
    • SERVICE_NAME, name of service in service.info file
    • SERVICE_VERSION, version of service in service.info file
    • SERVICE_REVISION, revision number from dist/revision file
    • DIST_NAME, combines prefix macro argument with SERVICE_NAME to use it as a distribution name(see example below)
    • DIST_VERSION, combines SERVICE_VERSION and SERVICE_REVISION to use it as a distribution version(see example below)
    • SERVICE_SUMMARY, summary from service.info file
    • macro also changes PACKAGE and VERSION variables with values from proper DIST_, so it rewrites AC_INIT arguments(you have to use fake values for AC_INIT macro)
    • if [Library] section exists in service.info file, macro will
      • check if SERVICE_VERSION is a plain number
      • export SERVICE_AGE, age from [Library] section
      • export SERVICE_LIBRARY_VERSION which is libtool's -version-info parameter
      • if service.pc.in presents in root directory, rename it to DIST_NAME.pc.in
  • SERVICE_ECHO convenient macro to print various variables

These macros could be used like in Toolkit serivce.

Tips

  • Via binding's variables, service's root directory will be accessible for service users, so for python services(for example), better to use subdirectory for service's import modules like Toolkit does - there is toolkit subdirectory in Toolkit's root.
  • Be careful with machine architecture, especially in making binaries in VMs e.g. XO-1 arch is i586 but built in XO VM, binaries could have i686 thus won't start(0install won't let) on XO-1. Use --arch 0sugar's argument to set targeted architecture explicitly.

Documentation

Retrieved from "https://wiki.sugarlabs.org/index.php?title=Platform_Team/Guide/Sweets_Packaging&oldid=43051"