Difference between revisions of "Platform Team/Guide/Sweets Packaging"

Revision as of 13:03, 15 January 2010

Introduction

The purpose of this Guide is describing how to get benefits of Sugar Services for your library/application, simplify support fork flows and let your users(developers) use your code in simple and convenient way.

This guide is about how to package your code to services, if you are looking for method how to install some software from native packages, see Native packages usage guide.

If service is not wrapper for native package and could be used by other developers, please consider possibility to create service page using template.

Detailed description

The result for service development process will be regular 0install files that are placed to subdirectory on http://download.sugarlabs.org/services/ with name which is identity for the service in the rest of 0sugar infrastructure. Service's subdirectory will contain:

runtime.xml, regular 0install feed file which will be used as a runtime dependency
buildtime.xml, regular 0install feed file which will be used as a buildtime dependency
bunch of tarballs with sources and binaries, feed files contain links to these files

Feed files, in addition to several information fields, contain tags that describe implementations of the service(i.e. releases). At the same time, feed file contains implementations for various service versions and the same version could be represented by several implementations:

source implementation in <implementation> tag with attribute arch == *-src, with links to source tarball and instructions how to build it
no arch implementation in <implementation> tag with attribute arch == *-*, which can be used as-is on different platforms e.g. pure python services
one or several binary implementations in <implementation> tags, that contain binaries of the same source implementation but for various architectures(attribute arch)
package implementation in <package-implementation> tag, to reuse native packages, see Native packages usage for more info.

0sugar tool

To start developing services, install injector tool, and 0sugar - main tool to maintain services infrastructure.

0alias 0sugar http://download.sugarlabs.org/services/0sugar/runtime.xml

While working, 0sugar creates dist/ directory in project's root and uses it as a rsynced copy of service's directory on the server. So, you need ssh access to sunjammer server, create ticket for shell.sugarlabs.org category on http://bugs.sugarlabs.org/.

0sugar commands are:

init, create service/ directory and default service.info file
clone, rsync entirely dist/ from the server
dist, create source or any arch tarball/implementation in dist/ directory and tweak dist/feed.xml correspondingly
If there is second option-less argument, it will be treated as a path of tarball to release(e.g. you can create such tarball by make distcheck command for autotools based projects), otherwise 0sugar will tar all project files(and try to avoid temporary files as much as possible)
dist will create
- any arch archive, if build-command field was omitted in service.info
- source archive, if service.info has build-command field
dist_bin, create binary tarball/implementation for current arch in dist/ directory and tweak dist/feed.xml correspondingly
if second argument was passed, it will be treated as a name of service.info section with additional requires fields
push, rsync dist/ to the server
lint, check feed file on the server
will run FeedLint application for http://download.sugarlabs.org/services/ feed

Feed files in dist/ directory could be changed manually but keep in mind that dist* commands will rewrite entries for current service version.

Versioning scheme

There could be two different versioning scheme for services.

Upstream versions

If service is just a wrapper around existed project with its own versioning scheme then there is even no need in any mentioning of versions in service.info file. Particular version will be chosen on every 0sugar dist* command invoking or will be fetched from native packaging systems(if service doesn't provide binaries and only contains information about already well packaged in various GNU/Linux distributions).

Native versions

The rest of services will use scheme:

<version>-<revision>

<version> is a version field from service.info file which is a plain number
<revision> is an auto incremented number which is beeing changed by 0sugar push command (like Subversion revision)

Commands dist and dist_bin rewrite releases with the same version versions and don't remove other releases.

Libraries have also age parameter which is libtool like age version part. age won't be exposed in version string but will be present in feed files and 0sugar will check it while deciding what dependency version should be chosen.

Stability status

One of core differences from Sucrose development process is having not stable "releases" (here, release is a result of ``0sugar push`` command). So, services need stability status. Internally each service release will have one of 0install stability statuses:

stable, final stable release
testing, let interested in testing people, try last changes that are not yet stable
developer, just more extreme version of testing
buggy, already released versions(of any status) could be marked as buggy after a while to force people upgrade to new bugfix version
insecure, extreme version of buggy

0sugar push w/o arguments will push local changes as is. If you want to push with particular stability status, use 0sugar push <status>.

By default all users will use only stable versions and won't upgrade to new versions even if there are stable ones. To force people to upgrade from buggy versions, see bugfix releases workflow.

Development workflows

Service developer can follow one of several step-by-step scenarios:

Activity specific Services HOWTO if your activity uses its own binaries
Binary-less Services HOWTO to make any architecture service e.g. python based
Binary Services HOWTO services that contain binaries
Upstream Services HOWTO if your activity or service uses project which is not part of Sugar Platform, follow this howto to create wrapper around upstream project and just add its name to requires section of .info file

Release workflows

Having only stable releases

All releases of your services will have stable stability status.

when your code is ready to release
0sugar dist* to make bundles
0sugar push stable to rsync bundles to the server
if it was wrong time to release and you want to re-release it, repeat previous steps to rewrite wrong release

Bugfix releases

Services assume simple versioning scheme, if you found bugs in stable version:

either 0sugar push stable release once more(users will not mess this new copy of release with previous ones)
or 0sugar push stable release with incremented version field
if bugs are critical, mark previous release as buggy or insecure to force users upgrade their old releases. If you re-pushed release w/o incrementing version number, you should use revision number

0sugar push {buggy|insecure} {version[-revision]}

Development releases

If you want to test your new version among interested people:

do all steps that are mentioned in previous workflows
but use either testing or developer status for push command
users who interested in testing(enabled testing mode) will get your new development version
you don't have to change version all time for micro releases, 0sugar will implicitly change revision of new pushes and users will download last changes

Tips

Via binding's variables, service's root directory will be accessible for service users, so for python services(for example), better to use subdirectory for service's import modules like Toolkit does - there is toolkit subdirectory in Toolkit's root.

Be careful with machine architecture, especially in making binaries in VMs e.g. XO-1 arch is i586 but built in XO VM, binaries could have i686 thus won't start(0install won't let) on XO-1. Use --arch 0sugar's argument to set targeted architecture explicitly.

Documentation

@@ Line 1: / Line 1: @@
-<noinclude>{{GoogleTrans-en}}{{TOCright}}
+<noinclude>
+{{TOCright}}
 [[Category:Activity Team]]
 </noinclude>
@@ Line 25: / Line 26: @@
 * one or several binary implementations in ''<implementation>'' tags, that contain binaries of the same source implementation but for various architectures(attribute ''arch'')
 * package implementation in ''<package-implementation>'' tag, to reuse native packages, see [[Documentation_Team/Services/Native_packages_usage|Native packages usage]] for more info.
-== service.info file ==
-Service publishing workflow is similar to activities. There is an info file, here ''service/service.info'', which describes current status(not history) of development process. All these fields, finally, go to feed file while publishing new version.
-=== [Service] section ===
-File ''service.info' should contain at least ''[Service]'' section:
-* ''name'', the identity of service, this field defines name of feed's root directory on the server http://download.sugarlabs.org/services/ as well
-* ''summary'', short descriptive line
-* ''description'', long descriptive line, to wrap long text, all lines after second, should start with spaces<br>could be omited to reuse ''summary'' field
-* ''license'', in 0install, typically a Trove category, as used on freshmeat.net, but could be [http://fedoraproject.org/wiki/Licensing#Good_Licenses fedora names], since the rest of sugar uses them
-* ''homepage'' web link to project home page
-=== [Runtime] section ===
-This section describes parameters that should be taken into account only if service has 0install implementations(including source implementations) in contrast with [[Documentation Team/Services/Native packages usage|pure]] wrappers around native packages.
-Section contains:
-* ''version'', used only for services with [[#Native_versions|native]] versioning scheme
-* ''binding'', what environment variables, 0install should export to session which uses this service
- binding = [prepend|append|replace] ''<variable-name>'' [''<insert-text-to-prepand-variable-value>''] [; ...]
-* ''main'', for applications, path to exec file from service root directory
-* ''requires'', for any arch and binary implementations, list of runtime services that should exist before using service
- requires = <service-name> [<not-before-version>[-<before-version>]] [; ...]
-* exec, command how to make bundle for ''dist'' command, ''0sugar'' will pick up newly created file in service root directory<br>Shell command, executed from service root directory. If this command starts to get complicated, you should move it to a script and just set this attribute to the command to run the script.<br>By default ''0sugar'' just bundle entirely service directory excluding temporary files.
- exec = make distcheck
-=== [Library] section ===
-Adds additional parameters if service is a library. Libraries should have plain ''version'' value and with ''age'' field, it will be used in [http://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html#Libtool-versioning libtool] like style e.g. in [[#Autotools|autotools]] based projects.
-Section contains:
-* ''age'' libtool like age of version
-=== [Buildtime] section ===
-File ''service.info'' could have optional ''[Buildtime]'' section which describes additional parameters that should be taken into account during other service(which uses this one) building i.e. it is an equivalent for *-dev(el) packages in GNU/Linux distributions.
-Section contains:
-* ''requires'', additional dependencies that will be required during building other service which depends on this one
-* ''exec'', command how to make buildtime bundle for ''dist'' command<br>Shell command, executed from service root directory. If this command starts to get complicated, you should move it to a script and just set this attribute to the command to run the script.<br>By default ''0sugar dist'' uses the same bundle as for runtime feed
-* ''generate-documentation'', if set, documentation project will be created on [http://api.sugarlabs.org/services/], for now only ''epydoc'' value is supported
-=== [Build] section ===
-If service contains binary implementations, this section should present to describe building process.
-Section contains:
-* ''requires'', what services should be installed before building this one from sources
- requires = <service-name> [<not-before-version>[-<before-version>]] [; ...]
-* ''exec'', command how to build binaries<br>Its value is a shell command, executed inside the build directory $BUILDDIR. It must compile the source in $SRCDIR, putting the final result (ready for distribution) in $DISTDIR. If this command starts to get complicated, you should move it to a script (either inside the main source archive or in a separate dependency) and just set this attribute to the command to run the script.<br>'''NOTE''': This command will be executed not only in service developer environment but also on user side if proper binary wasn't found, so do not use here any development related commands like ''autogen.sh''<br>For example, followed command builds regular autotools based project
- exec = "$SRCDIR"/configure --prefix="$DISTDIR" && make install
-=== Support several ABIs of service dependencies ===
-File ''service.info'' could contain other sections for various binary implementations. These sections are intended to describe particular dependencies environment and should contain only:
-* ''requires'' that overwrite ''Service'''s section ''requires''
-For example if service developer is going to add binary implementations for f9/f11, and the same service dependency in these distributions has different ABIs, activity developer has to build two additional binaries, for f9 and f11. In that case developer creates two sections ''F9'' and ''F11'', put exact dependency versions(to separate particular section from others) and invoke ''0sugar dist_bin <section-name>'' in proper f9/f11 environment.
 == 0sugar tool ==
@@ Line 115: / Line 47: @@
 Feed files in ''dist/'' directory could be changed manually but keep in mind that ''dist*'' commands will rewrite entries for current service version.
-== Development workflows ==
-How to build services.
-=== Any arch services ===
-Work flow for services that don't require compilation stage thus could be just packaged and used as is on server side:
-* create local initial service.info by invoking ''0sugar init'', tweak newly created ''service/service.info'' file
-* increase/change version field in ''service.info'' file
-* ''0sugar dist'' to change local feed in ''dist/'' and place there tarball
-* ''0sugar push stable'' to rsync changed files from ''dist/'' to the server<br>see [[#Release workflows]] for other release scenarios
-=== Binary services ===
-Work flow for services that require compilation stage:
-* create local initial service.info by invoking ''0sugar init'', tweak newly created ''service/service.info'' file
-* provide relevant value for ''build-command'' field in ''service.info'' file
-* if you want to support binaries for different dependencies environments, add additional sections to ''service.info'' file with ''requires'' fields that describe dependencies versions of particular environment
-* increase/change version field in ''service.info'' file
-* ''0sugar dist'' to add sources tarball to ''dist/'' and change local feed correspondingly
-* on every platform, you are trying to support binaries for, run ''0sugar dist_bin'' to build, and add to local field, binaries for this particular platform
-* if you added additional sections to ''service.info'', run ''0sugar dist_bin <section-name>'' in every environment, ''services.info'' has additional sections for
-* ''0sugar push stable'' to rsync changed files from ''dist/'' to the server<br>see [[#Release workflows]] for other release scenarios
 == Versioning scheme ==
@@ Line 176: / Line 82: @@
 By default all users will use only ''stable'' versions and won't upgrade to new versions even if there are stable ones. To force people to upgrade from buggy versions, see [[#Bugfix_releases|bugfix releases]] workflow.
+== Development workflows ==
+Service developer can follow one of several step-by-step scenarios:
+* [[Documentation Team/Services/Activity specific Services HOWTO|Activity specific Services HOWTO]] if your activity uses its own binaries
+* [[Documentation Team/Services/Binary-less Services HOWTO|Binary-less Services HOWTO]] to make any architecture service e.g. python based
+* [[Documentation Team/Services/Binary Services HOWTO|Binary Services HOWTO]] services that contain binaries
+* [[Documentation Team/Services/Upstream Services HOWTO|Upstream Services HOWTO]] if your activity or service uses project which is not part of Sugar Platform, follow this howto to create wrapper around upstream project and just add its name to ''requires'' section of .info file
 == Release workflows ==
-Useful scenarios for service release process.
+'''Having only stable releases'''
-=== Having only stable releases ===
 All releases of your services will have ''stable'' stability status.
 * when your code is ready to release
-* ''0sugar dist''/''0sugar dist_bin'' to make local releases
+* ''0sugar dist*'' to make bundles
-* ''0sugar push stable'' to rsync local releases to the server
+* ''0sugar push stable'' to rsync bundles to the server
 * if it was wrong time to release and you want to re-release it, repeat previous steps to rewrite wrong release
-=== Bugfix releases ===
+'''Bugfix releases'''
 Services assume simple versioning scheme, if you found bugs in stable version:
@@ Line 200: / Line 113: @@
 sugar push {buggy|insecure} {version[-revision]}
-=== Development releases ===
+''' Development releases '''
 If you want to test your new version among interested people:
@@ Line 207: / Line 120: @@
 * but use either ''testing'' or ''developer'' status for ''push'' command
 * users who interested in testing(enabled testing mode) will get your new development version
-* you don't have to change ''compatibility-version'' and ''version'' all time for micro releases, ''0sugar'' will implicitly change revision of new pushes and user's saccharin will download last changes
+* you don't have to change ''version'' all time for micro releases, ''0sugar'' will implicitly change revision of new pushes and users will download last changes
-== Development tools support ==
-Useful addons to popular development tools. These are optional features that simplify service development process.
-=== Autotools ===
-Add [http://git.sugarlabs.org/projects/toolkit/repos/mainline/blobs/master/m4/service.m4 service.m4] macros file to your project. It provides:
-* ''SERVICE_INIT'' macro to initialize services support, use it after other (AC|AM)_INIT macros like in [http://git.sugarlabs.org/projects/toolkit/repos/mainline/blobs/master/configure.ac]<br>It takes optional string to define prefix value for ''DIST_'' variables,  in most cases it should be just ''sugar''<br>Macro exports follows variables:
-** ''SERVICE_NAME'', name of service in service.info file
-** ''SERVICE_VERSION'', version of service in service.info file
-** ''SERVICE_REVISION'', revision number from dist/revision file
-** ''DIST_NAME'', combines prefix macro argument with ''SERVICE_NAME'' to use it as a distribution name(see example below)
-** ''DIST_VERSION'', combines ''SERVICE_VERSION'' and ''SERVICE_REVISION'' to use it as a distribution version(see example below)
-** ''SERVICE_SUMMARY'', summary from service.info file
-** macro also changes ''PACKAGE'' and ''VERSION'' variables with values from proper ''DIST_'', so it rewrites ''AC_INIT'' arguments(you have to use fake values for ''AC_INIT'' macro)
-** if ''[Library]'' section exists in service.info file, macro will
-*** check if ''SERVICE_VERSION'' is a plain number
-*** export ''SERVICE_AGE'', age from ''[Library]'' section
-*** export ''SERVICE_LIBRARY_VERSION'' which is libtool's -version-info parameter
-*** if ''service.pc.in'' presents in root directory, rename it to ''DIST_NAME''.pc.in
-* ''SERVICE_ECHO'' convenient macro to print various variables
-These macros could be used like in [http://git.sugarlabs.org/projects/toolkit/repos/mainline/trees/master Toolkit] serivce.
 == Tips ==