Difference between revisions of "Platform Team/Infrastructure"

From Sugar Labs
Jump to navigation Jump to search
m
Line 1: Line 1:
 +
This page describes the infrastructure map that the Platform Team provides. See also Sweets [[Platform_Team/Sweets|introduction page]].
 +
 
== Introduction ==
 
== Introduction ==
  
The entire workflow that the Platform Team provides for Sugar doers is whirling around three major services:
+
Technologies involved within the Platform Team infrastructure:
  
* GNU/Linux distributions,
+
* [http://0install.net Zero Install] - Decentralized cross-distribution software installation system that Sweets is based on.
* Sugar Bazaar, and
+
* [http://www.packagekit.org/ PackageKit] - Is used as a backend for Zero Install to work with native package management systems.
* [[Activity Library]].
+
* [http://openbuildservice.org/ Open Build Service (OBS)] - The hosting and build service.
 +
* [https://addons.mozilla.org addons.mozilla.org (AMO)] - The source software for [[#Activity_Library|Activity Library]].
 +
 
 +
The entire workflow is whirling following major services:
  
 
[[File:Platform.png]]
 
[[File:Platform.png]]
  
'''GNU/Linux distributions'''<br>
+
== Components ==
Provide Sugar dependencies and, for major distributions, Sugar packages that might be used as-is.
 
  
'''Bazaar'''<br>
+
=== GNU/Linux distributions ===
In cases where Sugar is not packaged, or there are no required dependencies, packages will be built on Bazaar. But the major purpose of Bazaar is to be a place to consolidate all the efforts of Sugar doers:
 
  
* hosting released sources (so, there is no need in requesting a shell account to upload files to http://download.sugarlabs.org),
+
Provide dependencies for sweets and, for major distributions, Sugar packages that might be used as-is.
* being a universal build farm for binary-based projects,
 
* supporting QA workflows where Bazaar packages might be branched, tested, and released.
 
  
'''Activity Library'''<br>
+
=== obs.sugarlabs.org ===
Once Bazaar is in service, the Activity Library will be just a catalog of activities, all downloading will happen from Bazaar.
 
  
 +
The cornerstone component and intended to be a place to consolidate all the efforts of Sugar doers:
  
== Tools ==
+
* hosting released sources (so, there is no need in requesting a shell account to upload files to http://download.sugarlabs.org),
 +
* being a universal build farm for binary-based projects,
 +
* supporting QA workflows where sweet packages might be branched, tested, and released.
  
To interact with the doer environment from a command line, only the [[Platform Team/Guide/sweets command|sweets]] command is needed. See walk-through [[Platform_Team/Sweets#Further_reading|tutorials]] for various use cases.
+
It is a [http://git.sugarlabs.org/0sugar/build-service patched] version of [http://openbuildservice.org Open Build Service].
  
Technologies involved within the doers environment:
+
In many cases, Bazaar will be used implicitly:
 +
* uploading a new version will be called from the {{Code|sweets commit}} command,
 +
* launching will happen just by calling a sweet url.
  
* [http://0install.net 0install] - The only package management system that is being used ultimately.<br>In some sense, ''0install + sweets, and related infrastructure'' might look similar to ''rpm + yum, and related infrastructure''.
+
The only thing required is being registered on Bazaar. For the default http://bazaar.sugarlabs.org instance, that means being registered in the [https://cas.sugarlabs.org/ Sugar Labs Central Login] system.
* [http://www.packagekit.org/ PackageKit] - Is used as a backend for 0install to work with native package management systems.
 
* [https://build.opensuse.org OBS] - The source software for [[#Bazaar|Bazaar]].
 
* [https://addons.mozilla.org AMO] - The source software for [[#Activity_Library|Activity Library]].
 
  
 +
=== activity.sugarlabs.org ===
  
== Software projects, sweets ==
+
Is the [[Activity Library]]. Once obs.sugarlabs.org is in service, the Activity Library will be just a catalog of activities, all downloading will happen from obs.sugarlabs.org.
  
All software projects (not just the Sugar related ones) are designated ''sweets'' within the doers environment. ''sweets'' might be one of several types:
+
In the doers' environment, the Activity Library will be a catalog of sweets. In other words, activities.sugarlabs.org might be treated as a front-end for Sugar development process (where obs.sugarlabs.org is a back-end) and an analog of the current Developer Hub on Activity Library.
  
* aliases to native GNU/Linux packages, which map the same sweet name to the appropriate package name for a particular GNU/Linux distribution (henceforth, aliases),
+
From usage point of view, only Zero Install interface URL is needed to obtain a new Activity Library entity; the rest will be done by Sweets under the hood.
* recipe based, i.e., native to sugar environment, projects (henceforth, sweets).
 
  
=== [[Platform Team/Recipe Specification|Recipes]] ===
+
=== packages.sugarlabs.org ===
  
This is the starting point for both users' and doers' environments for a particular sweet project. Its major task&mdash;specifying how to prepare the code to launch.
+
It is an [https://build.opensuse.org/ original] Web interface to Open Build Service. It seems to be an overkill for regular Sugar doers' workflow (since OBS is primarily designed to support native packages workflow) when Activity Library's Developers Hub can be used as a more appropriate OBS client. But packages.sugarlabs.org will be useful when people need full OBS control for creating native packages (original ones or basing on existed sweets).
  
Sweets contain a recipe file in the sources directory. Recipes are improved versions of {{Code|activity.info}} files with the difference that recipes fully describe the sweet (i.e., like GNU/Linux distribution spec files), and are intended not only for activities.
+
All packages on packages.sugarlabs.org split into several types:
  
Recipes contain a variety of metadata about the sweet, including important things like version, dependencies, and ''sweet_id''. The value of a ''sweet_id'' is just a short (not unique) name, which is used in various sweet [[#Identifiers and implementations|identifiers]].
+
* regular OBS packages,
 
+
* aliases that map native packages names of several GNU/Linux distributions into one OBS level name,
=== Identifiers and implementations ===
+
* sweets built only for Zero Install usage,
 
+
* native packages built basing on sweets.
All sweets are identified by Web URLs:
 
 
 
* urls for unique identification:
 
<nowiki>http</nowiki>://sweets.''bazaar-hostname''/''user-account''/''sweet_id''
 
* urls to collect all implementations for the same {{Code|sweet_id}}:
 
<nowiki>http</nowiki>://sweets.''bazaar-hostname''/''sweet_id''
 
 
 
Where:
 
 
 
* ''bazaar-hostname'' - The [[#Bazaar|Bazaar]] instance that is being used to host projects.
 
* ''user-account'' - The account name of the developer on [[#Bazaar|Bazaar]].
 
* ''sweet_id'' - The sweet identifier from the project recipe.
 
 
 
The same unique project identifier might be associated with several sweet implementations from the same doer. An implementation is just a copy of the sweet software in a ready-to-use state, e.g., several versions of the sweet, several binary implementations of the same sweet version built against several environments. Only one implementation will be chosen for a running environment, based on operating system, hardware architecture, GNU/Linux distribution, or user preferences (like running only stable versions).
 
 
 
A second type of sweet identifier is used for collecting sweet implementations from several doers. For example, a doer might tweak an existing project, and share it, and want other people to be aware of its existence as another sweet implementation. The process of choosing the proper implementation to run will be extended by additional options, such as choosing implementations only from a particular doer.
 
 
 
=== Development implementations ===
 
 
 
Ready-to-use sweets implementations, i.e., those locally available, might be in two states:
 
 
 
* read-only implementations, which can only be used,
 
* under-development implementations, which might be changed at any time.
 
 
 
In other words, development implementations are just checked-out, sweets sources, placed in one of the default directories, {{Code|~/sweets}} and {{Code|~/Activities}}.
 
 
 
Development implementations have the highest running priority, but that may be changed by the user at any time, just like regular implementations.
 
 
 
Such implementations are a good way for [http://en.wikipedia.org/wiki/Sneakernet sneakernet] sharing of sweets, by just bundling a sweet directory, and extracting it on another machine, as needed.
 
 
 
 
 
== Sweets tracker ==
 
 
 
The Sweets tracker is a DBus service that is being used by the {{Code|sweets}} command and the Sugar Shell to run sweets, and handle other high-level sweets related procedures.
 
 
 
The reasons for having a DBus service are these:
 
 
 
* the need to monitor development implementations and not to scan directories every time,
 
* to cache 0install internal procedures so to speed up the launch process,
 
* to enable background updates, and
 
* to support [http://0install.net/0mirror.html mirroring] of 0install implementations so to share already downloaded files within a local network to decrease Internet traffic.
 
 
 
 
 
== [[Platform_Team/Bazaar|Bazaar]] ==
 
 
 
Bazaar is a patched version of [https://build.opensuse.org/ OBS]. It handles all server-client share related procedures, like hosting sweet sources and implementations. For binary-based sweets, Bazaar is a build farm, as well.
 
 
 
In many cases, Bazaar will be used implicitly:
 
* uploading a new version will be called from the {{Code|sweets commit}} command,
 
* launching will happen just by calling a sweet url.
 
 
 
The only thing required is being registered on Bazaar. For the default http://bazaar.sugarlabs.org instance, that means being registered in the [https://cas.sugarlabs.org/ Sugar Labs Central Login] system.
 
  
 +
=== sweets.sugarlabs.org ===
  
== [[Activity Library]] ==
+
It is being used to host Zero Install feeds produced from sweet sources. After being uploaded to obs.sugarlabs.org, sweet source will be build on OBS and the final result will be shared also as Zero Install feeds on sweets.sugarlabs.org.
  
In the doers' environment, the Activity Library will be a catalog of sweets. It might be treated as a front-end for Sugar development, where [[#Bazaar|Bazaar]] is a back-end, and an analog of the current Developer Hub on Activity Library.
+
=== download.sugarlabs.org ===
  
A sweet URL is all that is needed to obtain a new Activity Library entity; the rest will be fetched from the sweet at the other end of that URL. At some point, [[#Bazaar|Bazaar]] might automatically publish announcements of newly created sweet implementations.
+
All files, sources and binaries, that are hosted or produced on obs.sugarlabs.org, will be finally copied to download.sugarlabs.org. The reason is to reuse existed mirroring infrastructure.

Revision as of 09:35, 14 September 2011

This page describes the infrastructure map that the Platform Team provides. See also Sweets introduction page.

Introduction

Technologies involved within the Platform Team infrastructure:

The entire workflow is whirling following major services:

Platform.png

Components

GNU/Linux distributions

Provide dependencies for sweets and, for major distributions, Sugar packages that might be used as-is.

obs.sugarlabs.org

The cornerstone component and intended to be a place to consolidate all the efforts of Sugar doers:

  • hosting released sources (so, there is no need in requesting a shell account to upload files to http://download.sugarlabs.org),
  • being a universal build farm for binary-based projects,
  • supporting QA workflows where sweet packages might be branched, tested, and released.

It is a patched version of Open Build Service.

In many cases, Bazaar will be used implicitly:

  • uploading a new version will be called from the sweets commit command,
  • launching will happen just by calling a sweet url.

The only thing required is being registered on Bazaar. For the default http://bazaar.sugarlabs.org instance, that means being registered in the Sugar Labs Central Login system.

activity.sugarlabs.org

Is the Activity Library. Once obs.sugarlabs.org is in service, the Activity Library will be just a catalog of activities, all downloading will happen from obs.sugarlabs.org.

In the doers' environment, the Activity Library will be a catalog of sweets. In other words, activities.sugarlabs.org might be treated as a front-end for Sugar development process (where obs.sugarlabs.org is a back-end) and an analog of the current Developer Hub on Activity Library.

From usage point of view, only Zero Install interface URL is needed to obtain a new Activity Library entity; the rest will be done by Sweets under the hood.

packages.sugarlabs.org

It is an original Web interface to Open Build Service. It seems to be an overkill for regular Sugar doers' workflow (since OBS is primarily designed to support native packages workflow) when Activity Library's Developers Hub can be used as a more appropriate OBS client. But packages.sugarlabs.org will be useful when people need full OBS control for creating native packages (original ones or basing on existed sweets).

All packages on packages.sugarlabs.org split into several types:

  • regular OBS packages,
  • aliases that map native packages names of several GNU/Linux distributions into one OBS level name,
  • sweets built only for Zero Install usage,
  • native packages built basing on sweets.

sweets.sugarlabs.org

It is being used to host Zero Install feeds produced from sweet sources. After being uploaded to obs.sugarlabs.org, sweet source will be build on OBS and the final result will be shared also as Zero Install feeds on sweets.sugarlabs.org.

download.sugarlabs.org

All files, sources and binaries, that are hosted or produced on obs.sugarlabs.org, will be finally copied to download.sugarlabs.org. The reason is to reuse existed mirroring infrastructure.