Difference between revisions of "Platform Team/Infrastructure"

Latest revision as of 00:30, 23 September 2012

This page describes the infrastructure map that the Platform Team provides. See also Sweets introduction page.

Introduction

Technologies involved within the Platform Team infrastructure

Zero Install - Decentralized cross-distribution software installation system that Sweets is based on.
PackageKit - Is used as a backend for Zero Install to work with native package management systems.
Open Build Service (OBS) - The hosting and build service.
addons.mozilla.org (AMO) - The source software for Activity Library.

The entire workflow is whirling around the following major services: <imagemap> File:Platform.png rect 8 144 192 184 [1] rect 280 160 423 200 [2] rect 504 200 679 240 [3] rect 72 223 239 264 [4] rect 8 296 184 336 [5] desc none </imagemap>

Services

obs.sugarlabs.org

The cornerstone component that's intended to be a place to consolidate all the efforts of Sugar doers regarding the sharing of code:

hosting released sources (so, there is no need in requesting a shell account to upload files to http://download.sugarlabs.org),
being a universal build farm for binary-based projects.

It is an instance of Open Build Service.

In many cases, obs.sugarlabs.org will be used implicitly:

while uploading a new version from the sweets commit command,
while managing already released versions (and uploading new ones using the Web UI) from Activity Library's Developers Hub,
launching will happen just by calling a sweet URL.

The only thing required is being registered in the Sugar Labs Central Login system.

activities.sugarlabs.org

This is the Activity Library. Once obs.sugarlabs.org is in service, the Activity Library will be (for now, it is entirely .xo based) just a catalog of activities, all downloading will happen from download.sugarlabs.org.

In the doers' environment, the Activity Library will be a catalog of sweets. In other words, activities.sugarlabs.org might be treated as a front-end for the Sugar development process (where obs.sugarlabs.org is a back-end) and an analog of the current Developer Hub on Activity Library.

From a usage point of view, only the Zero Install interface URL is needed to obtain a new Activity Library entity; the rest will be done by Sweets under the hood.

packages.sugarlabs.org

It is an original Web interface to Open Build Service. It seems to be an overkill for regular Sugar doers' workflow (since OBS is primarily designed to support a native packages workflow) when Activity Library's Developers Hub can be used as a more appropriate OBS client. But packages.sugarlabs.org will be useful when people need full OBS control for creating native packages (original ones or those based on existing sweets).

All packages on packages.sugarlabs.org split into several types:

regular OBS packages,
aliases that map native packages names of several GNU/Linux distributions into one OBS level name,
sweets built only for Zero Install usage,
native packages built based on sweets.

sweets.sugarlabs.org

It is being used to host Zero Install feeds produced from sweet sources. After being uploaded to obs.sugarlabs.org, a sweet source will be built on OBS, and the final result will be shared as Zero Install feeds on sweets.sugarlabs.org.

download.sugarlabs.org

All files, sources, and binaries, that are hosted or produced on obs.sugarlabs.org, will be finally copied to download.sugarlabs.org. This allows us to reuse the existing mirroring infrastructure.

@@ Line 1: / Line 1: @@
-Core issues that need to be fixed to accomplish the purpose:
+This page describes the infrastructure map that the Platform Team provides. See also Sweets [[Platform_Team/Sweets|introduction page]].
-# Sugar projects dependencies.
+== Introduction ==
-#* There is no decent way to treat activity dependencies for now.
-# Binary based Sugar projects.
-#* Existing scheme works fine only if Sugar projects are Python-based or intended to be launched only on a particular platform (or a particular version of a Sugar distribution).
-#* There is only one regular way to build development versions of Glucose projects - sugar-jhbuild, which is far less obvious than the activity development process, i.e., ''creating a sub-environment that includes glucose and all dependencies'' vs. ''checkout glucose project and run it''.
-# Convenient sharing of Sugar projects.
+Technologies involved within the Platform Team infrastructure
-#* Partially, complete? relates to sharing dependencies and binary based projects/dependencies issues.
-#* Excluding network-less environments, the only thing anyone should know about any Sugar project to run it from anywhere, is an unique Web url.
-=== Tools ===
+* [http://0install.net Zero Install] - Decentralized cross-distribution software installation system that Sweets is based on.
+* [http://www.packagekit.org/ PackageKit] - Is used as a backend for Zero Install to work with native package management systems.
+* [http://openbuildservice.org/ Open Build Service (OBS)] - The hosting and build service.
+* [https://addons.mozilla.org addons.mozilla.org (AMO)] - The source software for [[#Activity_Library|Activity Library]].
-To interact with doer environment from command line, only [[Platform Team/Guide/sweets command|sweets]] command is needed. See walk-through [[Platform_Team/Sugar_Doers_Kit#Start_from|tutorials]] for various use cases.
+The entire workflow is whirling around the following major services:
+<imagemap>
+File:Platform.png
+rect 8 144 192 184 [http://packages.sugarlabs.org/]
+rect 280 160 423 200 [http://obs.sugarlabs.org/]
+rect 504 200 679 240 [http://activities.sugarlabs.org/]
+rect 72 223 239 264 [http://sweets.sugarlabs.org/]
+rect 8 296 184 336 [http://download.sugarlabs.org/]
+desc none
+</imagemap>
-Involved technologies within doers environment:
+== Services ==
-* [http://0install.net 0install] the only package management system which is being used on high level.<br>In some meaning, ''0install + sweets and related infrastructure'' might look similar to ''rpm + yum and related infrastructure''.
+=== obs.sugarlabs.org ===
-* [http://www.packagekit.org/ PackageKit] as a backend for 0install to work with native package management systems.
-* [https://build.opensuse.org OBS] as the source for [[#Bazaar|Bazaar]].
-* [https://addons.mozilla.org AMO] as the source for [[#Activity_Library|Activity Library]].
-=== Software projects, sweets ===
+The cornerstone component that's intended to be a place to consolidate all the efforts of Sugar doers regarding the sharing of code:
-Software projects (not only Sugar related) are named ''sweets'' within doers environment and might be one of several types:
+* hosting released sources (so, there is no need in requesting a shell account to upload files to http://download.sugarlabs.org),
+* being a universal build farm for binary-based projects.
-* aliases to native GNU/Linux packages that map the same sweet name to appropriate package name for particular GNU/Linux distribution (further, aliases),
+It is an [[Platform Team/Open Build Service|instance]] of [http://openbuildservice.org Open Build Service].
-* recipe based, i.e., native for sugar environment, projects (further, sweets).
-==== [[Platform Team/Recipe Specification|Recipes]] ====
+In many cases, obs.sugarlabs.org will be used implicitly:
+* while uploading a new version from the {{Code|sweets commit}} command,
+* while managing already released versions (and uploading new ones using the Web UI) from [[Activity Library]]'s [http://activities.sugarlabs.org/en-US/developers Developers Hub],
+* launching will happen just by calling a sweet URL.
-This is the departing point for both, users and doers, environments for the particular sweet project. It is intended to accomplish the major task - how to prepare the code to launch.
+The only thing required is being registered in the [https://cas.sugarlabs.org/ Sugar Labs Central Login] system.
-Sweets contain recipe file in sources directory. Recipes are improved versions of {{Code|activity.info}} files with the difference that recipes fully describe the sweet (i.e., like GNU/Linux distribution spec files do) and intended not only for activities.
+=== activities.sugarlabs.org ===
-Recipes contain various metadata about the sweet, including important ones like version, dependencies and ''sweet_id''. The value of ''sweet_id'' is just a short (not unique) name which is being used in various sweet [[#Identifiers and implementations|identifiers]].
+This is the [[Activity Library]]. Once obs.sugarlabs.org is in service, the Activity Library will be (for now, it is entirely [[Development_Team/Almanac/Activity_Bundles|.xo]] based) just a catalog of activities, all downloading will happen from download.sugarlabs.org.
-==== Identifiers and implementations ====
+In the doers' environment, the Activity Library will be a catalog of sweets. In other words, activities.sugarlabs.org might be treated as a front-end for the Sugar development process (where obs.sugarlabs.org is a back-end) and an analog of the current Developer Hub on Activity Library.
-All sweets are identified by Web URLs:
+From a usage point of view, only the Zero Install interface URL is needed to obtain a new Activity Library entity; the rest will be done by Sweets under the hood.
-* urls for unique identification:
+=== packages.sugarlabs.org ===
- <nowiki>http</nowiki>://sweets.''bazaar-hostname''/''user-account''/''sweet_id''
-* urls to collect all implementations for the same {{Code|sweet_id}}:
- <nowiki>http</nowiki>://sweets.''bazaar-hostname''/''sweet_id''
-Where:
+It is an [https://build.opensuse.org/ original] Web interface to Open Build Service. It seems to be an overkill for regular Sugar doers' workflow (since OBS is primarily designed to support a native packages workflow) when Activity Library's Developers Hub can be used as a more appropriate OBS client. But packages.sugarlabs.org will be useful when people need full OBS control for creating native packages (original ones or those based on existing sweets).
-* ''bazaar-hostname'', [[#Bazaar|Bazaar]] instance which is being used to host projects
+All packages on packages.sugarlabs.org split into several types:
-* ''user-account'', account name of the developer on [[#Bazaar|Bazaar]]
-* ''sweet_id'', sweet identifier from the project recipe
-The same unique identifier might be associated with several sweet implementations from the same doer. Implementation is just a copy of the sweet in ready-to-use state, e.g., several versions of the sweet, several binary implementations of the same sweet version built against several environments. Only one implementation will be chosen in running environment basing on operating system, hardware architecture, GNU/Linux distribution or user preferences (like running only stable versions).
+* regular OBS packages,
+* aliases that map native packages names of several GNU/Linux distributions into one OBS level name,
+* sweets built only for Zero Install usage,
+* native packages built based on sweets.
-The reason to have second type of sweet identifiers is collecting sweet implementations from several doers. For example, doer might tweak existed project and share it, other people will be aware of existence of another sweet implementations. The process of choosing the proper implementation to run will be extended by additional options like choosing implementations only from particular doer.
+=== sweets.sugarlabs.org ===
-==== Development implementations ====
+It is being used to host Zero Install feeds produced from sweet sources. After being uploaded to obs.sugarlabs.org, a sweet source will be built on OBS, and the final result will be shared as Zero Install feeds on sweets.sugarlabs.org.
-Ready to use sweets implementations, i.e., locally available, might be in two states:
+=== download.sugarlabs.org ===
-* read-only implementations that might be only used,
+All files, sources, and binaries, that are hosted or produced on obs.sugarlabs.org, will be finally copied to download.sugarlabs.org. This allows us to reuse the existing mirroring infrastructure.
-* under development implementations that might be changed at anytime.
-In other words, development implementations are just checked out sweets sources placed to one of default directories, by default {{Code|~/sweets}} and {{Code|~/Activities}}.
-Development implementations have the highest running priority but it might be changed by user at any time since these are regular implementations.
-Such implementations is a good way for [http://en.wikipedia.org/wiki/Sneakernet sneakernet] sharing of sweets, just bundling sweet directory and extracting it on another machine is needed.
-=== Sweets tracker ===
-Sweets tracker is a DBus service that is being used by {{Code|sweets}} command and Sugar Shell to run sweets and handle other high-level sweets related procedures.
-The reasons to have DBus service are:
-* need to monitor development implementations to not scan directories every time,
-* cache 0install internal procedures to speed up launch process,
-* background updates,
-* support [http://0install.net/0mirror.html mirroring] of 0install implementations to share already downloaded files within local network to decrease internet traffic.
-=== [[Platform_Team/Bazaar|Bazaar]] ===
-Bazaar is a patched version of [https://build.opensuse.org/ OBS]. It handles all server-client share related procedures, like hosting sweet sources and implementation. For binary based sweets, Bazaar is a build farm as well.
-In many cases Bazaar will be used implicitly:
-* uploading new version will be called from {{Code|sweets commit}} command,
-* launch will happen just by mentioning sweet url.
-The only thing is needed, being registered on Bazaar. For default http://bazaar.sugarlabs.org instance, being registered in [https://cas.sugarlabs.org/ Sugar Labs Central Login] system.
-=== [[Activity Library]] ===
-In doers environment, Activity Library will be a catalog of sweets. It might be treated as a frontend of Sugar development, when [[#Bazaar|Bazaar]] is a backend and an analog of the current Developer Hub on Activity Library.
-The only sweet URL is needed to add new Activity Library entity, the rest will be fetched from the sweet. Also [[#Bazaar|Bazaar]] might automatically publish information of newly created sweet implementations.