User:Alsroot/trash/Object Bundles: Difference between revisions

(41 intermediate revisions by 5 users not shown)

Line 8:

== Summary ==

~~Package~~ Journal objects to bundles to preserve sugar related metadata.

* package Journal objects to bundles to preserve sugar related metadata

* support composite Journal objects e.g. in case of library bundles instead of having .xol in Journal and unpacked files in ~/Library, all library files will be represented by one Journal entry, could be opened in Browse(''[Metadata]/mime_type'' should be text/html for that purpose) and could be transformed back to .xo on demand(e.g. for uploading library in Browse's pick-file field).

* Object Bundles exist only out of sugar (after uploading to the Journal .xo will be removed)

== Owner ==

Line 17:

Line 19:

== Current status ==

* Targeted release: ~~0.86~~

* Targeted release:

* Last updated: ~~Tue~~ Jul ~~21 02~~:18:17 UTC 2009

* Last updated: Mon Jul 27 05:31:42 UTC 2009

* Percentage of completion: 0%

* Percentage of completion: 75%

== Detailed Description ==

Line 25:

Line 27:

This feature is a first approach to unified format for all types of bundles(in 0.86 it will support only Journal entries and new library bundles).

~~=== Bundle files hierarchy ===~~

Object bundle should have ''METADATA'' file in the top directory of .xo bundle.

* ''MANIFEST'' EOL-terminalted list of files inside bundle

=== METADATA file ===

* ''METADATA'' file in [http://docs.python.org/library/configparser.html INI] format which describes bundle

This file is in [http://docs.python.org/library/configparser.html INI] format which describes how to setup bundle.

~~=== METADATA file ===~~

Object bundle can be installed to Journal in two forms.

~~Any field in METADATA~~ file can have ''~~_file~~'' ~~suffix~~, in that ~~case content of this field~~(~~substring~~ w/o ''~~_file~~'' ~~suffix~~) ~~will~~ be ~~fetched from file inside of the bundle~~.

* '''Each file from bundle as separate Journal entry''' files from bundle will be unpacked and installed as separate Journal entries, bundle itself will be removed(similar to .xoj) final Journal entries can have ''activity'' field, so they are regular Journal entries that imported to Journal(w/o ''activity'' field) or created by activities(with ''activity'' field) ''METADATA'' should contain ''[Entry]'' sections for each Journal entry if bundle has more then one entry, sections should be parted by different suffixes e.g. ''[Entry2]'', ''[Entry.additional]'' etc.

~~====~~ [~~Metadata~~] section ~~====~~

* '''Composite Journal entry''' bundle will be installed as a composite object i.e. as a directory of packaged to the bundle files that are represented by one Journal entry final Journal entry can't have ''activity'' field ''METADATA'' should contain only one ''[Bundle]'' section

~~This section is mandatory. It describes Datastore~~ metadata fields of final entry in Journal.

METADATA file can have one or several sections(depends on installation method) that describe metadata fields of final entry(ies) in Journal.

{| border=1 cellpadding=3 style="border: 1px solid white; border-collapse: collapse; background: #e3e4e5;"

Line 44:

Line 47:

! Notes

|-

| ''~~journal_file~~''

| ''entry''

| ~~optional~~

| mandatory

| if ~~exists~~, ~~this file will be unpacked while uploading bundle~~ to ~~Journal and current section defines its metadata~~ otherwise defines ~~metadata for bundle itself and bundle~~ will be ~~stored as is in~~ Journal

| if bundle is composite, ''entry'' defines access point to composite object (e.g. index.html for library bundles); otherwise it defines file which will be installed to Journal

|-

| ''mime_type''

| mandatory

| define metadata for ~~a file which will be stored in~~ Journal~~(according to~~ ''~~journal_file~~'')

| define metadata for final Journal entry

|-

| ''uid''

| ignored

|-

| ''*''

Line 58:

Line 64:

|}

METADATA file ~~could~~ have ~~several~~ ''~~[Manifest]~~'' ~~sections~~, in that case ~~they would be parted by different suffixes e.g. '~~'~~[Manifest2]~~'', '~~'[Manifest.additional]'' etc. Multi-object bundles could~~ be ~~utilized in >0.86 for collections of objects or actions.~~

Any field in ''METADATA'' file can have ''_file'' suffix, in that case content of this field(substring w/o ''_file'' suffix) will be fetched from file inside of the bundle.

~~=== MANIFEST~~ file ~~===~~

~~File which contains a new-line-terminated list~~ of ~~file names inside~~ bundle.

== Benefit to Sugar ==

Line 82:

Line 84:

** journal entries, former .xoj bundles (0.86)

* make Browse upload object bundles when the server says so

* ~~make~~ Browse ~~open bundles with libraries~~

* in an ideal implementation, composite feature should come from Datastore, but since DS has been extensively hacked in 0.86-0.88 cycles, for the first time, it could be a part of shell code

* InfoSlicer generates object bundles in 0.~~86 environment~~

==== Implementation ====

* [http://git.sugarlabs.org/projects/sugar/repos/objectbundle sugar]

* [http://git.sugarlabs.org/projects/sugar-toolkit/repos/objectbundle sugar-toolkit]

* [http://git.sugarlabs.org/projects/browse/repos/objectbundle Browse]

Example of PHP code which could be used on server side to unpack object bundles to get metadata (like title, description, preview etc.)

require_once('Archive/Zip.php');

function rmtree($dir) {

$dir = "$dir";

if ($dh = opendir($dir)) {

while (FALSE !== ($item = readdir($dh))) {

if ($item != '.' && $item != '..') {

$subdir = $dir . '/' . "$item";

if (is_dir($subdir))

rmtree($subdir);

else

unlink($subdir);

}

closedir($dh);

rmdir($dir);

}

function unbundle($bundle) {

$zip = new Archive_Zip($bundle);

$tmpdir = tempnam('/tmp', 'sugar');

if (file_exists($tmpdir))

unlink($tmpdir);

if (!mkdir($tmpdir)) {

return null;

}

$out = $zip->extract(array('add_path' => $tmpdir, 'by_name' => array('METADATA')));

if (empty($out)) {

rmtree($tmpdir);

return null;

}

$metadata = parse_ini_file($out[0]['filename'], true);

if (!$metadata) {

rmtree($tmpdir);

return null;

}

$result = array();

foreach ($metadata as $section => $items) {

if (!(substr($section, 0, 5) == 'Entry' || $section == 'Bundle'))

continue;

foreach ($items as $name => $value) {

if (substr($name, -5) == '_file') {

$name = substr($name, 0, -5);

$out = $zip->extract(array('add_path' => $tmpdir, 'by_name' => array($value)));

if (empty($out)) {

rmtree($tmpdir);

return null;

}

$value = file_get_contents($tmpdir.'/'.$value);

}

$result[$name] = $value;

}

break;

}

rmtree($tmpdir);

return $result;

}

// example usage of unbundle() function

$bundle = 'foo.xo';

$metadata = unbundle($bundle);

if (!$metadata)

echo 'Can not open '.$bundle."\n";

else

foreach ($metadata as $name => $value)

echo $name.'='.$value."\n";

== How To Test ==

How to test non-composite objects:

# Create a new TurtleArt activity

# Upload the new entry to the SL wiki using Browse

# Use Browse to download the entry back to Journal

# Resume it from Journal

How to test composite object (unfortunately [http://dev.sugarlabs.org/ticket/1101 #1101] prevents uploading bundles with library to wiki):

# Download any .xol bundle

# Activate it(it should be opened in Browse)

# Upload Journal entry with library to SL wiki using Browse

# Download .xo with library and open it

== User Experience ==

~~Except that~~ [~~http:~~/~~/wiki~~.~~laptop.org/go/Journal_entry_bundles .xoj] has:~~

We could have HTTP servers(see [[Features/Server Objects Sharing]]) that request .xo files, in that case Browse will provide object bundle for chosen(in ObjectChooser) Journal entry otherwise Browse will upload raw Journal entries.

* users can create object ~~bundles outside of sugar~~(~~they do not have to follow JSON rules~~)

== Dependencies ==

Line 107:

Line 201:

== Documentation ==

* [http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg06874.html Email thread]

* [[Unified Bundles]]

* [[User:Alsroot/trash/Unified Objects | Unified Objects]]

* [http://wiki.laptop.org/go/Journal_entry_bundles Journal entry bundles]

* [http://wiki.laptop.org/go/Bundle_concepts Bundle concepts]

* [http://wiki.laptop.org/go/Manifest_Specification#Contents_Manifests MANIFEST specification]

== Release Notes ==

Line 116:

Line 213:

== Comments and Discussion ==

* See [[{{TALKPAGENAME}}|discussion tab for this feature]]

~~[[Category:Feature Page Incomplete]]~~

~~[[Category:Feature]]~~

@@ Line 8: / Line 8: @@
 == Summary ==
-Package Journal objects to bundles to preserve sugar related metadata.
+* package Journal objects to bundles to preserve sugar related metadata
+* support composite Journal objects e.g. in case of library bundles instead of having .xol in Journal and unpacked files in ~/Library, all library files will be represented by one Journal entry, could be opened in Browse(''[Metadata]/mime_type'' should be text/html for that purpose) and could be transformed back to .xo on demand(e.g. for uploading library in Browse's pick-file field).
+* Object Bundles exist only out of sugar (after uploading to the Journal .xo will be removed)
 == Owner ==
@@ Line 17: / Line 19: @@
 == Current status ==
-* Targeted release: 0.86
+* Targeted release:
-* Last updated: Tue Jul 21 02:18:17 UTC 2009
+* Last updated: Mon Jul 27 05:31:42 UTC 2009
-* Percentage of completion: 0%
+* Percentage of completion: 75%
 == Detailed Description ==
@@ Line 25: / Line 27: @@
 This feature is a first approach to unified format for all types of bundles(in 0.86 it will support only Journal entries and new library bundles).
-=== Bundle files hierarchy ===
+Object bundle should have ''METADATA'' file in the top directory of .xo bundle.
-* ''MANIFEST'' EOL-terminalted list of files inside bundle
+=== METADATA file ===
-* ''METADATA'' file in [http://docs.python.org/library/configparser.html INI] format which describes bundle
+This file is in [http://docs.python.org/library/configparser.html INI] format which describes how to setup bundle.
-=== METADATA file ===
+Object bundle can be installed to Journal in two forms.
-Any field in METADATA file can have ''_file'' suffix, in that case content of this field(substring w/o ''_file'' suffix) will be fetched from file inside of the bundle.
+* '''Each file from bundle as separate Journal entry'''<br>files from bundle will be unpacked and installed as separate Journal entries, bundle itself will be removed(similar to .xoj)<br>final Journal entries can have ''activity'' field, so they are regular Journal entries that imported to Journal(w/o ''activity'' field) or created by activities(with ''activity'' field)<br>''METADATA'' should contain ''[Entry]'' sections for each Journal entry<br>if bundle has more then one entry, sections should be parted by different suffixes e.g. ''[Entry2]'', ''[Entry.additional]'' etc.
-==== [Metadata] section ====
+* '''Composite Journal entry'''<br>bundle will be installed as a composite object i.e. as a directory of packaged to the bundle files that are represented by one Journal entry<br>final Journal entry can't have ''activity'' field<br>''METADATA'' should contain only one ''[Bundle]'' section
-This section is mandatory. It describes Datastore metadata fields of final entry in Journal.
+METADATA file can have one or several sections(depends on installation method) that describe metadata fields of final entry(ies) in Journal.
 {| border=1 cellpadding=3 style="border: 1px solid white; border-collapse: collapse; background: #e3e4e5;"
@@ Line 44: / Line 47: @@
 ! Notes
 |-
-| ''journal_file''
+| ''entry''
-| optional
+| mandatory
-| if exists, this file will be unpacked while uploading bundle to Journal and current section defines its metadata<br>otherwise defines metadata for bundle itself and bundle will be stored as is in Journal
+| if bundle is composite, ''entry'' defines access point to composite object (e.g. index.html for library bundles);<br>otherwise it defines file which will be installed to Journal
 |-
 | ''mime_type''
 | mandatory
-| define metadata for a file which will be stored in Journal(according to ''journal_file'')
+| define metadata for final Journal entry
+|-
+| ''uid''
+| ignored
 |-
 | ''*''
@@ Line 58: / Line 64: @@
 |}
-METADATA file could have several ''[Manifest]'' sections, in that case they would be parted by different suffixes e.g. ''[Manifest2]'', ''[Manifest.additional]'' etc. Multi-object bundles could be utilized in >0.86 for collections of objects or actions.
+Any field in ''METADATA'' file can have ''_file'' suffix, in that case content of this field(substring w/o ''_file'' suffix) will be fetched from file inside of the bundle.
-=== MANIFEST file ===
-File which contains a new-line-terminated list of file names inside bundle.
 == Benefit to Sugar ==
@@ Line 82: / Line 84: @@
 ** journal entries, former .xoj bundles (0.86)
 * make Browse upload object bundles when the server says so
-* make Browse open bundles with libraries
+* in an ideal implementation, composite feature should come from Datastore, but since DS has been extensively hacked in 0.86-0.88 cycles, for the first time, it could be a part of shell code
-* InfoSlicer generates object bundles in 0.86 environment
+==== Implementation ====
+* [http://git.sugarlabs.org/projects/sugar/repos/objectbundle sugar]
+* [http://git.sugarlabs.org/projects/sugar-toolkit/repos/objectbundle sugar-toolkit]
+* [http://git.sugarlabs.org/projects/browse/repos/objectbundle Browse]
+Example of PHP code which could be used on server side to unpack object bundles to get metadata (like title, description, preview etc.)
+ require_once('Archive/Zip.php');
+ function rmtree($dir) {
+    $dir = "$dir";
+    if ($dh = opendir($dir)) {
+        while (FALSE !== ($item = readdir($dh))) {
+            if ($item != '.' && $item != '..') {
+                $subdir = $dir . '/' . "$item";
+                if (is_dir($subdir))
+                    rmtree($subdir);
+                else
+                    unlink($subdir);
+            }
+        }
+        closedir($dh);
+        rmdir($dir);
+    }
+ }
+ function unbundle($bundle) {
+    $zip = new Archive_Zip($bundle);
+    $tmpdir = tempnam('/tmp', 'sugar');
+    if (file_exists($tmpdir))
+        unlink($tmpdir);
+    if (!mkdir($tmpdir)) {
+        return null;
+    }
+    $out = $zip->extract(array('add_path' => $tmpdir, 'by_name' => array('METADATA')));
+    if (empty($out)) {
+        rmtree($tmpdir);
+        return null;
+    }
+    $metadata = parse_ini_file($out[0]['filename'], true);
+    if (!$metadata) {
+        rmtree($tmpdir);
+        return null;
+    }
+    $result = array();
+    foreach ($metadata as $section => $items) {
+        if (!(substr($section, 0, 5) == 'Entry' || $section == 'Bundle'))
+            continue;
+        foreach ($items as $name => $value) {
+            if (substr($name, -5) == '_file') {
+                $name = substr($name, 0, -5);
+                $out = $zip->extract(array('add_path' => $tmpdir, 'by_name' => array($value)));
+                if (empty($out)) {
+                    rmtree($tmpdir);
+                    return null;
+                }
+                $value = file_get_contents($tmpdir.'/'.$value);
+            }
+            $result[$name] = $value;
+        }
+        break;
+    }
+    rmtree($tmpdir);
+    return $result;
+ }
+ // example usage of unbundle() function
+ $bundle = 'foo.xo';
+ $metadata = unbundle($bundle);
+ if (!$metadata)
+    echo 'Can not open '.$bundle."\n";
+ else
+    foreach ($metadata as $name => $value)
+        echo $name.'='.$value."\n";
 == How To Test ==
+How to test non-composite objects:
 # Create a new TurtleArt activity
 # Upload the new entry to the SL wiki using Browse
 # Use Browse to download the entry back to Journal
 # Resume it from Journal
+How to test composite object (unfortunately [http://dev.sugarlabs.org/ticket/1101 #1101] prevents uploading bundles with library to wiki):
+# Download any .xol bundle
+# Activate it(it should be opened in Browse)
+# Upload Journal entry with library to SL wiki using Browse
+# Download .xo with library and open it
 == User Experience ==
-Except that [http://wiki.laptop.org/go/Journal_entry_bundles .xoj] has:
+We could have HTTP servers(see [[Features/Server Objects Sharing]]) that request .xo files, in that case Browse will provide object bundle for chosen(in ObjectChooser) Journal entry otherwise Browse will upload raw Journal entries.
-* users can create object bundles outside of sugar(they do not have to follow JSON rules)
 == Dependencies ==
@@ Line 107: / Line 201: @@
 == Documentation ==
+* [http://www.mail-archive.com/sugar-devel@lists.sugarlabs.org/msg06874.html Email thread]
 * [[Unified Bundles]]
+* [[User:Alsroot/trash/Unified Objects | Unified Objects]]
 * [http://wiki.laptop.org/go/Journal_entry_bundles Journal entry bundles]
 * [http://wiki.laptop.org/go/Bundle_concepts Bundle concepts]
+* [http://wiki.laptop.org/go/Manifest_Specification#Contents_Manifests MANIFEST specification]
 == Release Notes ==
@@ Line 116: / Line 213: @@
 == Comments and Discussion ==
 * See [[{{TALKPAGENAME}}|discussion tab for this feature]] <!-- This adds a link to the "discussion" tab associated with your page.  This provides the ability to have ongoing comments or conversation without bogging down the main feature page -->
-[[Category:Feature Page Incomplete]]
-[[Category:Feature]]