Features/GTK3
Summary
Sugar needs to rebase itself on new generations of its key underlying technologies: GTK+ 3 and PyGObject Introspection. This page aims to summarise options and community opinions around this challenging shift, and to help formulate a plan of how it will be executed. In other words, it tries to take community input, answer all the unanswered questions, and present a logical path forward that can be adopted by the developers.
Owner
- This plan/proposal maintained by Daniel Drake (other editors welcome!)
- A number of developers will be needed at each stage to successfully execute this; Daniel offers his assistance for a coordination/oversight role if that would be useful.
Current status
- Targeted release: (SUGAR_VERSION)
- Last updated: (DATE)
- Percentage of completion: XX%
Detailed Description
Two major changes have recently occurred in Sugar's underlying technologies. Firstly, GTK+ 2 has been obsoleted by GTK+ 3, and GNOME is now based on GTK+ 3. Secondly, PyGTK, the underlying Python library that Sugar uses to call into GTK+, has been deprecated in favour of PyGObject Introspection (hereafter "PyGI").
In order to remain innovative and current, and to take advantage of the latest developments, Sugar needs to follow suit and move to GTK3 and PyGI. Lagging behind on this conversion is already bringing negative consequences to Sugar; notably 2 of our most important activities (Read and Browse) are already broken and without a future until we catch up again.
Unfortunately, this is an API-incompatible change. As confirmed by the PyGI developers, PyGTK and PyGI cannot be mixed in the same process. This means that the conversion process can't be done on a fine, incremental basis, and if sugar-toolkit were to be simply replaced with a PyGI/GTK3 port, this means that all existing activities would stop working until they themselves have also been ported - all activities will need modifications as part of this feature. The community has expressed desire for old activities to continue working (many are unmaintained); unfortunately this is not realistic in the long term. As a compromise, this feature discussion includes the requirement that for a certain period of time, both PyGTK/GTK2 and PyGI/GTK3 activities must be able to function alongside each other.
This task will involve modifications to almost every file within Sugar and its activities. Although much of this work can be automated, and the porting process for a given chunk of code is usually trivial, it is still a huge task. This feature discussion attempts to identify ways that this porting process can be done in distinct stages, where Sugar remains functional at the completion of each stage, making this more manageable.
Benefit to Sugar
- PyGI is technologically better than PyGTK. It is a nicer way of calling into GObject-style libraries from Python that means less maintenance is needed upstream (PyGObject automates the creation of bindings to a degree much higher than PyGTK ever could, and automatically achieves more complete coverage).
- PyGTK is no longer maintained; PyGI is actively maintained.
- The move to GTK3 allows us to keep up with our GNOME neighbours, as they improve and refine the base technologies that we share.
- The move to PyGI is expected to result in lower memory usage and faster startup.
- Browse has no future under GTK2, it needs to move to WebKit and that move is dependent on Sugar moving to PyGI/GTK3.
- Similarly, Read has a grim future under GTK2 due to static evince bindings no longer being maintained and libevince itself moved to GTK3; we need to switch to PyGI/GTK3 to be able to keep calling into evince and let the Read activity live on.
Implementation Plan
Sugar is divided into different components, many of which run in different processes. This means that we are able to divide up the required work on a process-by-process basis. While this work is being conducted, some Sugar processes might be based on PyGI/GTK3 and others based on PyGTK/GTK2, but the platform would keep functioning at each stage. There would be some system resource overhead during this transitional time (as the system would need to have PyGTK, GTK2, PyGI and GTK3 all in memory) but this feature implementation would end with the whole sugar ecosystem using PyGI/GTK3.
HippoCanvas removal
A prerequisite to porting a Sugar process, component or activity is to remove all its usage of hippocanvas. This library is unmaintained, would be painful to port to GTK3, and can be done better with standard GTK+ code at the Python level. Most users of HippoCanvas can switch to custom GtkContainer widgets.
One complication is that hippo usage is not limited to Sugar's core; some activities use hippo, or they pull on sugar-toolkit classes which are implemented with hippo. These are: Chat (please list others here).
Backwards-compatibility for activities
As mentioned above, as mixing PyGTK/GTK2 and PyGI/GTK3 is impossible, a straightforward port of sugar-toolkit to PyGI/GTK3 would break all activities. However, having backwards compatibility for activities for a limited time is considered a requirement, and having a "flag day" where all activities stop working until ported is not seen as an option - there must be a transition period.
As the conversion process from PyGTK to PyGI makes minor changes to almost every single line of code that involves GTK/glib, maintaining both PyGI and PyGTK codepaths in the same files is not realistic (there would be an unrealistic amount of if conditions). We are therefore required to have two copies of sugar-toolkit during the transition period: one for ported PyGI/GTK3 activities, and another for PyGTK/GTK2 activities that have not yet been ported.
As we do not have plentiful developer resources, I propose that the PyGTK/GTK2 version of sugar-toolkit is frozen as soon as this feature development is underway. No bugfixes or improvements, just a copy of the code at that point.
Selecting a sugar-toolkit version
With two versions of sugar-toolkit available, activities need a mechanism for choosing which toolkit they will use. As we require existing activities to continue working without modifications, this selection mechanism must be opt-in - ported activities must somehow mark themselves as being GTK3-ported, and hence will receive the GTK3 version of sugar-toolkit, and all other activities will receive the GTK2 version.
One easy way to achieve this would be to give the GTK3 version of sugar-toolkit a different name. The unported GTK2 activities could continue to import sugar.foo and ported GTK3 activities would import sugar1.foo. "sugar1" is chosen as an example to indicate Sugar-1.0, but other naming schemes would obviously work equally well.
If the community has strongly desires it, it would be possible maintain the sugar module name through a Python trick which could be deployed during the transition period, and then removed once the GTK2 version is dropped. Here is how it could be done:
- The new GTK3 version of sugar toolkit would be installed with name sugar, and the old GTK2 version would be installed with name sugar_gtk2.
- Before sugar-activity imports sugar.activity (or any other sugar-toolkit class), it would look for an empty file called "GTK3_PORTED" in the activity's directory. If present, the GTK3_PORTED environment variable would be set.
- sugar/__init__.py would then include the following code snippet:
import os
import sys
if not 'GTK3_PORTED' in os.environ:
import sugar_gtk2
sys.modules["sugar"] = sugar_gtk2
- The result is that all unported/unmodified activities (without the GTK3_PORTED file) would import sugar.foo as before, but the above trick that modifies Python's module table would result in them (transparently, magically, automatically) being redirected to sugar_gtk2.foo.
- At the end of the transition period, sugar_gtk2 would be deleted in entirity, the above addition to sugar/__init__.py would be dropped, and activities could drop the GTK3_PORTED files at their leisure.
Proposed plan of action
The steps below prioritise the porting of sugar-toolkit, as this is where Sugar would see the most immediate benefit: the revival of Browse and Read. The steps that follow can largely be parallelised.
- Remove hippocanvas from sugar-toolkit
- Port sugar-toolkit to GTK3 while keeping backwards compatibility
- Rescue Browse and Read, and allow independent activity porting efforts to begin
- Remove hippocanvas from all other parts of Sugar, including activities
- Port Sugar core to hippocanvas
- Some time later, remove sugar-toolkit's GTK2 compatibility.
How to port
I plan to start a page at Features/GTK3/Porting that details the porting process, that could be provided as documentation to anyone involved in these efforts (including thsoe working on porting Sugar core, but also those working on activities). The content covered will be:
- How to remove hippo and what to replace it with (links to commits that have done this for other activities, etc)
- How to select the GTK3 version of sugar-toolkit
- How to handle each of the sugar-toolkit API changes (detailed in the following section)
- How to port from PyGTK/GTK2 to PyGI/GTK3.
- This basically involves reading the PyGObject porting documentation, using pygi-convert.sh, then checking the result.
API changes
The fact that almost all Sugar components and activities require sweeping changes as part of this shift presents an interesting opportunity for us to make API changes in sugar-toolkit. The importance here should still be placed on the technology shift, rather than on the opportunity to produce a perfect API (which we could spend all eternity designing and discussing), but this is an opportunity that we should not miss.
I propose that once sugar-toolkit is ported and mostly operational, we run a 30-day window where API changes that have seen some kind of planning and discussion below can be made and committed.
markup thing
http://article.gmane.org/gmane.linux.laptop.olpc.sugar/30466
removal of keep button
removal of old toolbars
Scope
What work do the developers have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?
How To Test
Dependencies
What other packages (RPMs) depend on this package? Are there changes outside the developers' control on which completion of this feature depends? In other words, does your feature depend on completion of another feature owned by someone else or that you would need to coordinate, which might cause you to be unable to finish on time? Other upstream projects like Python?
Contingency Plan
If you cannot complete your feature by the final development freeze, what is the backup plan? This might be as simple as "None necessary, revert to previous release behaviour." Or it might not. If your feature is not completed in time, we want to assure others that other parts of Sugar will not be in jeopardy.
Release Notes
The Sugar Release Notes inform end-users about what is new in the release. An Example is 0.84/Notes. The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns. If there are any such changes involved in this feature, indicate them here. You can also link to upstream documentation if it satisfies this need. This information forms the basis of the release notes edited by the release team and shipped with the release.