Features/GTK3
Summary
Sugar needs to rebase itself on new generations of its key underlying technologies: GTK+ 3 and PyGObject Introspection. This page aims to summarise options and community opinions around this challenging shift, and to help formulate a plan of how it will be executed. In other words, it tries to take community input, answer all the unanswered questions, and present a logical path forward that can be adopted by the developers.
Owner
- This plan/proposal maintained by Daniel Drake (other editors welcome!)
- A number of developers will be needed at each stage to successfully execute this; Daniel offers his assistance for a coordination/oversight role if that would be useful.
Current status
- Targeted release: (SUGAR_VERSION)
- Last updated: (DATE)
- Percentage of completion: XX%
Detailed Description
Two major changes have recently occurred in Sugar's underlying technologies. Firstly, GTK+ 2 has been obsoleted by GTK+ 3, and GNOME is now based on GTK+ 3. Secondly, PyGTK, the underlying Python library that Sugar uses to call into GTK+, has been deprecated in favour of PyGObject Introspection (hereafter "PyGI").
In order to remain innovative and current, and to take advantage of the latest developments, Sugar needs to follow suit and move to GTK3 and PyGI. Lagging behind on this conversion is already bringing negative consequences to Sugar; notably 2 of our most important activities (Read and Browse) are already broken and without a future until we catch up again.
Unfortunately, this is an API-incompatible change. As confirmed by the PyGI developers, PyGTK and PyGI cannot be mixed in the same process. This means that the conversion process can't be done on a fine, incremental basis, and if sugar-toolkit were to be simply replaced with a PyGI/GTK3 port, this means that all existing activities would stop working until they themselves have also been ported - all activities will need modifications as part of this feature. The community has expressed desire for old activities to continue working (many are unmaintained); unfortunately this is not realistic in the long term. As a compromise, this feature discussion includes the requirement that for a certain period of time, both PyGTK/GTK2 and PyGI/GTK3 activities must be able to function alongside each other.
This task will involve modifications to almost every file within Sugar and its activities. Although much of this work can be automated, and the porting process for a given chunk of code is usually trivial, it is still a huge task. This feature discussion attempts to identify ways that this porting process can be done in distinct stages, where Sugar remains functional at the completion of each stage, making this more manageable.
Benefit to Sugar
- PyGI is technologically better than PyGTK. It is a nicer way of calling into GObject-style libraries from Python that means less maintenance is needed upstream (PyGObject automates the creation of bindings to a degree much higher than PyGTK ever could, and automatically achieves more complete coverage).
- PyGTK is no longer maintained; PyGI is actively maintained.
- The move to GTK3 allows us to keep up with our GNOME neighbours, as they improve and refine the base technologies that we share.
- The move to PyGI is expected to result in lower memory usage and faster startup.
- Browse has no future under GTK2, it needs to move to WebKit and that move is dependent on Sugar moving to PyGI/GTK3.
- Similarly, Read has a grim future under GTK2 due to static evince bindings no longer being maintained and libevince itself moved to GTK3; we need to switch to PyGI/GTK3 to be able to keep calling into evince and let the Read activity live on.
Implementation Plan
Sugar is divided into different components, many of which run in different processes. This means that we are able to divide up the required work on a process-by-process basis. While this work is being conducted, some Sugar processes might be based on PyGI/GTK3 and others based on PyGTK/GTK2, but the platform would keep functioning at each stage. There would be some system resource overhead during this transitional time (as the system would need to have PyGTK, GTK2, PyGI and GTK3 all in memory) but this feature implementation would end with the whole sugar ecosystem using PyGI/GTK3.
HippoCanvas removal
A prerequisite to porting a Sugar process, component or activity is to remove all its usage of hippocanvas. This library is unmaintained, would be painful to port to GTK3, and can be done better with standard GTK+ code at the Python level. Most users of HippoCanvas can switch to custom GtkContainer widgets.
One complication is that hippo usage is not limited to Sugar's core; some activities use hippo, or they pull on sugar-toolkit classes which are implemented with hippo. These are: Chat (please list others here).
Backwards-compatibility for activities
As mentioned above, as mixing PyGTK/GTK2 and PyGI/GTK3 is impossible, a straightforward port of sugar-toolkit to PyGI/GTK3 would break all activities. However, having backwards compatibility for activities for a limited time is considered a requirement, and having a "flag day" where all activities stop working until ported is not seen as an option - there must be a transition period.
As the conversion process from PyGTK to PyGI makes minor changes to almost every single line of code that involves GTK/glib, maintaining both PyGI and PyGTK codepaths in the same files is not realistic (there would be an unrealistic amount of if conditions). We are therefore required to have two copies of sugar-toolkit during the transition period: one for ported PyGI/GTK3 activities, and another for PyGTK/GTK2 activities that have not yet been ported.
As we do not have plentiful developer resources, I propose that the PyGTK/GTK2 version of sugar-toolkit is frozen as soon as this feature development is underway. No bugfixes or improvements, just a copy of the code at that point.
Selecting a sugar-toolkit version
With two versions of sugar-toolkit available, activities need a mechanism for choosing which toolkit they will use. As we require existing activities to continue working without modifications, this selection mechanism must be opt-in - ported activities must somehow mark themselves as being GTK3-ported, and hence will receive the GTK3 version of sugar-toolkit, and all other activities will receive the GTK2 version.
One easy way to achieve this would be to give the GTK3 version of sugar-toolkit a different name. The unported GTK2 activities could continue to import sugar.foo and ported GTK3 activities would import sugar1.foo. "sugar1" is chosen as an example to indicate Sugar-1.0, but other naming schemes would obviously work equally well.
If the community has strongly desires it, it would be possible maintain the sugar module name through a Python trick which could be deployed during the transition period, and then removed once the GTK2 version is dropped. Here is how it could be done:
- The new GTK3 version of sugar toolkit would be installed with name sugar, and the old GTK2 version would be installed with name sugar_gtk2.
- Before sugar-activity imports sugar.activity (or any other sugar-toolkit class), it would look for an empty file called "GTK3_PORTED" in the activity's directory. If present, the GTK3_PORTED environment variable would be set.
- sugar/__init__.py would then include the following code snippet:
import os
import sys
if not 'GTK3_PORTED' in os.environ:
import sugar_gtk2
sys.modules["sugar"] = sugar_gtk2
- The result is that all unported/unmodified activities (without the GTK3_PORTED file) would import sugar.foo as before, but the above trick that modifies Python's module table would result in them (transparently, magically, automatically) being redirected to sugar_gtk2.foo.
- At the end of the transition period, sugar_gtk2 would be deleted in entirity, the above addition to sugar/__init__.py would be dropped, and activities could drop the GTK3_PORTED files at their leisure.
Proposed plan of action
The steps below prioritise the porting of sugar-toolkit, as this is where Sugar would see the most immediate benefit: the revival of Browse and Read. The steps that follow can largely be parallelised.
- Remove hippocanvas from sugar-toolkit
- Port sugar-toolkit to GTK3 while keeping backwards compatibility
- Rescue Browse and Read, and allow independent activity porting efforts to begin
- Remove hippocanvas from all other parts of Sugar, including activities
- Port Sugar core to hippocanvas
- Some time later, remove sugar-toolkit's GTK2 compatibility.
- I propose that this happens 1 year after release of the first sugar-toolkit GTK3 version
How to port
I plan to start a page at Features/GTK3/Porting that details the porting process, that could be provided as documentation to anyone involved in these efforts (including those working on porting Sugar core, but also those working on activities). The content covered will be:
- How to remove hippo and what to replace it with (links to commits that have done this for other activities, etc)
- How to select the GTK3 version of sugar-toolkit
- How to handle each of the sugar-toolkit API changes (detailed in the following section)
- How to port from PyGTK/GTK2 to PyGI/GTK3.
- This basically involves reading the PyGObject porting documentation, using pygi-convert.sh, then checking the result.
API changes
The fact that almost all Sugar components and activities require sweeping changes as part of this shift presents an interesting opportunity for us to make API changes in sugar-toolkit. The importance here should still be placed on the technology shift, rather than on the opportunity to produce a perfect API (which we could spend all eternity designing and discussing), but this is an opportunity that we should not miss.
I propose that once sugar-toolkit is ported and mostly operational, we run a 30-day window where API changes that have seen some kind of planning and discussion below can be made and committed.
Plain text as the default for palettes
Sugar's palette classes currently accept strings, but then they pass those strings to GTK as markup (always, unconditionally). However, markup is only used in a handful of places, and this means that all users of palettes that draw in translations or strings from other sources which might contain characters such as < or > must escape their arguments, leading to big patches like this.
As agreed here, it would make more sense for palettes to pass those strings as standard text by default, and to have different functions to opt-in to receive markup processing. Then the excessive escaping would go, and the only users would have to escape would be those who use markup.
Removal of keep button
Sugar-0.94 proposes the removal of the Keep button, but the old KeepButton classes would be kept around for activities that directly use it. Porting sugar-toolkit to GTK3 would be a good time to remove these classes.
Removal of old toolbars
Sugar-0.86 redesigned activity toolbars, with the new toolbars implemented by new classes, and the old classes being kept around so that old activities are not immediately broken. This would be a good opportunity to remove the long-deprecated old activity toolbar classes.
Dependencies
Sugar would start depending on PyGObject built with gobject introspection support, GTK3, and (at the end of the transition period) would drop its dependencies on GTK2 and PyGTK.