Features/Onscreen Keyboard
Summary
An on-screen keyboard for Sugar, targeted primarily at tablet like devices (which have no physical keyboards).
Owner
This should link to your home wiki page so we know who you are
- Name: Sayamindu
- Email: sayamindu@sugarlabs.org
Current status
- Targeted release: N/A
- Last updated: 06/29/2010
- Percentage of completion: 5%
Detailed Description
With OLPC announcing the XO-3 tablet, and with hardware like the JooJoo tablet, Sugar should become better suited for touch-based, physical keyboard-less devices. The on-screen keyboard feature aims to provide one such functionality. On-screen keyboards are also important for certain accessibility related contexts, but use cases like that are better handled by specialized software like Features/Accessibility_virtualkeyboard.
Software
While a number of FOSS on-screen keyboard software exists, the initial experiments are being carried out with fvkbd (Free Virtual KeyBoarD). The advantages of fvkbd include:
- Minimal external dependency outside the Sugar platform (only libfakekey is required as an extra)
- Actively maintained
- Internationalization friendly
- Has a library providing most of functionality which can be used later via Python bindings/introspection to achieve tighter integration with Sugar
Benefit to Sugar
A virtual/on-screen keyboard is probably the first component people think about when it comes to Sugar on a tablet device like the XO-3. It can be seen as a stepping stone for making Sugar more tablet-friendly and is essential for easily debugging and developing Sugar on a tablet machine.
Scope
- Work needed for fvkbd
- While the fvkbd binary is good enough for initial prototyping, for tighter integration in the future, it may be useful to utilize the libfvkbd library and implement the keyboard as a part of the sugar-shell itself.
- The current version of fvkbd handles all non-primary layouts as "temporary" layouts, which mean that once a key is clicked, the keyboard state goes back to the primary layout. This makes it unusable for multiple keyboard layouts. I have proposed a patch. The maintainer suggests an alternate way to do this, and I think his idea makes sense (though there is no patch for it yet).
- The number of layouts for fvkbd is quite low, and it is not possible to do a 1:1 transition from XKB to fvkbd layouts. I have made an experimental Spanish layout, along with a dual layout with English and Bengali (it requires a patched fvkbd, and the patch can be obtained from the bug mentioned above).
- Work needed in Sugar
- It is still not clear on how to best integrate the keyboard into Sugar. The most easy and obvious approach is using a device icon in the frame, but it would be useful to have the keyboard pop-out automatically when an editable widget has focus. See the discussion on GTK+ modules below.
- One major drawback of an on-screen keyboard is that in its current design, it blocks out a part of the Sugar UI. There's no immediate answer on how to handle this problem.
- Starting up the keyboard process is currently done in the prototypes using sugar-session. However, it may be useful to have this capability as a Sugar service, so that the keyboard can be invoked by activities as and when required.
- Work needed elsewhere
- GTK+ module to handle focus-in/out events
- A GTK+ module which would automatically invoke/close the keyboard when the user focuses into/out of an editable widget would probably be a good addition.However, while standard GTK+ widgets such as GtkTextView, etc can be easily identified as being editable, widgets such as those used by Write (Abiword) and Browse would be difficult to detect. Moreover, for activities like Etoys and Scratch, which do not use GTK+ at all, one would need to use some other approach.
- Code for a proof of concept GTK+ module which tries to detect focus-in events on common editable widgets can be downloaded. Usage and compilation instructions are in the comments.
- GTK+ module to handle focus-in/out events
UI Design
How To Test
User Experience
If this feature is noticeable by its target audience, how will their experiences change as a result? Describe what they will see or notice.
Dependencies
What other packages (RPMs) depend on this package? Are there changes outside the developers' control on which completion of this feature depends? In other words, does your feature depend on completion of another feature owned by someone else or that you would need to coordinate, which might cause you to be unable to finish on time? Other upstream projects like Python?
Contingency Plan
If you cannot complete your feature by the final development freeze, what is the backup plan? This might be as simple as "None necessary, revert to previous release behaviour." Or it might not. If your feature is not completed in time, we want to assure others that other parts of Sugar will not be in jeopardy.
Documentation
Is there upstream documentation on this feature, or notes you have written yourself? Has this topic been discussed in the mailing list or during a meeting? Link to that material here so other interested developers can get involved.
Release Notes
The Sugar Release Notes inform end-users about what is new in the release. An Example is 0.84/Notes. The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns. If there are any such changes involved in this feature, indicate them here. You can also link to upstream documentation if it satisfies this need. This information forms the basis of the release notes edited by the release team and shipped with the release.