Sugar Labs - User contributions [en]

Summer of Code/Interviews

2009-04-15T06:54:25Z

Gkovacs: Undo revision 27461 by Gkovacs (Talk) actually that time would indeed be fine

We would like to interview the candidates for Google Summer of Code under Sugar Labs. Please sign up for a time below. If you have been told the name of your provisional prospective mentor, and your mentor has put his/her preferred times below, please try to find a time which works for your mentor. (Note that mentor assignments are still in flux; if your provisional prospective mentor changes, we will try to accommodate your chosen time or reschedule at a convenient time for you.)

Interviews will be conducted over IRC; even hours on #sugar-interview0 and half-hours on #sugar-interview1 . Please treat these like real rooms, ie, do not lurk unless you are a mentor. If you have some quick pre-interview questions, we will be in #sugar too.

If you have specifically been told to sign up to interview, please sign up here. Even if your provisional prospective mentor has not indicated availability, put your available times, and your mentor will renegotiate with you later if necessary. We have invited the 9-10 proposals which are currently ranked highest, for the 4-6 slots we will probably get. However, we ''do'' want to interview you even if you have not specifically been invited; although your proposal is a long shot in that case, you may impress us, and in any case we'd like to talk to you about participating in Sugar outside of GSoC.

Like many other organizations participating in GSoC, we have received more good applications than we will have funding from Google for. Almost all of the applications we've received are things that we'd really love to have you working on, even if we don't have space for you in GSoC. If you are interested in finishing your project and would like a mentor assigned even though we can't give you a slot, you can also contact us on the sugar-devel or gsoc mailing list and we'll work something out.

== Sunday Apr 1 ==

Mentors attending: Guido van Rossum, Linus Torvalds, Bill Gates ...

* [http://www.timeanddate.com/counters/customcounter.html?day=1&month=04&year=2009&hour=14&min=0&sec=0&p0=0 1400 UTC]: [[User:example|example]] 18:11, 13 April 2009 (UTC) (Spanish interview preferred) (just an example - I used <nowiki>~~~~</nowiki> to add a signature, and added any special requests after.)

== Wed Apr 15 ==

Mentors attending: Jameson Quinn, Luis G. Lira, [[User:Aa|aa]], [[User:Tomeu|Tomeu]], [[User:Bobbyp|Bobby]] ...

* [http://www.timeanddate.com/counters/customcounter.html?day=15&month=04&year=2009&hour=14&min=0&sec=0&p0=0 1400 UTC]: Vamsi Krishna Davuluri (iwikiwi)

* 1430 UTC: Eduardo Silva (edsiper)

* 1500 UTC: Lucian Branescu (lucian1900)

* 1530 UTC: Deepank Gupta (deepank)

== Thurs Apr 16 (Wed 15 in the western hemisphere) ==

Mentors attending: Jameson Quinn, [[User:Aa|aa]], [[User:BryanWB|BryanWB]], [[User:Nrp|Nirav Patel]], [[User:Bobbyp|Bobby]] ...

* [http://www.timeanddate.com/counters/customcounter.html?day=16&month=04&year=2009&hour=0&min=0&sec=0&p0=0 0000 UTC]:
* 0030 UTC:

* 0100 UTC:Felipe López Toledo (subzero) ?Can we move this to 0100 UTC? I usually just wake up at this time. [[User:BryanWB|BryanWB]]- sure (subzero).

* 0130 UTC: [http://wiki.sugarlabs.org/go/Summer_of_Code/2009/Multimedia-broadcasting Geza Kovacs (gkovacs)]

== Thurs Apr 16 ==

Mentors attending: Jameson Quinn, Sayamindu Dasgupta, Assim Deodia, [[User:Tomeu|Tomeu]], ...

* [http://www.timeanddate.com/counters/customcounter.html?day=16&month=04&year=2009&hour=18&min=0&sec=0&p0=0 1800 UTC]: Satya Komaragiri (mavu)

* 1830 UTC: [http://wiki.sugarlabs.org/go/speech-synthesis chirag jain (chirag)]

* 1900 UTC: [http://wiki.sugarlabs.org/go/Marbles Puneet_Girdhar]

* 1930 UTC: Kartik Rustagi

Summer of Code/Interviews

2009-04-15T06:51:52Z

Gkovacs: misread time, sorry

We would like to interview the candidates for Google Summer of Code under Sugar Labs. Please sign up for a time below. If you have been told the name of your provisional prospective mentor, and your mentor has put his/her preferred times below, please try to find a time which works for your mentor. (Note that mentor assignments are still in flux; if your provisional prospective mentor changes, we will try to accommodate your chosen time or reschedule at a convenient time for you.)

Interviews will be conducted over IRC; even hours on #sugar-interview0 and half-hours on #sugar-interview1 . Please treat these like real rooms, ie, do not lurk unless you are a mentor. If you have some quick pre-interview questions, we will be in #sugar too.

If you have specifically been told to sign up to interview, please sign up here. Even if your provisional prospective mentor has not indicated availability, put your available times, and your mentor will renegotiate with you later if necessary. We have invited the 9-10 proposals which are currently ranked highest, for the 4-6 slots we will probably get. However, we ''do'' want to interview you even if you have not specifically been invited; although your proposal is a long shot in that case, you may impress us, and in any case we'd like to talk to you about participating in Sugar outside of GSoC.

Like many other organizations participating in GSoC, we have received more good applications than we will have funding from Google for. Almost all of the applications we've received are things that we'd really love to have you working on, even if we don't have space for you in GSoC. If you are interested in finishing your project and would like a mentor assigned even though we can't give you a slot, you can also contact us on the sugar-devel or gsoc mailing list and we'll work something out.

== Sunday Apr 1 ==

Mentors attending: Guido van Rossum, Linus Torvalds, Bill Gates ...

* [http://www.timeanddate.com/counters/customcounter.html?day=1&month=04&year=2009&hour=14&min=0&sec=0&p0=0 1400 UTC]: [[User:example|example]] 18:11, 13 April 2009 (UTC) (Spanish interview preferred) (just an example - I used <nowiki>~~~~</nowiki> to add a signature, and added any special requests after.)

== Wed Apr 15 ==

Mentors attending: Jameson Quinn, Luis G. Lira, [[User:Aa|aa]], [[User:Tomeu|Tomeu]], [[User:Bobbyp|Bobby]] ...

* [http://www.timeanddate.com/counters/customcounter.html?day=15&month=04&year=2009&hour=14&min=0&sec=0&p0=0 1400 UTC]: Vamsi Krishna Davuluri (iwikiwi)

* 1430 UTC: Eduardo Silva (edsiper)

* 1500 UTC: Lucian Branescu (lucian1900)

* 1530 UTC: Deepank Gupta (deepank)

== Thurs Apr 16 (Wed 15 in the western hemisphere) ==

Mentors attending: Jameson Quinn, [[User:Aa|aa]], [[User:BryanWB|BryanWB]], [[User:Nrp|Nirav Patel]], [[User:Bobbyp|Bobby]] ...

* [http://www.timeanddate.com/counters/customcounter.html?day=16&month=04&year=2009&hour=0&min=0&sec=0&p0=0 0000 UTC]:
* 0030 UTC:

* 0100 UTC:Felipe López Toledo (subzero) ?Can we move this to 0100 UTC? I usually just wake up at this time. [[User:BryanWB|BryanWB]]- sure (subzero).

* 0130 UTC:

== Thurs Apr 16 ==

Mentors attending: Jameson Quinn, Sayamindu Dasgupta, Assim Deodia, [[User:Tomeu|Tomeu]], ...

* [http://www.timeanddate.com/counters/customcounter.html?day=16&month=04&year=2009&hour=18&min=0&sec=0&p0=0 1800 UTC]: Satya Komaragiri (mavu)

* 1830 UTC: [http://wiki.sugarlabs.org/go/speech-synthesis chirag jain (chirag)]

* 1900 UTC: [http://wiki.sugarlabs.org/go/Marbles Puneet_Girdhar]

* 1930 UTC: Kartik Rustagi

Summer of Code/Interviews

2009-04-15T06:46:42Z

Gkovacs:

Summer of Code/2009/Multimedia-broadcasting

2009-04-12T01:41:34Z

Gkovacs: moved back personal information to match format of the rest of the proposals

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia broadcasting to students on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users (primarily teachers or students presenting material in clas) to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers (the rest of the classroom). This would be of use for various other purposes as well, such as for displaying lecture slides. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software; rather it aims to displace the usage of expensive projectors for displaying lecture slides and multimedia. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information, whether it be a video feed from a presentation or just lecture slides, to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience and cost; it eliminates the need for an expensive projector to display lecture slides, and if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

Since Icecast lacks support for multicast broadcasting, another FOSS streaming server such as Helix Server or VLC may need to be used; alternatively Farsight's RTP multicast may also be of use; I will investigate this prior to the start of GSoC.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

Alternatively, bandwidth usage when broadcasting to a large classroom with unicast could also be limited by broadcasting a low-framerate stream based on the number of users. While this would lead to a clear degradation in quality in the case of full-motion video, this is a rather niche medium to be shown in class; rather, lecture slides, which are what projectors are usually used for, would appear perfectly with a low-framerate stream.

The usage of UDP multicast broadcasting will also help conserve bandwidth and prevent lag in the video, thus as mentioned in "Architecture" if a streaming server such as Helix Server is used instead then this can be taken advantage of. However the combined usage of multicast and unicast streams on a single access point introduces many performance issues, so this may not be realistic in the context of broadcasting full-motion video (though for the primary target usage, lecture slides, which requires considerably less bandwidth to broadcast, there shouldn't performance issues with either unicast or multicast).

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. Capturing X11 desktop output would additionally allow for presentation of lecture slides via this system. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

'''Contingency Plan'''

Discussions on the mailing list have made it clear that full-motion video broadcasting over a wireless network to a large classroom, either via unicast or multicast, is going to be a very difficult technical challenge. Potential solutions to this are listed under "Addressing Bandwidth Issues". Should all of this nevertheless run into technical issues that prevent large-scale broadcasting of full-motion video due to bandwidth and airtime issues, this activity could still be a useful means to broadcast lecture slides, which require much lower framerates and bandwidth and are thus less likely to run into bandwidth issues, or perhaps for small-scale audio and video broadcasting (aka one-to-one video conferencing).

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I have created successful (2 million+ downloads) FOSS projects in the past, with similar intense development timelines of a few weeks (please see details on the two major FOSS projects I created, [http://wubi-installer.org/ Wubi] and [http://unetbootin.sourceforge.net/ UNetbootin], described above, as well as my [http://launchpad.net/~gezakovacs Launchpad] profile).

With respect to experience this field of multimedia broadcasting I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast multimedia such as lecture slides, live lab demonstrations, live demonstrations on software usage, or other multimedia. The potential for this project to reduce teachers' dependence on projectors, and even replace the need for expensive projectors, would also make it attractive financially for any budget-limited schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-12T01:29:45Z

Gkovacs: added contingency plan

{{TOCright}}

====About you====

Personal information has been moved to [[User:Gkovacs]]

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia broadcasting to students on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users (primarily teachers or students presenting material in clas) to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers (the rest of the classroom). This would be of use for various other purposes as well, such as for displaying lecture slides. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software; rather it aims to displace the usage of expensive projectors for displaying lecture slides and multimedia. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information, whether it be a video feed from a presentation or just lecture slides, to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience and cost; it eliminates the need for an expensive projector to display lecture slides, and if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

Since Icecast lacks support for multicast broadcasting, another FOSS streaming server such as Helix Server or VLC may need to be used; alternatively Farsight's RTP multicast may also be of use; I will investigate this prior to the start of GSoC.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

Alternatively, bandwidth usage when broadcasting to a large classroom with unicast could also be limited by broadcasting a low-framerate stream based on the number of users. While this would lead to a clear degradation in quality in the case of full-motion video, this is a rather niche medium to be shown in class; rather, lecture slides, which are what projectors are usually used for, would appear perfectly with a low-framerate stream.

The usage of UDP multicast broadcasting will also help conserve bandwidth and prevent lag in the video, thus as mentioned in "Architecture" if a streaming server such as Helix Server is used instead then this can be taken advantage of. However the combined usage of multicast and unicast streams on a single access point introduces many performance issues, so this may not be realistic in the context of broadcasting full-motion video (though for the primary target usage, lecture slides, which requires considerably less bandwidth to broadcast, there shouldn't performance issues with either unicast or multicast).

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. Capturing X11 desktop output would additionally allow for presentation of lecture slides via this system. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

'''Contingency Plan'''

Discussions on the mailing list have made it clear that full-motion video broadcasting over a wireless network to a large classroom, either via unicast or multicast, is going to be a very difficult technical challenge. Potential solutions to this are listed under "Addressing Bandwidth Issues". Should all of this nevertheless run into technical issues that prevent large-scale broadcasting of full-motion video due to bandwidth and airtime issues, this activity could still be a useful means to broadcast lecture slides, which require much lower framerates and bandwidth and are thus less likely to run into bandwidth issues, or perhaps for small-scale audio and video broadcasting (aka one-to-one video conferencing).

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I have created successful (2 million+ downloads) FOSS projects in the past, with similar intense development timelines of a few weeks (please see details on the two major FOSS projects I created, [http://wubi-installer.org/ Wubi] and [http://unetbootin.sourceforge.net/ UNetbootin], described above, as well as my [http://launchpad.net/~gezakovacs Launchpad] profile).

With respect to experience this field of multimedia broadcasting I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast multimedia such as lecture slides, live lab demonstrations, live demonstrations on software usage, or other multimedia. The potential for this project to reduce teachers' dependence on projectors, and even replace the need for expensive projectors, would also make it attractive financially for any budget-limited schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-12T01:11:32Z

Gkovacs:

{{TOCright}}

====About you====

Personal information has been moved to [[User:Gkovacs]]

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia broadcasting to students on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users (primarily teachers or students presenting material in clas) to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers (the rest of the classroom). This would be of use for various other purposes as well, such as for displaying lecture slides. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software; rather it aims to displace the usage of expensive projectors for displaying lecture slides and multimedia. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information, whether it be a video feed from a presentation or just lecture slides, to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience and cost; it eliminates the need for an expensive projector to display lecture slides, and if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

Since Icecast lacks support for multicast broadcasting, another FOSS streaming server such as Helix Server or VLC may need to be used; alternatively Farsight's RTP multicast may also be of use; I will investigate this prior to the start of GSoC.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

Alternatively, bandwidth usage when broadcasting to a large classroom with unicast could also be limited by broadcasting a low-framerate stream based on the number of users. While this would lead to a clear degradation in quality in the case of full-motion video, this is a rather niche medium to be shown in class; rather, lecture slides, which are what projectors are usually used for, would appear perfectly with a low-framerate stream.

The usage of UDP multicast broadcasting will also help conserve bandwidth and prevent lag in the video, thus as mentioned in "Architecture" if a streaming server such as Helix Server is used instead then this can be taken advantage of. However the combined usage of multicast and unicast streams on a single access point introduces many performance issues, so this may not be realistic in the context of broadcasting full-motion video (though for the primary target usage, lecture slides, which requires considerably less bandwidth to broadcast, there shouldn't performance issues with either unicast or multicast).

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. Capturing X11 desktop output would additionally allow for presentation of lecture slides via this system. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I have created successful (2 million+ downloads) FOSS projects in the past, with similar intense development timelines of a few weeks (please see details on the two major FOSS projects I created, [http://wubi-installer.org/ Wubi] and [http://unetbootin.sourceforge.net/ UNetbootin], described above, as well as my [http://launchpad.net/~gezakovacs Launchpad] profile).

With respect to experience this field of multimedia broadcasting I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast multimedia such as lecture slides, live lab demonstrations, live demonstrations on software usage, or other multimedia. The potential for this project to reduce teachers' dependence on projectors, and even replace the need for expensive projectors, would also make it attractive financially for any budget-limited schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-12T01:08:08Z

Gkovacs: noting that this can be used for low-bandwidth lecture slides as well

{{TOCright}}

====About you====

Personal information has been moved to [[User:Gkovacs]]

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia broadcasting to students on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users (primarily teachers or students presenting material in clas) to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers (the rest of the classroom). This would be of use for various other purposes as well, such as for displaying lecture slides. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software; rather it aims to displace the usage of expensive projectors for displaying lecture slides and multimedia. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information, whether it be a video feed from a presentation or just lecture slides, to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience and cost; it eliminates the need for an expensive projector to display lecture slides, and if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

Since Icecast lacks support for multicast broadcasting, another FOSS streaming server such as Helix Server or VLC may need to be used; alternatively Farsight's RTP multicast may also be of use; I will investigate this prior to the start of GSoC.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

Alternatively, bandwidth usage when broadcasting to a large classroom with unicast could also be limited by broadcasting a low-framerate stream based on the number of users. While this would lead to a clear degradation in quality in the case of full-motion video, this is a rather niche medium to be shown in class; rather, lecture slides, which are what projectors are usually used for, would appear perfectly with a low-framerate stream.

The usage of UDP multicast broadcasting will also help conserve bandwidth and prevent lag in the video, thus as mentioned in "Architecture" if a streaming server such as Helix Server is used instead then this can be taken advantage of. However the combined usage of multicast and unicast streams on a single access point introduces many performance issues, so this may not be realistic in the context of broadcasting full-motion video (though for the primary target usage, lecture slides, which requires considerably less bandwidth to broadcast, there shouldn't performance issues with either unicast or multicast).

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. Capturing X11 desktop output would additionally allow for presentation of lecture slides via this system. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I have created successful (2 million+ downloads) FOSS projects in the past, with similar intense development timelines of a few weeks (please see details on the two major FOSS projects I created, [http://wubi-installer.org/ Wubi] and [http://unetbootin.sourceforge.net/ UNetbootin], described above, as well as my [http://launchpad.net/~gezakovacs Launchpad] profile).

With respect to experience this field of multimedia broadcasting I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-10T16:10:33Z

Gkovacs: farsight may work as well

{{TOCright}}

====About you====

Personal information has been moved to [[User:Gkovacs]]

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

Since Icecast lacks support for multicast broadcasting, another FOSS streaming server such as Helix Server or VLC may need to be used; alternatively Farsight's RTP multicast may also be of use; I will investigate this prior to the start of GSoC.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

The usage of UDP multicast broadcasting will also help conserve bandwidth and prevent lag in the video, thus as mentioned in "Architecture" if a streaming server such as Helix Server is used instead then this can be taken advantage of.

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-10T15:54:56Z

Gkovacs: may need to use Helix or VLC as streaming server instead due to Icecast's lack of multicast support

{{TOCright}}

====About you====

Personal information has been moved to [[User:Gkovacs]]

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

Since Icecast lacks support for multicast broadcasting, another FOSS streaming server such as Helix Server or VLC may need to be used; I will investigate this prior to the start of GSoC.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

The usage of UDP multicast broadcasting will also help conserve bandwidth and prevent lag in the video, thus as mentioned in "Architecture" if a streaming server such as Helix Server is used instead then this can be taken advantage of.

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

User:Gkovacs

2009-04-06T22:28:19Z

Gkovacs:

===Projects===

[[Summer_of_Code/2009/Multimedia-broadcasting]]

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

Summer of Code/2009/Multimedia-broadcasting

2009-04-06T22:27:34Z

Gkovacs: moved personal info

{{TOCright}}

====About you====

Personal information has been moved to [[User:Gkovacs]]

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

User:Gkovacs

2009-04-06T22:27:21Z

Gkovacs: created page including

Multimedia-broadcasting

2009-04-06T22:25:01Z

Gkovacs: Multimedia-broadcasting moved to Summer of Code/2009/Multimedia-broadcasting

#REDIRECT [[Summer of Code/2009/Multimedia-broadcasting]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-06T22:25:01Z

Gkovacs: Multimedia-broadcasting moved to Summer of Code/2009/Multimedia-broadcasting

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-04T16:30:45Z

Gkovacs:

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Broadcasting Interface'''

Upon launching the activity, the user will be presented with a "Broadcast" button. Upon clicking this, the user will be presented with a list of possible video sources the user can select - webcam or screen. The user will also be given the option to broadcast an audio source - in this case a microphone. The user will also be shown a map of laptops on the local neighborhood to which the stream can be broadcast and the associated laptop's user's name. and the user may either individually check laptops to broadcast the stream to, or press a "broadcast to all" button which would broadcast to all the local laptops, or if bandwidth issues would be encountered due to the presence of too many laptops, the situation would be handled as mentioned in the next section. After selecting multimedia sources and broadcasting targets, the user would press a "Start Broadcasting" button, and the laptop would display the stream that is currently being broadcast, and a "stop broadcasting" button to end the broadcast.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

'''Viewing Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-04T16:12:42Z

Gkovacs:

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Addressing Bandwidth Issues'''

For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-03T11:12:06Z

Gkovacs:

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

At this time I not certain of whether I will be spending my summer in Cambridge, MA, or Garden Grove, CA. I believe, however, that I will likely be at the former location. I believe there are already some deployments nearby that I can visit, and have no particular preference for the other questions.

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-03T11:06:15Z

Gkovacs:

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. This would benefit Sugar Labs by providing more interested pilot schools. The potential for this project to reduce teachers' dependence on projectors, and arguably even replace them in certain contexts, would also make it an attractive feature for any schools considering participation in the pilot program.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

The first time I entered the local public library with the intention of checking out books for personal reading enjoyment, I went through the fiction section and checked out many well-known fantasy novels, then spent the next 3 weeks reading through them, probably doing more reading than I had ever done before. Though I didn't learn much factual information, the experience instilled in me a passion for reading.

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-03T10:56:51Z

Gkovacs:

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server. The Icecast server does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop; it would merely need to advertise its service via mDNS. Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult. By utilizing an external Icecast server to handle the video broadcasting itself, even a resource-constrained machine such as an XO-1 should be able to broadcast video, albeit encoded with less cpu-intensive codecs and settings to compensate for the lack of cpu power on such machines.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. This would benefit Sugar Labs by providing more interested pilot schools.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-03T10:20:56Z

Gkovacs:

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. This would benefit Sugar Labs by providing more interested pilot schools.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:Gkovacs-reboot-task.png|thumb|right|Replaced "Reboot" with "gkovacs@mit.edu".]]

'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

File:Gkovacs-reboot-task.png

2009-04-03T10:18:50Z

Gkovacs:

Summer of Code/2009/Multimedia-broadcasting

2009-04-03T09:50:13Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.

To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed. If the stream is being cached to the disk, the student will also have an option to pause and rewind the video stream.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video stream; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

'''If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.'''

Answer from me:

The completion of this project would give teachers more dedicated to traditional, presentation-based curriculum a new means to broadcast their information. This would benefit Sugar Labs by providing more interested pilot schools.

'''Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?'''

'''What will you do if you get stuck on your project and your mentor isn't around?'''

I would utilize the mailing lists to get help from others. Given that the bulk of this project consists of several mostly-independent backend components, I could also fix issues with the other backends, or work on writing documentation and cleaning code up while awaiting help.

'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

For documenting general progress and milestones achieved, I will be using a page on the Sugar Wiki which can be watched by any interested members. For any problems or questions I encounter, I will be sending email to the appropriate mailing lists, so those subscribed to them would be informed about it.

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
'''What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)'''

L

'''Describe a great learning experience you had as a child.'''

'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-03T09:32:45Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

The interface will also include options to save streams to the journal, from which they could later be viewed.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented. For the sake of cpu usage efficiency on less powerful machines like XOs, annotations will not be encoded as part of the video strema; they will instead be transmitted seperately from the video file, much like external subtitles, in a format using a simple coordinate system (time, x, y, color) and would be overlayed on top of the video via following decoding.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-03T09:24:05Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used. As XOs do not apparently have enough cpu power to encode Theora video live at acceptable resolution and framerates, older, less cpu-intensive codecs, such as MPEG-1, MJPEG, or H.261 may be used as video codecs instead.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T22:39:11Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T22:37:58Z

Gkovacs: Undo revision 24927 by Gkovacs (Talk)

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T22:37:26Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

Q: '''What is your IRC nickname?'''

A: gkovacs

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

----

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T22:36:25Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used.

----

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T22:35:29Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

'''Audio and Video Sources'''

For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The GStreamer libraries will be used for communicating with the Icecast server and uploading multimedia. Ogg will probably be used as the container, Theora will be used as the video codec, and Vorbis will be used as the audio codec, since this is a relatively well-supported combination (supported for in-browser playback on upcoming browsers) and poses no licensing issues, though thanks to the flexibility of GStreamer any other supported codecs and container could alternatively be used.

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

'''Prior to the start of the coding period'''

Review the GStreamer and Sugar APIs.

'''Week 1'''

Write a GStreamer-based webcam video capture backend and X11 desktop video capture backends.

'''Week 2'''

Write the microphone audio capture backend, and the Icecast server uploading code.

'''Week 3'''

Write the server-side authentication code and security mechanisms.

'''Milestone 1 Reached'''

'''Week 4'''

This week is set aside for finishing up work from weeks 1-3 if needed, soliciting and addressing any feedback regarding the backend, documenting the backend code on the wiki, and performing testing and bug-fixing of the backend with a rudimentary, very simple client-side frontend.

'''Week 5'''

Create the client-side frontend, as described in "Interface".

'''Week 6'''

Finish work on the client-side frontend, and integrate options to broadcast multimedia into the Neighborhood view.

'''Week 7'''

Finish any work on the client-side frontend that has yet to be finished, perform a second cycle of feedback gathering, and testing, then write client-side documentation on the wiki.

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.

Regarding development pace, I will have no other commitments this summer so I will be able to spend my full time on development.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T21:54:49Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

'''Rationale'''

This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, the client will automatically send a request, authentication and identification details, to the classroom's Icecast server (which does not necessarily have to be a dedicated server; the software could, for example, be running on the teacher's laptop). Once confirmed, the stream is then uploaded and streamed to the local Icecast server via GStreamer, creating a unique URL for the stream. When the user invites others, or perhaps the entire class, to view the stream, then it would send the URL of the stream to the target user, and the peer's identification details. This would be done through the central server to increase security by making spoofing more difficult.

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

'''Video Sources'''

Webcam
X11 desktop output
Icecast will be used for streaming video sources to the laptop.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented.

'''Potential Enhancements'''

These enhancements are probably beyond the scope of a single GSoC project. However, since this architecture has the potential to replace the standard projector-based lecture presentation mechanism, perhaps other, non-video formats, namely PDF lecture slides, might be supported as a future enhancement as well.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The Telepathy and Farsight2 libraries will be used for implementing the multimedia chat feature.

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

A: Week 1:
Week 2:
Week 3:
Week 4:
Week 5:
Week 6:
Week 7:

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T21:48:55Z

Gkovacs:

Please apply also in [http://socghop.appspot.com melange, google's web app]; if you do not apply there '''before''' April 3, we will not be able to accept your application.

Please keep an eye on your talk page (the "discussion" link above). If you hit "watch" above, you can set up your "prefs" to email you the first time it changes since you last saw it.

{{TOCright}}

====About you====

Q: '''What is your name?'''

A: Geza Kovacs

----

Q: '''What is your email address?'''

A: gkovacs -at- mit -dot- edu

----

Q: '''What is your Sugar Labs wiki username?'''

A: gkovacs

----

Q: '''What is your IRC nickname?'''

A: gkovacs

----

Q: '''What is your primary language? (We have mentors who speak multiple languages and can match you with one of them if you'd prefer.)'''

A: English

----

Q: '''Where are you located, and what hours do you tend to work? (We also try to match mentors by general time zone if possible.)'''

A: USA, either Pacific or Eastern time zones. I tend to work anytime between 8AM to midnight.

----

Q: '''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

A: My two most successful open-source projects to date (with over 2 million downloads apiece) are Wubi and UNetbootin, both of which I launched during my high school years. I have also worked on some other minor projects for which I have open-sourced code, most of which can be found around my launchpad page if they are of particular interest; however the major projects I am currently working on as part of undergraduate research (mostly related to audio and video analysis in the context of emotion recognition based on facial and speech features) are unfortunately currently proprietary (but we expect to open-source it in May).

http://wubi-installer.org/ Designed, initially led development of, and created the prototypes and early versions of the Windows-based Ubuntu Installer, now part of Ubuntu 8.04, which allows Windows users to safely install Ubuntu Linux without repartitioning their hard drives.

http://unetbootin.sourceforge.net/ Creator, Lead Developer, and Maintainer of UNetbootin, a cross-platform utility to perform network installations or create bootable USB flash drives for a wide variety of Linux distributions.

Further details about of my FOSS development activities (code repository, bug reports, specs) can be found on http://launchpad.net/~gezakovacs

====About your project====

Q: '''What is the name of your project?

A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network

----

Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.

Rationale: This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:

As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received. This also has the advantage of

'''Architecture'''

This will utilize an architecture involving multiple peers and a single local Icecast server, all communicating over the local wireless network/mesh. When a user intends to broadcast a new video, . The Icecast server can optionally also record

'''Accepting New Streams'''

There is an obvious security risk of unsolicited streams appearing on students' laptops, especially if the broadcasting occurs from an online source (though this GSoC project does not aim to address online sources, since most in-class activity originates locally anyhow). It is also crucial to make the display of new streams from teachers or peers as easy as possible. Thus, an identification scheme (PGP keys? OpenID?) should be available to uniquely identify users on the network and their role as either peers or teachers. For notification, there can either be a background service which will monitor the port designated for announcing new streams and display a notification, or this monitoring functionality could be built into the activity itself, though this would require that students have the activity open when the stream broadcast is announced.

For locally-originating sources (within a classroom), a confirmation dialog will display the name of the broadcaster, and if confirmed, will launch the multimedia broadcasting/receiving activity if it has not already been launched, and will display the stream. For higher-priority sources, such as teachers in that particular classroom, the stream could be launched without confirmation to ensure that students do not accidentally miss the confirmation notice.

'''Interface'''

Since students may be interested in viewing multiple streams at once (for example, one stream showing the overall demonstration of the experiment and one showing a magnified view of a particular bacteria culture), then the interface should be able to automatically resize and tile individual streams to display them all at once while best utilizing screen space.

'''Video Sources'''

Webcam
X11 desktop output
Icecast will be used for streaming video sources to the laptop.

'''Video Interaction'''

There should be some equivalent of a "pointer" on the video broadcasting-side, for highlighting important information. There might also be a "pointer" for pointing out comments on the receiver's side, though this would probably cause more trouble and distraction in the classroom rather than promote useful feedback to the presenter, thus this probably won't be implemented.

'''Potential Enhancements'''

These enhancements are probably beyond the scope of a single GSoC project. However, since this architecture has the potential to replace the standard projector-based lecture presentation mechanism, perhaps other, non-video formats, namely PDF lecture slides, might be supported as a future enhancement as well.

'''Programming Languages and Libraries'''

Since sugar-chat-activity uses Python, I will likely be using it for this project as well. The Telepathy and Farsight2 libraries will be used for implementing the multimedia chat feature.

Q: '''What is the timeline for development of your project? The Summer of Code work period is 7 weeks long, May 23 - August 10; tell us what you will be working on each week. (As the summer goes on, you and your mentor will adjust your schedule, but it's good to have a plan at the beginning so you have an idea of where you're headed.) Note that you should probably plan to have something "working and 90% done" by the midterm evaluation (July 6-13); the last steps always take longer than you think, and we will consider cancelling projects which are not mostly working by then.'''

A: Week 1:
Week 2:
Week 3:
Week 4:
Week 5:
Week 6:
Week 7:

----

Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''

I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde

In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.

====You and the community====

# If your project is successfully completed, what will its impact be on the Sugar Labs community? Give 3 answers, each 1-3 paragraphs in length. The first one should be yours. The other two should be answers from members of the Sugar Labs community, at least one of whom should be a Sugar Labs GSoC mentor. Provide email contact information for non-GSoC mentors.
# Sugar Labs will be working to set up a small (5-30 unit) Sugar pilot near each student project that is accepted to GSoC so that you can immediately see how your work affects children in a deployment. We will make arrangements to either supply or find all the equipment needed. Do you have any ideas on where you would like your deployment to be, who you would like to be involved, and how we can help you and the community in your area begin it?
# What will you do if you get stuck on your project and your mentor isn't around?
# How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

====Miscellaneous====
[[Image:New-developer-challenge.png|thumb|right|An example of the kind of screenshot of your first modification to your development environment which you should include in your application. Note that the drop-down menu text has Mel's email address in place of the word "Restart" - your screenshot should contain your email instead.]]
# We want to make sure that you can set up a [[DevelopmentTeam#Development_systems|development environment]] before the summer starts. Please send us a link to a screenshot of your Sugar development environment with the following modification: when you hover over the XO-person icon in the middle of Home view, the drop-down text should have your email in place of "Restart." See the image on the right for an example. It's normal to need assistance with this, so please visit our IRC channel, #sugar on irc.freenode.net, and ask for help.
# What is your t-shirt size? (Yes, we know Google asks for this already; humor us.)
# Describe a great learning experience you had as a child.
# Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?

Note: you will post this application on the wiki in the category [[:Category:2009 GSoC applications]]. We encourage you to browse this category and comment on the talk page of other applications. Also, others' comments and your responses on the talk page of your own application are viewed favorably, and, while we don't like repetitive spam, we welcome honest questions and discussion of your project idea on the mailing list(s) (primarily [http://lists.sugarlabs.org/listinfo/sugar-devel sugar-devel] for technical issues and [http://lists.sugarlabs.org/listinfo/iaep It's An Education Project] for educational issues) and/or [[IRC]].

The NeL project has some good general recommendations for [http://dev.ryzom.com/projects/nel/wiki/GSoC2009WritingProposals writing proposals]. We endorse them all; although Sugar is (regrettably) not test driven development (yet - your project could change that!), we encourage GSoC code to include tests.

[[Category:2009 GSoC applications]]

[[Category:GSoC]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T18:03:39Z

Gkovacs:

Local-multimedia-chat

2009-04-02T17:52:37Z

Gkovacs: Local-multimedia-chat moved to Multimedia-broadcasting

#REDIRECT [[Multimedia-broadcasting]]

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T17:52:37Z

Gkovacs: Local-multimedia-chat moved to Multimedia-broadcasting

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T17:52:04Z

Gkovacs: changing description to focus more on broadcasting aspect

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T06:19:04Z

Gkovacs:

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T05:49:44Z

Gkovacs: add to applications category

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T05:42:18Z

Gkovacs:

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T05:41:35Z

Gkovacs: formatting

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T05:26:41Z

Gkovacs: Local-multimedia-chat moved to Summer of Code/2009/Local-multimedia-chat: moving to summer of code 2009 subdir

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T05:23:06Z

Gkovacs: personal information

Summer of Code/2009/Multimedia-broadcasting

2009-04-02T05:08:00Z

Gkovacs: copied out application template

Sugar on a Stick Ubuntu

2009-02-06T02:46:31Z

Gkovacs: aesthetic fixes

''Note: this procedure works for these circumstances'' Taken from http://dev.laptop.org/~probono/sbuntu/

''* Ubuntu 8.10 Intrepid Ibex''
''* Built November 29, 2008''
''* Browse activity works''

= How to make a Ubuntu 8.10 Sugar Live USB system =

# Download the stock [http://releases.ubuntu.com/intrepid/ubuntu-8.10-desktop-i386.iso ubuntu-8.10-desktop-i386.iso] and burn it
# Create a Live USB system using the built-in Live USB installer (In Ubuntu 8.10 and later: System --> Administration --> Create a USB startup disk). If you choose "Stored in reserved extra space", do '''NOT''' use up the entire available space, but leave at least 100 MB so that we can add sugar in the next step.
# Add the file [http://dev.laptop.org/~probono/sbuntu/sugar.squashfs sugar.squashfs] to the directory casper/ on the USB stick

= How to make a Ubuntu 8.10 Sugar Live USB system from Windows =

(Note that these instructions can be used to make a Fedora-based Sugar Live USB, or can be run from Linux as well)

# Download the stock [http://releases.ubuntu.com/intrepid/ubuntu-8.10-desktop-i386.iso ubuntu-8.10-desktop-i386.iso]
# Download [http://unetbootin.sourceforge.net/ UNetbootin] [http://unetbootin.sourceforge.net/unetbootin-windows-latest.exe for Windows] (or [http://unetbootin.sourceforge.net/unetbootin-linux-latest Linux]) and run the application
# Supply the Ubuntu 8.10 desktop ISO file in the "Diskimage" option, select your target USB drive, and press "OK" to make an Ubuntu Live USB
# Add the file [http://dev.laptop.org/~probono/sbuntu/sugar.squashfs sugar.squashfs] to the directory casper/ on the USB stick

==Done==

The file sugar.squashfs is kept separate from the Ubuntu system itself, which stays unmodified. This way, sugar can easily be updated without having to download the entire Ubuntu system. Just replace sugar.squashfs with the updated version.

The instructions below are only interesting if you want to make your own sugar.squashfs instead of the one provided for download here.
How to make your own sugar.squashfs

First, create a stock Ubuntu 8.10 live USB system as described above and boot from it. Next, we need to fix a bug the initrd so that /cow is shown when we use the "show-cow" boot option by executing [http://launchpadlibrarian.net/18966594/apply-casper-patch.sh this] script. Now, we add "show-cow" to the boot arguments in syslinux.conf. Then, boot the system using the fix initrd, and do the following in the running system:

sudo su
echo "deb http://archive.ubuntu.com/ubuntu intrepid universe" >> /etc/apt/sources.list
echo "deb http://ppa.launchpad.net/sugarteam/ubuntu intrepid main" >> /etc/apt/sources.list
apt-get update
apt-get --force-yes -y install sugar-* squashfs-tools

* Then do

cat > exclude <<\EOF
etc
casper
cdrom
cow
home
media
rofs
tmp
var/cache
var/log
var/run
var/tmp
var/lock
var/mail
EOF

* It is important to exclude etc because otherwise permissions are mixed up

mount /cdrom -o remount,rw

time mksquashfs /cow/ /cdrom/casper/sugar.squashfs -ef exclude

*If you want the system to boot straight into Sugar (instead of Gnome) during boot, do:

mkdir -p add/etc/skel/
cat > add/etc/skel/.dmrc <<\EOF
[Desktop]
Session=sugar
EOF
mksquashfs add/ /cdrom/casper/sugar.squashfs

==Acknowledgments==

* OLPC Foundation
* Sugar Labs
* Debian Edu Developers and Debian OLPC
* Ubuntu sugarteam

===Author===
Simon Peter http://wiki.laptop.org/go/User:probono

Sugar on a Stick Ubuntu

2009-02-06T02:44:07Z

Gkovacs: instructions for Windows and non-Ubuntu distributions using UNetbootin

''Note: this procedure works for these circumstances'' Taken from http://dev.laptop.org/~probono/sbuntu/

''* Ubuntu 8.10 Intrepid Ibex''
''* Built November 29, 2008''
''* Browse activity works''

= How to make a Ubuntu 8.10 Sugar Live USB system =

# Download the stock [http://releases.ubuntu.com/intrepid/ubuntu-8.10-desktop-i386.iso ubuntu-8.10-desktop-i386.iso] and burn it
# Create a Live USB system using the built-in Live USB installer (In Ubuntu 8.10 and later: System --> Administration --> Create a USB startup disk). If you choose "Stored in reserved extra space", do '''NOT''' use up the entire available space, but leave at least 100 MB so that we can add sugar in the next step.
# Add the file [http://dev.laptop.org/~probono/sbuntu/sugar.squashfs sugar.squashfs] to the directory casper/ on the USB stick

= How to make a Ubuntu 8.10 Sugar Live USB system from Windows or on a non-Ubuntu Distribution =

(Note that these instructions work on Fedora as well)

# Download the stock [http://releases.ubuntu.com/intrepid/ubuntu-8.10-desktop-i386.iso ubuntu-8.10-desktop-i386.iso]
# Download [http://unetbootin.sourceforge.net/ UNetbootin] [http://unetbootin.sourceforge.net/unetbootin-windows-latest.exe for Windows] (or [http://unetbootin.sourceforge.net/unetbootin-linux-latest Linux]) and run the application
# Supply the Ubuntu 8.10 desktop ISO file in the "Diskimage" option, select your target USB drive, and press "OK" to make an Ubuntu Live USB
# Add the file [http://dev.laptop.org/~probono/sbuntu/sugar.squashfs sugar.squashfs] to the directory casper/ on the USB stick

==Done==

The file sugar.squashfs is kept separate from the Ubuntu system itself, which stays unmodified. This way, sugar can easily be updated without having to download the entire Ubuntu system. Just replace sugar.squashfs with the updated version.

The instructions below are only interesting if you want to make your own sugar.squashfs instead of the one provided for download here.
How to make your own sugar.squashfs

First, create a stock Ubuntu 8.10 live USB system as described above and boot from it. Next, we need to fix a bug the initrd so that /cow is shown when we use the "show-cow" boot option by executing [http://launchpadlibrarian.net/18966594/apply-casper-patch.sh this] script. Now, we add "show-cow" to the boot arguments in syslinux.conf. Then, boot the system using the fix initrd, and do the following in the running system:

sudo su
echo "deb http://archive.ubuntu.com/ubuntu intrepid universe" >> /etc/apt/sources.list
echo "deb http://ppa.launchpad.net/sugarteam/ubuntu intrepid main" >> /etc/apt/sources.list
apt-get update
apt-get --force-yes -y install sugar-* squashfs-tools

* Then do

cat > exclude <<\EOF
etc
casper
cdrom
cow
home
media
rofs
tmp
var/cache
var/log
var/run
var/tmp
var/lock
var/mail
EOF

* It is important to exclude etc because otherwise permissions are mixed up

mount /cdrom -o remount,rw

time mksquashfs /cow/ /cdrom/casper/sugar.squashfs -ef exclude

*If you want the system to boot straight into Sugar (instead of Gnome) during boot, do:

mkdir -p add/etc/skel/
cat > add/etc/skel/.dmrc <<\EOF
[Desktop]
Session=sugar
EOF
mksquashfs add/ /cdrom/casper/sugar.squashfs

==Acknowledgments==

* OLPC Foundation
* Sugar Labs
* Debian Edu Developers and Debian OLPC
* Ubuntu sugarteam

===Author===
Simon Peter http://wiki.laptop.org/go/User:probono

Sugar on a Stick

2009-02-06T02:40:13Z

Gkovacs: fixed slight typo on UNetbootin instructions

==About Sugar on a Stick (SoaS) ==

[[Image:Soas.jpg|thumb]]

The goal of the Sugar on a Stick project is to give children access to *their* Sugar on any computer in their environment with just a USB key.

We are still in preAlpha on this project. It is not ready to use with children yet.

===Creating a USB Stick===

The basics of creating a Live, bootable USB are you download a '.iso' image. Then you use a USB creator program to copy it to a USB, create a space for users to store files on the USB (if required) and set it to be bootable.

In practice this means the first thing you need is a Live USB creator.

We are working with live USBs based on both Fedora and Ubuntu (these are different distributions of Linux), you can try either or both.

If you are helping us with testing, please be sure to use one of the Sugar .iso's from this page. Please keep notes on which you got and when. Please report bugs to dev.sugarlabs.org and finally, please join the Moodle Class where we are coordinating testing. http://schools.sugarlabs.org/course/view.php?id=17

This is still in testing, its not ready for production use yet.

====Fedora based Sugar on a Stick from Windows====

To create a Fedora based Sugar from a Windows machine: https://fedorahosted.org/liveusb-creator/

Download that file to a windows machine and install it. (I couldn't get it to run in wine... --[[User:Walter|Walter]] 17:40, 3 December 2008 (UTC))

Next download current version of Sugar that we are testing for Sugar on a Stick:

http://download.sugarlabs.org/soas/snapshots/1/Soas-200902022242.iso

#Plug in a 1GB or larger USB stick into your computer.
#Open "liveusb-creator.exe"
#Use "Browse" to select the iso file that you just downloaded.
#Set the Target Device to your USB device
#Move the slider to set some persistent storage (hint, set it high then it will tell you how much space there is and you can adjust it to the right point)
#Click "Create Live USB". It will take a few minutes.

How much persistent memory you set will depend on the size of the .iso and the size of your USB but make sure you have some so people can save files.

=====Problem=====
I tried to do this and was able to create the files on the stick, but when I try to book on a Toshiba Tecra M4
(which can boot from USB sticks) I get
<pre>
SYSLINUX 3.72 2008-0925-25 EBIOS Copyright (C) 1994-2008 H. Peter Unvin
Could not find kernel image: linux
boot:
</pre>

====Ubuntu based Sugar on a Stick====

* Detailed instructions are here: http://dev.laptop.org/~probono/sbuntu/ and in [[Sugar on a Stick Ubuntu]]

====Ubuntu based Sugar on a Stick from Windows====

(Note that these instructions work on Fedora as well, and also can be run from any Linux distribution or Windows)

# Download the stock [http://releases.ubuntu.com/intrepid/ubuntu-8.10-desktop-i386.iso ubuntu-8.10-desktop-i386.iso]
# Download [http://unetbootin.sourceforge.net/ UNetbootin] [http://unetbootin.sourceforge.net/unetbootin-windows-latest.exe for Windows] (or [http://unetbootin.sourceforge.net/unetbootin-linux-latest Linux]) and run the application
# Supply the Ubuntu 8.10 desktop ISO file in the "Diskimage" option, select your target USB drive, and press "OK" to make an Ubuntu Live USB
# Add the file [http://dev.laptop.org/~probono/sbuntu/sugar.squashfs sugar.squashfs] to the directory casper/ on the USB stick

===Using Sugar on a Stick===

The trick is to set the BIOS to boot from USB. Unfortunately this is slightly different on each computer. Try Google on BIOS your computer model and explore the setup screen for your system.

If you have trouble try creating a "Boot Helper" CD using the .iso below. This will start the boot from the CD then read files from the USB.

http://www.sugarlabs.org/~marco/boot.iso

You can also use the 'PLoP Boot Manager' to create a boot floppy for machines without the ability to boot from CD or USB, see http://syslinux.zytor.com/wiki/index.php/No_native_BIOS_support

===VirtualBox and Sugar on a Stick===

At our pilot school for Sugar on a Stick the teachers has MacBooks. Our plan is to have teachers run Sugar in emulation mode on their personal Macbooks, while the students use the USB to access a variety of computers.

Instructions for using a preconfigured VirtualBox disk image with Sugar on a Stick [[Sugar on a Stick VirtualBox]]

===Cloning Sugar on a Stick===

The vision is that an adult will be able to create a Sugar on a Stick, add activities, set the language and the collaboration service and then "clone" the stick to create Sugar on a Stick for all the students in a class or a school.

[[Specs for Cloning Sugar on a Stick]]

===Join the development effort===

We are coordinating work on this project on Sugar Labs Moodle system. Please create an account and join the Sugar on a Stick Class. http://schools.sugarlabs.org/course/view.php?id=17

====Report Sugar on a Stick Bugs====

Use the Sugar Trac system at http://dev.sugarlabs.org to report bugs. Use the "SoaS" component.

===Project Goals===

* Sugar on a Stick as 1 to 1 computing in an elementary school
* Sugar on a Stick to empower middle and high school students to help test [[Sugar as Service Learning]]
* Sugar on a Stick for conferences to let people try Sugar and collaborate with other conference participants

===Sugar on a Stick ToDo List===

* Create an Easy Emulation for Boston School teachers macbooks - http://dev.sugarlabs.org/ticket/114
* Create a boot helper CD that lets Macs boot from the stick
* Install the latest version of Sugar on a Stick, pick an activity and test it
* Create a system for tracking activity testing
* Solicit USB stick donations from companies
* Create a design for a CD sticker to put on boot helper CDs

=== See Also ===
*[[DeploymentTeam/School_Key]]

Sugar on a Stick

2009-02-06T02:38:49Z

Gkovacs: added instructions for creating Live USB from Windows using UNetbootin

==About Sugar on a Stick (SoaS) ==

[[Image:Soas.jpg|thumb]]

The goal of the Sugar on a Stick project is to give children access to *their* Sugar on any computer in their environment with just a USB key.

We are still in preAlpha on this project. It is not ready to use with children yet.

===Creating a USB Stick===

The basics of creating a Live, bootable USB are you download a '.iso' image. Then you use a USB creator program to copy it to a USB, create a space for users to store files on the USB (if required) and set it to be bootable.

In practice this means the first thing you need is a Live USB creator.

We are working with live USBs based on both Fedora and Ubuntu (these are different distributions of Linux), you can try either or both.

If you are helping us with testing, please be sure to use one of the Sugar .iso's from this page. Please keep notes on which you got and when. Please report bugs to dev.sugarlabs.org and finally, please join the Moodle Class where we are coordinating testing. http://schools.sugarlabs.org/course/view.php?id=17

This is still in testing, its not ready for production use yet.

====Fedora based Sugar on a Stick from Windows====

To create a Fedora based Sugar from a Windows machine: https://fedorahosted.org/liveusb-creator/

Download that file to a windows machine and install it. (I couldn't get it to run in wine... --[[User:Walter|Walter]] 17:40, 3 December 2008 (UTC))

Next download current version of Sugar that we are testing for Sugar on a Stick:

http://download.sugarlabs.org/soas/snapshots/1/Soas-200902022242.iso

#Plug in a 1GB or larger USB stick into your computer.
#Open "liveusb-creator.exe"
#Use "Browse" to select the iso file that you just downloaded.
#Set the Target Device to your USB device
#Move the slider to set some persistent storage (hint, set it high then it will tell you how much space there is and you can adjust it to the right point)
#Click "Create Live USB". It will take a few minutes.

How much persistent memory you set will depend on the size of the .iso and the size of your USB but make sure you have some so people can save files.

=====Problem=====
I tried to do this and was able to create the files on the stick, but when I try to book on a Toshiba Tecra M4
(which can boot from USB sticks) I get
<pre>
SYSLINUX 3.72 2008-0925-25 EBIOS Copyright (C) 1994-2008 H. Peter Unvin
Could not find kernel image: linux
boot:
</pre>

====Ubuntu based Sugar on a Stick====

* Detailed instructions are here: http://dev.laptop.org/~probono/sbuntu/ and in [[Sugar on a Stick Ubuntu]]

====Ubuntu based Sugar on a Stick from Windows====

(Note that these instructions work on Fedora as well, and also can be run from any Linux distribution or Windows)

# Download the stock [http://releases.ubuntu.com/intrepid/ubuntu-8.10-desktop-i386.iso ubuntu-8.10-desktop-i386.iso]
# Download [http://unetbootin.sourceforge.net/unetbootin-windows-latest.exe UNetbootin for Windows] (or [http://unetbootin.sourceforge.net/unetbootin-linux-latest] Linux) and run the application
# Supply the Ubuntu 8.10 desktop ISO file in the "Diskimage" option, select your target USB drive, and press "OK" to make an Ubuntu Live USB
# Add the file [http://dev.laptop.org/~probono/sbuntu/sugar.squashfs sugar.squashfs] to the directory casper/ on the USB stick

===Using Sugar on a Stick===

The trick is to set the BIOS to boot from USB. Unfortunately this is slightly different on each computer. Try Google on BIOS your computer model and explore the setup screen for your system.

If you have trouble try creating a "Boot Helper" CD using the .iso below. This will start the boot from the CD then read files from the USB.

http://www.sugarlabs.org/~marco/boot.iso

You can also use the 'PLoP Boot Manager' to create a boot floppy for machines without the ability to boot from CD or USB, see http://syslinux.zytor.com/wiki/index.php/No_native_BIOS_support

===VirtualBox and Sugar on a Stick===

At our pilot school for Sugar on a Stick the teachers has MacBooks. Our plan is to have teachers run Sugar in emulation mode on their personal Macbooks, while the students use the USB to access a variety of computers.

Instructions for using a preconfigured VirtualBox disk image with Sugar on a Stick [[Sugar on a Stick VirtualBox]]

===Cloning Sugar on a Stick===

The vision is that an adult will be able to create a Sugar on a Stick, add activities, set the language and the collaboration service and then "clone" the stick to create Sugar on a Stick for all the students in a class or a school.

[[Specs for Cloning Sugar on a Stick]]

===Join the development effort===

We are coordinating work on this project on Sugar Labs Moodle system. Please create an account and join the Sugar on a Stick Class. http://schools.sugarlabs.org/course/view.php?id=17

====Report Sugar on a Stick Bugs====

Use the Sugar Trac system at http://dev.sugarlabs.org to report bugs. Use the "SoaS" component.

===Project Goals===

* Sugar on a Stick as 1 to 1 computing in an elementary school
* Sugar on a Stick to empower middle and high school students to help test [[Sugar as Service Learning]]
* Sugar on a Stick for conferences to let people try Sugar and collaborate with other conference participants

===Sugar on a Stick ToDo List===

* Create an Easy Emulation for Boston School teachers macbooks - http://dev.sugarlabs.org/ticket/114
* Create a boot helper CD that lets Macs boot from the stick
* Install the latest version of Sugar on a Stick, pick an activity and test it
* Create a system for tracking activity testing
* Solicit USB stick donations from companies
* Create a design for a CD sticker to put on boot helper CDs

=== See Also ===
*[[DeploymentTeam/School_Key]]