41
edits
Changes
Summer of Code/2009/Multimedia-broadcasting (view source)
Revision as of 20:08, 11 April 2009
, 20:08, 11 April 2009noting that this can be used for low-bandwidth lecture slides as well
Q: '''What is the name of your project?
Q: '''What is the name of your project?
A: A Framework for multimedia-rich conferencing with autodiscovered peers on the local network
A: A Framework for multimedia broadcasting to students on the local network
----
----
Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''
Q: '''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''
This project aims to create an Activity to allow users to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.
This project aims to create an Activity to allow users (primarily teachers or students presenting material in clas) to easily broadcast and stream audio and video from their webcam and microphone or their computer's display output to a classroom's central server, and share the live feed with their peers (the rest of the classroom). This would be of use for various other purposes as well, such as for displaying lecture slides. This will be integrated into the Neighborhood View (as a "broadcast audio and video to" option or similar) to allow for most seamless sharing of broadcasts.
'''Rationale'''
'''Rationale'''
This project does not aim to recreate Skype or similar central-server small-scale conferencing software. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:
This project does not aim to recreate Skype or similar central-server small-scale conferencing software; rather it aims to displace the usage of expensive projectors for displaying lecture slides and multimedia. This project's local-peer-discovery and mass-broadcasting architecture would be of use in a classroom setting for the following reasons:
As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information from a to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.
As we have seen in classroom settings such as MIT's TEAL project (for teaching introductory Physics), in the context of displaying live information, whether it be a video feed from a presentation or just lecture slides, to a large group of students for an experiment, lecture, or the like, a video feed must be established and broadcast, ideally via several video displays. In TEAL, many projectors placed throughout the classroom are used for this purpose; however for an elementary school this would be prohibitively expensive. Assuming said elementary school is running a pilot program of Sugar-equipped laptops, they can instead have each student's laptop "listen" to incoming audio and video broadcasts, and display it when received.
To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
To summarize, this is essentially more of an attempt to replicate what projectors, presentations, and laser pointers are used for today (broadcasting info to the students and soliciting their feedback), rather than an activity-specific rich-immersion one-on-one student
collaboration attempt. The advantage is merely convenience; if the student is receiving the stream from the teacher, he can view it from
collaboration attempt. The advantage is merely convenience and cost; it eliminates the need for an expensive projector to display lecture slides, and if the student is receiving the stream from the teacher, he can view it from
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
the convenience of his laptop, or he can for example pause, rewind, or record the stream if it's being cached, which he would be unable to do
if the teacher is merely displaying information on a projector.
if the teacher is merely displaying information on a projector.
For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.
For extremely large classrooms, bandwidth issues may arise due to limitations of the wireless network. In this case, to ensure that all students are able to view the stream without saturating the network, when the teacher selects "Broadcast to All", the software would first determine the number of new streams that could be streamed based on existing network activity, bandwidth required per stream, and the network's bandwidth capacity. Then, it would retrieve the locations of the laptops in the classroom, and select a number of laptops equivalent to the number of supported streams such that the selected laptops would be in proximity to the unselected laptops, thereby allowing those students who are not receiving the streams to their own laptop to view the stream on a nearby laptop.
The usage of UDP multicast broadcasting will also help conserve bandwidth and prevent lag in the video, thus as mentioned in "Architecture" if a streaming server such as Helix Server is used instead then this can be taken advantage of.
Alternatively, bandwidth usage when broadcasting to a large classroom with unicast could also be limited by broadcasting a low-framerate stream based on the number of users. While this would lead to a clear degradation in quality in the case of full-motion video, this is a rather niche medium to be shown in class; rather, lecture slides, which are what projectors are usually used for, would appear perfectly with a low-framerate stream.
The usage of UDP multicast broadcasting will also help conserve bandwidth and prevent lag in the video, thus as mentioned in "Architecture" if a streaming server such as Helix Server is used instead then this can be taken advantage of. However the combined usage of multicast and unicast streams on a single access point introduces many performance issues, so this may not be realistic in the context of broadcasting full-motion video (though for the primary target usage, lecture slides, which requires considerably less bandwidth to broadcast, there shouldn't performance issues with either unicast or multicast).
'''Viewing Interface'''
'''Viewing Interface'''
'''Audio and Video Sources'''
'''Audio and Video Sources'''
For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.
For the project, Webcams and X11 desktop output would be supported as video sources, and the microphone for would be an audio source. Capturing X11 desktop output would additionally allow for presentation of lecture slides via this system. These will of course not be hard-coded into the application, but will be represented as generic "media source" plugins. For example, since this architecture could have the potential to replace the standard projector-based lecture presentation mechanism, perhaps other media formats, namely PDF lecture slides, might be supported as a future enhancement (beyond the scope of this GSoC) as well.
'''Video Interaction'''
'''Video Interaction'''
Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''
Q: '''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. This is usually where people describe their past experiences, credentials, prior projects, schoolwork, and that sort of thing, but be creative. Link to prior work or other resources as relevant.'''
I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde
I have created successful (2 million+ downloads) FOSS projects in the past, with similar intense development timelines of a few weeks (please see details on the two major FOSS projects I created, [http://wubi-installer.org/ Wubi] and [http://unetbootin.sourceforge.net/ UNetbootin], described above, as well as my [http://launchpad.net/~gezakovacs Launchpad] profile).
In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.
With respect to experience this field of multimedia broadcasting I created a prototype application in Qt for an unrelated project with a feature that performs video broadcasting over Icecast to local devices (albeit via gstreamer-tools, in a more crude manner than I aim to do so) a few months ago, which can be found at http://launchpad.net/tde In that project, the focus was on interaction of mobile Internet Tablet devices with other household electronics, such as TVs and the like, as well as the exploration of a time-centric desktop computing paradigm. However, the icecast-based broadcasting architecture there is similar (albeit far more primitive, lacking security features, and restricted to only local devices as targets) to what I aim to implement here. Having learned more about the constraints and architecture of mass-video broadcasting over Icecast, I believe I now have the necessary knowledge to deliver a production-ready implementation of the described multimedia-broadcasting activity.
In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.
In addition, I have also worked with video APIs (primarily FFmpeg's libavcodec/libavformat, and OpenCV, though I have used GStreamer on another project) as part of my current undergaduate research project (unfortunately currently proprietary) focused on emotion recognition based on facial and speech features, so I am familiar with how to perform real-time video and audio capture and addressing the usual issues that show up in terms of laggy capture performance or locking threads; in this case this should be even simpler as no data processing must be done.