Summer of Code/2014/bliss-sid/Voice-interface: Difference between revisions

(4 intermediate revisions by 2 users not shown)

Line 5:

'''1. What is your name?'''

~~My name is~~ Siddharth Lalwani ~~and I am a 1st year undergraduate in Computer Science Engineering.~~

Siddharth Lalwani

'''2. What is your email address?'''

Line 25:

'''6. Where are you located, and what hours (UTC) do you tend to work? (We also try to match mentors by general time zone if possible.)'''

I live in Delhi,India . IST(Indian Standard Time) is UTC + 5:30 hours .During College,I will usually work from 13:30 to 17:30[UTC] on weekdays and will cover up with some extra time during weekends.~~After 2 June~~,I ~~can~~ work fulltime.

I live in Delhi,India . IST(Indian Standard Time) is UTC + 5:30 hours .During College,I will usually work from 13:30 to 17:30[UTC] on weekdays and will cover up with some extra time during weekends.Afterwards,I will be able to work fulltime ,that is, (6:30-10:30) and (13:30-18:30)[UTC] or when it suits the mentor.

'''7. Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''

I am a novice Open Source developer ~~and~~ have been ~~working to enhance~~ SugarLabs repository for some time now.I am proficient in programming in Python,C and C++. I am deeply impressed by the collaborative working in an open source community and am keen to work with Sugarlabs this summer.

I am a novice Open Source developer.I used Facebook GRAPH API in Social Reader project. My profile on Github is https://github.com/bliss-sid . I have been understanding the SugarLabs repository for some time now.I am proficient in programming in Python,C and C++. I am deeply impressed by the collaborative working in an open source community and am keen to work with Sugarlabs this summer.

Line 43:

Line 42:

'''2. Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''

With my project,I am aiming to add another dimension to Sugar Labs learning experience by fabricating a speech recognition engine with the help of Pocketsphinx. ~~If we say~~-~~”Hi Sugar”~~,~~then the speech recognition system~~ will ~~be activated~~.I will add speech recognition within activities and add a documentation for every activity with certain keywords that will perfrom a task.~~With speech recognition :-~~

With my project,I am aiming to add another dimension to Sugar Labs learning experience by fabricating a speech recognition engine with the help of Pocketsphinx.I will start with en_US Hidden Markov Acoustic Model included with Pocketsphinx-0.8.For making it more interactive,we will use a Text-to-Speech engine like eSpeak( Or any better alternative).I will add speech recognition within activities and add a documentation for every activity with certain keywords that will perfrom a task.

~~->We can give commands to do a task or open activities like if we say “Open Terminal”~~, ~~the terminal activity will be initiated.~~

'''After completion of this project''',

-~~>We can create new activities for communication with which various users can talk to each other like an audio call etc~~.

'''1.''' If we say-”Hi Sugar”,then the speech recognition system will be activated.

~~->Text and Speech inter-conversion~~ can ~~also be used~~ to ~~create~~ activities like ~~Story Reader. It can be done using available TTS engines like Pico TTS in Android. It~~ will be ~~an interactive experience for young users and their learning will become even more fun~~.

'''2.''' We can give commands to do a task or open activities like if we say “Open Terminal”, the terminal activity will be initiated.

~~->We can add~~ special support for ~~most commonly used activities~~ like for TurtleBlocks activity,if we say “Forward 20”,it goes 20 units forward.

'''3.''' Their will be special support for '''TurtleBlocks,Terminal,Chat and Browse activity''' like for TurtleBlocks activity,if we say

“Forward 20”,it goes 20 units forward.

~~->We can also add support for multiple languages and dialects.~~

~~One of the major advantages of speech recognition will be for people with disabilities who cannot type~~. ~~Visually impaired users~~ can ~~control many computer tasks~~ and ~~a sense of detachment that they feel from the world can be removed through this system~~.

'''4.'''We can also add support for multiple languages and dialects by using various acoustic models and translation support(Apertium,if needed).

For setting up voice interface, I will mainly use Python~~,Java Script~~ and other open source resources like PocketSphinx and Sphinxbase.

Voice Recognition has been around for over 50 years but it hasn't effectively arrived to the scene in education and learning.With voice interface in SugarLabs,I want to make learning fun for children.Speech Recognition will serve as a assistive technology in many future educational activities.The idea of Voice Interface is not only about implementing Speech recognition but also laying the foundation for Speech understanding where the computer can interpret what we say and answer back with their own ideas. It may sound like science fiction but think if you say “Sugar,open flappy bird game” and it says back - “Their is a reminder in your Calendar activity which says you have an Exam tomorrow.You should better study for that.”.In a matter of few years,it will become a reality.

For setting up voice interface, I will mainly use Python and other open source resources like PocketSphinx and Sphinxbase.I will also use acoustic model from various open source resources.

Line 73:

Line 76:

| 9 April- 21 April || Enhance understanding of the architecture of project

|-

| 21 April-5 May || Discuss about the design and features of project with mentors and Sugarlabs community

| 21 April-5 May || Discuss about the design and features of project with mentors and Sugarlabs community.

|-

| 5 May-12 May || Buffer Week 1(Practical Examinations in College)

Line 98:

Line 101:

| 27 June-7 July || Add voice support to Chat activity

|-

| 7 July-19 July || Add Multiple Language Functionality to Speech Recognition Engine. Finish previous work.

| 7 July-19 July || Add Multiple Language Functionality to Speech Recognition Engine.Use various acoustic models.Finish previous work.

|-

| 19 July-1 August || Run the Iteration-“File and remove Bugs”.

Line 109:

Line 112:

If some time is left,I will work on creating a activity where user can create his/her own acoustic model.It will help reduce the errors in recognizing what the user has to say.

Line 126:

Line 129:

I worked with a senior to develop a Social RSS Reader Application for Android Platform using Facebook GRAPH API in order to synchronize user subscriptions with Facebook News Feed thereby making it possible to receive updates from and send updates to Facebook friends regarding the Application.

I have uploaded it on github https://github.com/bliss-sid/Social-RSS-Reader

I have adequate knowledge of PocketSphinx and Hidden Markov Acoustic Models.I am also doing a course on Cryptography on www.coursera.org. I am keen to work with SugarLabs and put all my efforts to successfully complete this project.

I know that I lack on experience in Open Source but I will surely cover up for that before Community bonding time and work even harder to stay high on your expectations.I have worked pretty hard on this project and I really feel that I am the apt geek for this project.

I have configured speech recognition on my Ubuntu system with Google2Ubuntu.I have also studied the API documentation for the Pocketsphinx speech recognition engine and have learnt how to use it in this project.I am also doing a course on Cryptography on www.coursera.org. I am keen to work with SugarLabs and put all my efforts to successfully complete this project.

== '''You and the community''' ==

Line 136:

Line 143:

'''Me'''

Speech recognition is a ground breaking technology and it will make Sugar Labs more captivating. It will ~~give ease~~ to ~~users as they~~ can ~~do various tasks just by use~~ of ~~speech~~. ~~Users~~ who are unable to ~~write or see with~~ the ~~help of this application can perform~~ their ~~task like starting an activity or playing a song just by saying. It will lay off foundation~~ to ~~development of new set of activities which use voice interface~~. The barrier of language difference can also be removed with multiple language support.

Speech recognition is a ground breaking technology and it will make Sugar Labs more captivating.It has a vital place in education and with time,it is making inroads as an assistive technology.It will have innumerable benefits to Sugarlabs Community like -

> It will make the interface of Sugarlabs attractive and learning fun.And when we learn something with pleasure,we never forget it.Various activities like Play and Say games can be made to gauge pronounciation skills of children.

> Its most relevant application will be for users who are unable to type due to a physical disability. Speech recognition is the only way for their thoughts to get into readable form other than dictation to someone else.

> The barrier of language difference can also be removed with multiple language support and assistance of multiple dialects in the future.It will help bridge the learning gap between teachers and students located round the globe.

> Besides the advantages speech recognition will have in learning,it will also make using Sugarlabs easier and quick.Users can perform various tasks like switching between running activities just by saying certain keywords.

Line 144:

Line 159:

This could be due the fact that there is still a missing glue between the speech-recognition technologies and educational content developers. This project is about filling the gap –within the Sugar Learning Platform.

Would be great to have speech control in Sugar, for interaction with users and as a assistive tool. --[[User:Godiard|Godiard]] ([[User talk:Godiard|talk]]) 17:57, 18 March 2014 (EDT)

Line 153:

Line 171:

'''3. How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''

I will update my [http://nowirise.tumblr.com/ blog] weekly on the progress I have made on the project. Moreover, I will try to do the same on mailing list and IRC to keep the mentor and community informed.

Line 158:

Line 177:

'''

== Miscellaneous ==

'''

Line 180:

Line 200:

'''3. Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''

I read somewhere that Summer of Code is not about how much you know. It is about how quickly you can learn and how eager you are to contribute to your Open Source community.I have loved studying the Sugarlabs repository and learned a lot so far.

I feel it is important to start earlier on this project.All the codes I will make before the community bonding period will be uploaded at

https://github.com/bliss-sid/Voice-Interface

I ~~read somewhere that Summer of Code is not about how much you know. It is about how quickly you can learn and how eager you are to contribute to your Open Source community.~~ I ~~really want to give my everything~~ to ~~successfully complete this project~~.

Well,I feel it winds up all I have to say.

@@ Line 5: / Line 5: @@
 '''1. What is your name?'''
-My name is Siddharth Lalwani and I am a 1st year undergraduate in Computer Science Engineering.
+Siddharth Lalwani
 '''2. What is your email address?'''
@@ Line 25: / Line 25: @@
 '''6. Where are you located, and what hours (UTC) do you tend to work? (We also try to match mentors by general time zone if possible.)'''
-I live in Delhi,India . IST(Indian Standard Time) is UTC + 5:30 hours .During College,I will usually work from 13:30  to 17:30[UTC] on  weekdays and will cover up with some extra time during weekends.After 2 June,I can work fulltime.
+I live in Delhi,India . IST(Indian Standard Time) is UTC + 5:30 hours .During College,I will usually work from 13:30  to 17:30[UTC] on  weekdays and will cover up with some extra time during weekends.Afterwards,I will be able to work fulltime ,that is, (6:30-10:30) and (13:30-18:30)[UTC] or when it suits the mentor.
 '''7. Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects, or some other demonstration of the work that you have done in open-source. If not, why do you want to work on an open-source project this summer?'''
-I am a novice Open Source developer and have been working to enhance SugarLabs repository for some time now.I am proficient in programming in Python,C and C++. I am deeply impressed by the collaborative working in an open source community and am keen to work with Sugarlabs this summer.
+I am a novice Open Source developer.I used Facebook GRAPH API in Social Reader project. My profile on Github is https://github.com/bliss-sid . I have been understanding the SugarLabs repository for some time now.I am proficient in programming in Python,C and C++. I am deeply impressed by the collaborative working in an open source community and am keen to work with Sugarlabs this summer.
@@ Line 43: / Line 42: @@
 '''2. Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using?'''
-With my project,I am aiming to add another dimension to Sugar Labs learning experience by fabricating a speech recognition engine with the help of  Pocketsphinx. If we say-”Hi Sugar”,then the speech recognition system will be activated.I will add speech recognition within activities and add a documentation for every activity with certain keywords that will perfrom a task.With speech recognition :-
+With my project,I am aiming to add another dimension to Sugar Labs learning experience by fabricating a speech recognition engine with the help of  Pocketsphinx.I will start with en_US  Hidden Markov Acoustic Model included with Pocketsphinx-0.8.For making it more interactive,we will use a Text-to-Speech engine like eSpeak( Or any better alternative).I will add speech recognition within activities and add a documentation for every activity with certain keywords that will perfrom a task.
-->We can give commands to do a task or open activities like if we say “Open Terminal”, the terminal activity will be initiated.
+'''After completion of this project''',
-->We can create new activities for communication with which various users can talk to each other like an audio call etc.
+'''1.''' If we say-”Hi Sugar”,then the speech recognition system will be activated.
-->Text and Speech inter-conversion can also be used to create activities like Story Reader. It can be done using available TTS engines like Pico TTS in Android. It will be an interactive experience for young users and their learning will become even more fun.
+'''2.''' We can give commands to do a task or open activities like if we say “Open Terminal”, the terminal activity will be initiated.
-->We can add special support for most commonly used activities like for TurtleBlocks activity,if we say “Forward 20”,it goes 20 units forward.
+'''3.''' Their will be  special support for '''TurtleBlocks,Terminal,Chat and Browse activity''' like for TurtleBlocks activity,if we say
+                                         “Forward 20”,it goes 20 units forward.
-->We can also add support for multiple languages and dialects.
-One of the major advantages of speech recognition will be for people with disabilities who cannot type. Visually impaired users can control many computer tasks and a sense of detachment that they feel from the world can be removed through this system.
+'''4.'''We can also add support for multiple languages and dialects by using various acoustic models and translation support(Apertium,if needed).
-For setting up voice interface, I will mainly use Python,Java Script and other open source resources like PocketSphinx and Sphinxbase.
+Voice Recognition has been around for over 50 years but it hasn't effectively arrived to the scene in education and learning.With voice interface in SugarLabs,I want to make learning fun for children.Speech Recognition will serve as a assistive technology in many future educational activities.The idea of Voice Interface is not only about implementing Speech recognition but also laying the foundation for Speech understanding where the computer can interpret what we say and answer back with their own ideas. It may sound like science fiction but think if you say “Sugar,open flappy bird game” and it says back - “Their is a reminder in your Calendar activity which says you have an Exam tomorrow.You should better study for that.”.In a matter of few years,it will become a reality.
+For setting up voice interface, I will mainly use Python and other open source resources like PocketSphinx and Sphinxbase.I will also use acoustic model from various open source resources.
@@ Line 73: / Line 76: @@
 | 9 April- 21 April || Enhance  understanding of the architecture of project
 |-
-| 21 April-5 May || Discuss about the design and features of project with mentors and Sugarlabs community
+| 21 April-5 May || Discuss about the design and features of project with mentors and Sugarlabs community.
 |-
 | 5 May-12 May || Buffer Week 1(Practical Examinations in College)
@@ Line 98: / Line 101: @@
 | 27 June-7 July || Add voice support to Chat activity
 |-
-| 7 July-19 July || Add Multiple Language Functionality to Speech Recognition Engine.  Finish previous work.
+| 7 July-19 July || Add Multiple Language Functionality to Speech Recognition Engine.Use various acoustic models.Finish previous work.
 |-
 | 19 July-1 August || Run the Iteration-“File and remove Bugs”.
@@ Line 109: / Line 112: @@
+If some time is left,I will work on creating a activity where user can create his/her own acoustic model.It will help reduce the errors in recognizing what the user has to say.
@@ Line 126: / Line 129: @@
 I worked with a senior to develop a Social RSS Reader Application for Android Platform using Facebook GRAPH API in order to synchronize user subscriptions with Facebook News Feed thereby making it possible to receive updates from and send updates to Facebook friends regarding the Application.
+I have uploaded it on github https://github.com/bliss-sid/Social-RSS-Reader
+I have adequate knowledge of PocketSphinx and Hidden Markov Acoustic Models.I am also doing a course on Cryptography on www.coursera.org. I am keen to work with SugarLabs and put all my efforts to successfully complete this project.
+I know that I lack on experience in Open Source but I will surely cover up for that before Community bonding time and work even harder to stay high on your expectations.I have worked pretty hard on this project and I really feel that I am the apt geek for this project.
-I have configured speech recognition on my Ubuntu system with Google2Ubuntu.I have also studied the API documentation for the Pocketsphinx speech recognition engine and have learnt how to use it in this project.I am also doing a course on Cryptography on www.coursera.org. I am keen to work with SugarLabs and put all my efforts to successfully complete this project.
 == '''You and the community''' ==
@@ Line 136: / Line 143: @@
 '''Me'''
-Speech recognition is a ground breaking technology and it will make Sugar Labs more captivating. It will give ease to users as they can do various tasks just by use of speech. Users who are unable to write or see with the help of this application can perform their task like starting an activity or playing a song just by saying. It will lay off foundation to development of new set of activities which use voice interface. The barrier of language difference can also be removed with multiple language support.
+Speech recognition is a ground breaking technology and it will make Sugar Labs more captivating.It has a vital place in education and with time,it is making inroads as an assistive technology.It will have innumerable benefits to Sugarlabs Community  like -
+> It will make the interface of Sugarlabs attractive and learning fun.And when we learn something with pleasure,we never forget it.Various  activities like Play and Say games can be made to gauge pronounciation skills of children.
+> Its most relevant application will be for users who are unable to type due to a physical disability. Speech recognition is the only way for their thoughts to get into readable form other than dictation to someone else.
+> The barrier of language difference can also be removed with multiple language support and assistance of multiple dialects in the future.It will help bridge the learning gap between teachers and students located round the globe.
+>  Besides the advantages speech recognition will have in  learning,it will also make using Sugarlabs easier and quick.Users can perform various tasks like switching between running activities just by saying certain keywords.
@@ Line 144: / Line 159: @@
 This could be due the fact that there is still a missing glue between the speech-recognition technologies and educational content developers. This project is about filling the gap –within the Sugar Learning Platform.
+Would be great to have speech control in Sugar, for interaction with users and as a assistive tool. --[[User:Godiard|Godiard]] ([[User talk:Godiard|talk]]) 17:57, 18 March 2014 (EDT)
@@ Line 153: / Line 171: @@
 '''3. How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?'''
 I will update my [http://nowirise.tumblr.com/ blog] weekly on the progress I have made on the project. Moreover, I will try to do the same on mailing list and IRC to keep the mentor and community informed.
@@ Line 158: / Line 177: @@
 '''
 == Miscellaneous ==
 '''
@@ Line 180: / Line 200: @@
 '''3. Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''
+I read somewhere that Summer of Code is not about how much you know. It is about how quickly you can learn and how eager you are to contribute to your Open Source community.I have loved studying the Sugarlabs repository and learned a lot so far.
+I feel it is important to start earlier on this project.All the codes I will make before the community bonding period will be uploaded at
+	                        https://github.com/bliss-sid/Voice-Interface
-I read somewhere that Summer of Code is not about how much you know. It is about how quickly you can learn and how eager you are to contribute to your Open Source community. I really want to give my everything to successfully complete this project.
+Well,I feel it winds up all I have to say.