Summer of Code/2018/Music Blocks Widget Improvements
Music Blocks Widget Improvements
About You
What is your name?
My name is Ritwik Abhishek. I am 3rd year undergraduate student at Indian Institute of
Technology Guwahati.
What is your email address?
ritwikabhishek@gmail.com
What is your Sugar Labs wiki username ?
A-ritwik
What is your IRC nickname on irc.freenode.net?
a-ritwik
What is your first language?
My first language is Hindi. English is my language of communication and I am fluent in it.
Where are you located, and what hours (UTC) do you tend to work?
I live in Guwahati, India (UTC+5:30). I usually work from 5AM (UTC) to 2PM (UTC), But these
timing can vary depending on the need of project or mentor. I will remain active on IRC
during my working hours.
Have you participated in an open-source project before? If so, please send us URLs to your
profile pages for those projects, or some other demonstration of the work that you have
done in open-source. If not, why do you want to work on an open-source project this
summer?
I am new to the world of open-source. I like the idea very much that anyone can contribute
to a project of his/her choice and help to develop products which are comparable or better
than proprietary products. I chose Sugar Labs because Education is very close to my heart
and Sugar Labs works to provide kids equal opportunity to learn, irrespective of their
background.
About your project
What is the name of your project ?
Music Blocks Widget Improvement
Describe your project in 10-20 sentences. What are you making? Who are you making it
for, and why do they need it? What technologies (programming languages, etc.) will you
be using?
My project involves 3 separate parts
1) Fix widget bugs - There are several widgets in Music Blocks. As reported on github
repository https://github.com/walterbender/musicblocks/labels/widget , there are around
20 bugs reported currently. I think there would be another 10 bugs related to widgets which
would be reported during summers. I am planning to remove 2-3 bugs per day. Most of the bugs
involve pitch-time matrix, tempo, timbre, filters, rhythm ruler. Most of bugs are easy
and they would be fixed in around 10 days. Other bug reports involve adding some
functionality and are actually enhancements.
2) Pitch tracker - It involves making a widget which would record sound, process it to find
fundamental frequency for every small segment, and then choose blocks (whose frequency
is closest to fundamental frequency of each segment), and then use these blocks to code
the recorded sound in Music Blocks language.
I chose to use fundamental frequency because it can be calculated in real time and is easy to
calculate as well. For calculating fundamental frequency I will be using autocorrelation as
feature vector. Autocorrelation is correlation of a signal with its delayed copy.
There is a webapp written in javascript which calculates autocorrelation, whose code we
can use as it has MIT license. The repository is https://github.com/cwilso/PitchDetect/
If the fundamental frequency is not very strong then we will use fourier transform to get the
frequency domain representation of the signal, then we will find frequencies at which the
amplitude is higher. The smallest of these frequencies will be fundamental frequency. For
example in figure given below, fundamental frequency is shown by yellow arrow.
3) Integrating software keyboard in Music Blocks - The keyboard currently works fine but It
looks very different than rest of Music Blocks. I would like to work on design part of the
keyboard and change its design in following ways
- Currently the keyboard is fixed in size and it can not be moved, I would change it and
it would be available in at least 2 sizes and its position on the screen could be
changed.
- Currently the toggle button to minimize/show for keyboard is on bottom right
corner, I would change it such that It remains on the top left of keyboard just like
others.
- Making the edges of each key rounded, so it looks similar to other blocks
- Although actual keyboard is black and white, there should be option to change colors
of keys.
I think we should measure time for which a certain key was pressed, for that we need to
measure starting and stopping time. I have not finalized how to deal with this, and would
finalize this while going through the code of keyboard and after having detailed discussion
with my mentor.
Timeline
Was uploaded in main proposal.
Convince us, in 5-15 sentences that you will be able to successfully complete your project
in the timeline you have described.
I think I will be able to successfully complete the project because I am a hard working and
goal oriented person. I will be giving around 40-50 hours per week to the project. I will work
on every day of the week. I have no other commitment this summer. I am willing to learn
technologies that I don’t know and work hard to finish the project.
Some of the relevant projects done by me are described below
Audio class histogram - For a given audio segment we had to find out which part of audio
contains speech, music and silence. For this we had used 5 feature vectors. It was done in
MATLAB. Experience from Audio class histogram project would help me to complete pitch
tracker this summer. To know more about visit https://ee320.github.io/
Contribution - I have started contributing to music blocks. I have submitted a few bugs, and
am trying solve bugs as well.
https://github.com/walterbender/musicblocks/pull/1170
* will be adding more pull requests soon
Basic Website using Flask framework- Made a basic website using Flask framework. The
website has log in/out feature. I am proficient in HTML, CSS and JavaScript. To know more
visit - https://github.com/a-ritwik/flask-intro
To see smaller projects that I have done please visit my github page
https://github.com/a-ritwik
You and the community
If your project is successfully completed, what will its impact be on the Sugar Labs
community?
My Answer - My project has 3 parts, I have explained their impact one by one1- Fixing existing bugs of different widgets in Music Blocks - There are currently several
widgets in Music Blocks, and fixing existing bugs from them will enhance user experience,
remove improper system behavior, and make Music Blocks more robust.
2- Pitch tracker – After its completion, we can give input from microphone and use pitch
tracking to generate Music Blocks code. So after this we do not have to manually code for
each and every song, It could be done with good enough accuracy by the pitch tracker. As
Devin Ulibarri has suggested It could be a GCI task to choose a song, sing it, and then submit
both recording and Music Blocks code.
3- Integrating software keyboard with Music Blocks- This would be implemented as a widget
which on key press will give corresponding output as chunk. Currently the keyboard works
fine but it looks very much different than the theme of Music Blocks. The project will create
unity in design of keyboard with Music Blocks.