Revision as of 12:21, 16 March 2018

Sugarizer School Box - GSoC '18

About Me

Name: Rishabh Nambiar

Email: rishabhn@protonmail.com

Sugar Labs wiki username: Rishabhnambia

IRC nickname: rishabhnambiar

Languages: English

Where am I located? and what hours(UTC) do I tend to work?

I'm located in Mumbai, India (UTC +05:30).

Work Hours: 5:00 AM to 2:00 PM UTC

Past experiences with Open-Source projects:

I have worked with an open-source organization called ERPNext sporadically between 2016 and 2017.

I added an AWS S3 Integration to Frappe, a Python framework that powers ERPNext.
https://github.com/frappe/frappe/pull/4272
Here’s a PR I made that has some Ansible:
https://github.com/frappe/bench/pull/473/
I've been making minor contributions since May 2016.

Project Description

Goal #1: Sugarizer Raspberry Pi Images with build scripts

The first goal is to create a Raspberry Pi image and build scripts based on Raspbian that can serve Sugarizer to a classroom full of students.

On booting, the Raspberry Pi will:

act as a WiFi AP that clients(browsers, sugarizer apps) can connect to.
serve Sugarizer using Sugarizer-server on the local WiFi so there will be no requirement for Internet connectivity for now.
start a browser session running Sugarizer client so a single user/instructor can use it by connecting a display to the Pi.

This is being created because the Raspberry Pi is inexpensive, widely available and many such Sugarizer School Boxes can provide a cost-effective way of reaching out to more students and communities through Sugar.

Improvements to the Internet-In-A-Box Ansible playbook for Sugarizer will also be made in this section.

This will include making sure that the IIAB playbook uses the latest version of sugarizer with all available functionality and ensuring that it works.

Collaboration

Sugarizer already works really well when deployed locally on a Raspberry Pi and Neighbourhood view is also functional.

Goal #2: Heroku Button

The second goal is to simplify the way to deploy Sugarizer in the cloud. The priority in this section is get a Heroku Button for sugarizer up and running.

A Heroku Button is a simple HTML or Markdown snippet that can be added to READMEs. Clicking a Heroku Button will take you through a guided process to configure and deploy an app running the source code referenced by the button.

This is being done to provide users a simple, one-click and hassle-free deployment method that will let anyone set up their own instance of Sugarizer without opening a terminal window.

Minimum Deliverables

The community has suggested that I should treat improving the Sugarizer School Box as Goal #1 and the deployment scripts as Goal #2. I will start working on the deployment scripts only when Goal #1 is complete.
As 90% of the project should ideally be complete by the 2nd Evaluation phase, the Raspberry Pi build, the improved IIAB installer and the Heroku Button script will be ready by then.

Best Case Scenario

If things go as per the timeline till the 2nd evaluation phase, I will also be making a Bash + Docker script for sugarizer deployment. Ansible, Terraform or Packer can be added to the script to allow compatibility for different VPS providers. As the Docker build for Sugarizer-server and Ansible installer for IIAB already exist, this will not take long.

Timeline

Week	Date(2018)	Period	Task
	April 23 - May 13	Community Bonding	Finish node.js performance analysis for Sugarizer-server and post the results for the community. The analysis includes parameters like file download size for every page/activity, frequency of page visits, processing required per request and seeing how the Pi server behaves in different conditions. Set up a blog for posting my weekly progress.

01	May 14 - May 21		Create a Raspbian Image build for the Pi3 that has an Access Point and Sugarizer-Server running on boot. Use hostapd and dnsmasq to create an access point as shown in this article. This project will involve writing some bash for the build and I’ll use these resources for inspiration: https://github.com/sugarlabs/rpi23-gen-image https://github.com/iiab/iiab-factory/tree/master/box/rpi
02	May 21 - May 28		Continue to work on Raspbian Image build and test on my Raspberry Pi 3. Images should be available to the community and for testing. Publish the Raspberry Pi image and the build scripts.
03	May 29 - June 04		Improve the Ansible playbook for installing Sugarizer on the Internet-In-A-Box installer. This will include making sure that the IIAB play uses the latest version of sugarizer with all available functionality and ensuring that it works.
04	June 05 - June 11		On the IIAB Sugarizer install, add safeguards and checks to reduce MongoDB corruption issues caused by incorrect shutdown methods. Use the remaining week to make other improvements and fix issues that crop up during Week 3.

05	June 12 - June 20		Formulate a robust upgrade process to ensure that Sugarizer updates are installed successfully without losing any data to instances that have been deployed. Explore pm2 as a solution to do solve this or use other features of pm2 to enhance reliability. Monitoring Development Workflow with CI Run updates using pm2
06	June 21 - June 29	Midterm Evaluation	Generate a Heroku Button for one-click Sugarizer-server deployment. The project should ideally be 90% complete by the midterm evaluation phase. So my goal at this stage is to have a functional Raspberry Pi image build for Sugarizer, significantly improved IIAB Ansible install script and the Heroku Button.
07	June 30 - July 08		Study how to make the Sugarizer School Box communicate with the Android/iOS client apps. Test the working of the Sugarizer School Box with every possible type of client.
08	July 09 - July 13		Write a Bash + Docker script to deploy Sugarizer-server on a VPS.This will be done the traditional DevOps way using Ansible. The Ansible component will not take much time because a similar play has already been written here.

09	July 14 - July 20		If the Turtle/Music Blocks release is on schedule, I will add it to the Sugarizer Pi Image build. We can use this release as a test for the upgrade processes created in Week 5. Improvements to the upgrade process will be made.
10	July 21 - July 28		Test deployment options thoroughly on different VPS providers. Use Packer or Terraform on the Bash + Docker script written in Week 8 to improve compatibility between VPS providers.
11	July 29 - Aug 5		If all previous objectives are achieved, explore setting up a CI tool to improve the project and look for avenues where CI can help.
12	6 Aug - August 14		Buffer week for finding more issues and preparing the final submission.

	August 06 - August 14	Final Evaluations

Convince us why you can finish this project

I believe I can complete this project because of my love and fascination for Linux and my previous work experience with Ansible, Docker and Python.

This is the first time I’m applying for GSoC and Sugar Labs is the only organization I’m applying to because this project is a really good fit for my skill-set.

I mentioned my experience with an open-source organization called ERPNext above.

In the summer of 2016, I worked as a full-time intern for the same organization. I added automated AWS S3 Backups to their in-house deployment tool, Central.
This included writing Ansible playbooks and Python that would set up automated backups for all their production servers for their enterprise clients. This was NOT an open source contribution but I’m mentioning this here because it is a major reason for my familiarity with Linux and Deployment.
Some presentations depicting my work at ERPNext can be viewed here.

In the summer of 2017, I was at Cube as a Full-Stack Development Intern. Over the summer, I built a Python based (bottle.py) web application that automated a lot of maintenance tasks for the Operations team that included web scraping and API Development. I also containerized the application using Docker and Docker-Compose.

The Project and the Community

Rishabh Nambiar

rishabhn@protonmail.com, Twitter

If the project is successfully completed, it will provide Sugar Labs a more portable and inexpensive way of deploying Sugarizer. The Raspberry Pi is inexpensive, widely available and Sugarizer School Boxes can provide a cost-effective way of reaching out to more students and communities through Sugar. This method of deployment requires only one Raspberry Pi 3.

The convenience of using one-click deployment scripts can be instrumental in getting non-developers to using a project and using a Heroku Button is even better when you don’t have to use to a terminal to have your own instance of Sugarizer.

Answers from the Community

Michaël Ohayon - mohayon75@gmail.com (Potential GSoC Mentor)

1) I think that the Pi will be able to handle the load, we won’t have that much kids connected to the same Pi and the networking process is not that heavy.

We should try to launch the server in background and display a web browser in foreground. If the load is too high we will see it quickly.

2) Yes, you’re right, the Pi should provide a Wifi AP to allow devices to connect. This AP should bring routing from Ethernet if connected.

3) Everything is possible, Ansible is a great tool. Combined with Terraform and Packer we should be allowed to deploy things without having troubles handling multiple cloud providers.

The installation script could be performed using simple bash and docker.

4) I think you can start thinking on your proposal. You can continue to discuss with us to talk about the project but also the community and the impact of that project.

5) For the deployment part we have two options that would be nice to have.

a really simple automation like heroku single click deploy (https://blog.heroku.com/heroku-button). This is the killer feature we should definitely have.
the devops way using tools like Ansible and Terraform and Packer.

One major thing for all platforms is to think about the upgrade processes, how can we update the devices/server without losing data.

Tony Anderson - tony_anderson@usa.net - http://schoolserver.org/

This is exactly how the xsce server works so you may get valuable help from that community (xsce or iiab).

A continuing issue is performance of the server in a classroom or school. One metric is the number of simultaneous connections the device can support (a classroom of 40-60 is not uncommon). Response time to requests to the server can be limited by the size of memory, the speed of access to the sd card, or the processor speed. I would be very interested in the methodology you propose since that process would apply equally to the schoolserver.

One issue is to characterize the workload - how often does a user request a transaction from the server, what is the time between requests (when the user is reading the response to the previous request), how much processing is required for a request (e.g. a text search), how much information is required to satisfy a request (e.g. size of file download). So far as I know no one has attempted this characterization for a classroom. This load could be different for Sugarizer than for Sugar, but the effort would be valuable in any case)

Tim Moody - tim@timmoody.com - http://internet-in-a-box.org/

Speaking of reliability, we have experienced a number of occasions when mongodb was corrupt on the rpi, perhaps through disorderly shutdown. Perhaps you can sort that out.

I would also recommend focusing on the rpi install before turning to Heroku or AWS as most users are without an internet connection. All I'm suggesting is that you start with Sugarizer School Box item 1 and then proceed to item 2 when it is complete.

What will you do if you get stuck on your project and your mentor isn't around?

If I get stuck, I will first try my best to find a solution myself (Web Search, StackOverflow). I can ask for help on #sugar or from some friends and ex-colleagues who are Open Source enthusiasts.

How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?

In the Community Bonding period, I will create a blog for myself and I will share my weekly experiences and progress through it.

Miscellaneous

Setting up a Development Environment

I wasn’t sure if this project falls under the Sugar-Web projects or the Sugar projects.

I've deployed Sugarizer on my Raspberry Pi 3 and a VPS with Docker and without it.

It works really well and I haven’t run into any major issues.

Workload Analysis for sugarizer-server

I've been testing the page load times for some activities using Locust, a user load testing tool written in Python that lets you test site performance by swarming it with users.

This will fulfill one of the tasks in the node.js performance analysis of sugarizer-server which lets us know the size of the file download per page/activity and response times as the number of users increases.This analysis will be important when I run them on the Raspberry Pi WiFi AP along with the other tasks in parallel. I will try to replace this section with a more complete analysis before the final proposal.

PS: This screenshot is not a good indicator of real-world performance because:

300 users
30 requests/second

Describe a great learning experience you had as a child.

in-progress

@@ Line 42: / Line 42: @@
 :Improvements to the Internet-In-A-Box [https://github.com/iiab/iiab/tree/master/roles/sugarizer Ansible playbook] for Sugarizer will also be made in this section.
 :This will include making sure that the IIAB playbook uses the latest version of sugarizer with all available functionality and ensuring that it works.
+:'''Collaboration'''
+:Sugarizer already works really well when deployed locally on a Raspberry Pi and Neighbourhood view is also functional.
 ====Goal #2: Heroku Button====

Difference between revisions of "User:Rishabhnambia"