Changes

Jump to: navigation, search

WikiPort

343 bytes added, 13:45, 30 May 2018
m
no edit summary
{{DISPLAYTITLE:WikiPort GSoC 2018 - Migration activity page to git}}Activity Team - Setting up the new Activity Server
== Page Under Development GO SLOW! ==This page is under heavy development, thus for the time being, please refer to the proposal in richProject Status -formatting on [https://docs.google.com/document/d/1mPd_SeUqeJF_yKWVEDFuzU7LWsnUFSbLdQQ44HvsYyA/edit?usp=sharing Google Docs]ACTIVE
'''Retasked'''
- Details at [[Filehttp:Logo wikiport//www.tiny.cc/vgsoc www.tiny.png|frameless|center|WikiPort]cc/vgsoc]
- All code available at [https://www.github.com/vipulgupta2048/sugarport www.github.com/vipulgupta2048/sugarport]
== About You ==
'''Located at, and what hours (UTC) do you tend to work''' - I am based out of New Delhi, India (+5.30 GMT). I am flexible with timings and can work for 3-5 hours everyday.
 
== My Contributions/Love for Open-Source ==
'''Have you participated in an open-source project before? If so, please send us URLs to your profile pages for those projects or some other demonstration of the work that you have done in open-source?'''
Yes, I have participated been participating and maintained maintaining many open-source projects. Also been contributing to other community communities and conference projects. Some of the links Links and detailed description of those pages are as follows
* [https://github.com/asetalias/asetalias.github.io/pull/52 First pull request] on my college open-source community’s website, [http://www.github.com/asetalias ALiAS] - Maintainer since 2016.
* [https://github.com/vipulgupta2048/opengisproj_mirror vipulgupta2048/opengisproj_mirror]- Involved building a web app for real-time monitoring of pollution levels in rivers of India and to assess the impact on the health of people using the water from the river. The working repo is private, hence a public mirror for showcasing.
* [https://github.com/vipulgupta2048/scrape vipulgupta2048/scrape] Scraping major news websites/outlets in India using Scrapy [Python] Managing the data flow by employing PostgreSQL. Finding fake news using Machine Learning with help of libraries (Word2vec)
* Fixed bugs, wrote blogs, added new features using HTML, CSS , and JS for the official website of the premier conference on Python in India, PyCON PyCon India 2017 and now in 2018 workgroup ([https://github.com/pythonindia/inpycon2017/pulls?utf8=%E2%9C%93&q=is%3Apr+author%3Avipulgupta2048+is%3Amerged+sort%3Acreated-asc+ Here] and [https://github.com/pythonindia/inpycon-blog/pulls?utf8=%E2%9C%93&q=is%3Apr+author%3Avipulgupta2048+is%3Amerged+sort%3Acreated-asc+ here])
* [https://gitlab.com/asetalias/Community-Connect gitlab/asetalias/Community-Connect] [Ongoing]
* Completed: Hacktoberfest Challenge; [https://hacktoberfestchecker.herokuapp.com/?username=vipulgupta2048 Here].
Along with this, [http://meetup.com/pydelhi Python Delhi User Group] (PyDelhi) is a volunteer-driven organization comprising of Pythonistas, enthusiasts, who have a primary focus in Python. I help by volunteering and organizing meetups for them. <br />
I am a committee member of my college’s open source community. [http://www.asetalias.in ALiAS (Amity Linux Assistance Sapience)] We conduct hands-on workshops, seminars, talks and much more to try and help as many people as we can along the process and motivate them to make their first PR.<br />
I genuinely love spreading knowledge among peers/juniors/underprivileged children wherever anywhere I can through my voice ([https://github.com/asetalias/Logistics-and-Event-Contributions/issues?q=is%3Aissue+author%3Avipulgupta2048+is%3Aclosed Talks] and [https://github.com/ILUGD/talks/issues?q=is%3Aissue+author%3Avipulgupta2048+is%3Aclosed workshops]) and my words ([http://www.mixstersite.wordpress.com Tech-Blog])
'''What is the name of your project?''' <br />
Wikiport
 
 
[[File:Logo wikiport.png|frameless|center|WikiPort]]
 
'''Describe your project in 10-20 sentences. What are you making? Who are you making it for, and why do they need it? What technologies (programming languages, etc.) will you be using? ''' <br />
'''My understanding of the problem and why it needs to be solved?'''<br />
Documentation, as mentioned before, is important to all open-source projects. Sugar Labs has all documentation for its learner applications called “Activities” at one place in the Activities#Sugar_Activities. Some problem (#risks) that I found with this sort of implementation is as follows:- <br />
# '''Difficult to add, edit and manage''' - In reference to the conversation [archive/sugar-devel] As a user, I need to create an account for editing privileges. Which is very tiresome to be very honest, hence discouraging contributions.
# '''Size issues''' - Wiki is being hosted on the a a server. With the growing number of activities, the size occupied by them will increase occupiewould l increasl leading to rise atsurge i in maintenance cost. Not to mention buying more space.
# '''No clear format/classification''' - Some links are for one activity only while other activities have multiple links for them (example Turtle Art) Results in time being wasted of the user even if they are justified.
- Found 2 wiki pages of the same activity. (this page and this too)I did extensive research on the same and found many such problems with the wiki. According to my experience in open-source, users just need information fast, easily accessible and neatly organized. <br />A new method to make that information organized should be brought in place for the betterment of the whole community. GitHub could be the solution to all our problems (Check here for reference[https://insights.stackoverflow.com/survey/2018/#work-version-control stackoverflow/survey/2018]) WikiPort is a tool that helps in migration process of wiki’s to Github hosted README.md.
'''== Objective''' == <br />
To migrate documentation and information of each activity in Activities#Sugar_Activities of Sugar Labs in MediaWiki format to their respective git repositories hosted on GitHub in rich markdown format by a program/tool with special attention to the transfer of all information and media associated with the activity with least redundancies and zero duplication of content if not already present in GitHub@README.md or user documentation in help-activity.
'''Technologies and Skills Used''' <br />
This list is just a small glimpse of technologies that I am thinking to use for this project. I am proficient in each of them, through my past projects. (Refer)
* '''Programming Language''': Python 3.6 using libraries such as requests, LXML, Scrapy, Beautifulsoup etc. Python Python3 will be used widely in this project.
* '''Markup Languages''' - Markdown, MediaWiki, reStructuredText
* '''For Automation''' - Ansible or Bash (To automate each step of the migration process and installation of the tool.)
* '''Tools''' - git, shell, MediaWiki API - Pywikibot,GitHub API (Experimental) - PyGitHub
* '''Others''' - Yaml (If Ansible is used)
'''Detailed Description of the steps mentioned '''
'''One time task''' - Use a spider Used GitHub API and PyGithub to scrape GitHub find all repository links for each activity in that don't have a README.md and are a proper JSON format (with names of each activity)under SugarLabs organisation. [https://github. IF GitHub links not found for the repository, then scripts throws back exception com/sugarlabs/sugar-docs/issues/102 Completed] and then links can be found manually[https://gist.github. com/vipulgupta2048/6eef7de43ac628e8e7ed11217fbb1803 Scripts availabe here]
*'''Step 0''' - Extracting content; from these methods that offers the least redundancies. My research yielded these results.
: 3. GET page in simple HTML ( Activities/Sugar_Network&printable=yes)
*'''Step 1 -''' Convert the data sourced from step 0; using Pandoc to Markdown. Removing syntax, comments and Wikimedia code that might be left after conversion using regex wherever required.
* '''Step 2''' - Fork and clone the respective GitHub repository from the links already scraped or found manually of the correct activity * Step 3 - Python script to find fetch all image links and then fetches download media (images) associated with the activity from the web-page; renames them in correct format; adds them to repository folder cloned under “ images” directory “images” Fix image links and testing.
* '''Step 4 -''' Ensuring content is not duplicated by being already present either in the README.md file or the user documentation in the help-activity. Make the necessary changes, fix broken links. Testing; Commit and push to fork.
* '''Step 5 -''' Pull request submitted with the detailed description of the changes in activity documentation in clear format containing all details. When PR merged, move to step 6. [Refer [https://etherpad.net/p/Sugar_Migration_Checklist Sugar_Migration_Checklist]]
* '''Step 6 -''' Use Pywikibot (Check delete.py) to delete pages/media associated with wiki page using admin privileges and link back to their repositories. This can also be achieved using the mediawiki API which saves edit history as I mentioned before. (Read more here) (End-to-end Migration of wiki page complete)
* '''Step 7''' - Repeat Step 1 for next activity.
== Other Deliverables (and Future Work) ==
# '''Contribute more''' - After and during the project, will help in solving bigger challenges and bugs faced as my experience lies in DevOps tasks and start reading code of SugarizerSugar.Already working on [https://github.com/sugarlabs/sugar-live-build/issues/6 Sugar-live-build]# '''Add Features''' - Edit history of wikis that are being migrated by WikiPort can be exported as commits on GitHub . (implementing thisExperimental).
# '''Create New Activities''' - Being well versed in Python (tried making a game with pygame) Would enjoy the new learning curve as I never really explored GTK. Have some new ideas that could be later transformed into fun, interactive learning activities.
# '''Complete Misc. Tasks''' - Wiki pages contain task waiting to be completed I would like to volunteer and finish them for the welfare of the community like this one here IRC#Wishlist.
# Remain as an '''active contributor for Sugar Labs''' and take part in discussions for the future projects. Maybe if the community agrees, also mentor someone '''myself''' in the foreseeable future.
# Use Automation to our favor so that many tasks are completed in WikiPort using bash or Ansible scripts like environment setups, installation and fine workings of the entire tool. (Aim - one command to work it all out with the least number of user interventions)“WikiPort converts all MediaWiki documents to rich markdown format and commit them to GitHub or other hosting services without losing any of your data. A tool created by Sugar Labs with love.” This seems to me like an excellent repository description of an awesome tool.I always wanted to create an open-source tool, that people would use and benefit from and take the time to contribute back. WikiPort feels likes that chance.
== Timeline ==
'''Convince us, in 5-15 sentences, that you will be able to successfully complete your project in the timeline you have described. ''' <br />
* I love writing and reading the code. I learned almost everything by reading the docs. I, later on, started writing documentation and code for others too. Helping me always know exactly where I can find the information I am looking for and how to improve on it, how necessary it is. * All my past projects, showcase a strong dedication to the deadline and the determination to get work done. WikiPort will be no different. * I have been working on Python2 and Python3 for a year now over a lot of projects and activities. * As for my skills, <br />
- Articles written, published using Pelican, Wordpress (Markdown).<br />
- I use git in all project that I start. Document and collaborate on GitHub. <br />
- Comfortable with both Linux and Windows OS, (preferring Linux all the way) <br />
- Since Python is used widely, my previous project projects justify my experience skill and skill flexibility in the language . <br />- There is no better to prove that I can accomplish and execute my vision for WikiPort by actually showing that work has been done. (hereRefer Miscellaneous section) <br />- There are many points to keep in mind while migrating is taking place, hence for a '''more methodical''' approach a [https://etherpad.net/p/Sugar_Migration_Checklist checklist] has been implemented by me which has a summary of all the - steps. (Both manual/automated) - Completed the '''one time task''' in my '''migration process''' beforehand and also solving a [https://github.com/sugarlabs/sugar-docs/issues/102 pending issue] related to it. Used '''GitHub API''' and '''PyGithub''' which seems much more effective then Scraping (using Scrapy) as mentioned before. Scripts are availabe [https://gist.github.com/vipulgupta2048/6eef7de43ac628e8e7ed11217fbb1803 here].<br />
== You and the community ==
'''If your project is successfully completed, what will its impact be on the Sugar Labs community?'''<br />
*If WikiPort is completed and deployed as a tool for this project. It would help in making the work easier for developers to not just Sugar Labs but everybody and even experienced users.
*WikiPort will help users port their wikis to GitHub with minimum user inputs, which would ultimately benefit everyone involved and for many generations of developers to come who will use that documentation.
*As previously stated, documentation is an important part of any development process. Bad documentation impacts the entire community as. But if the documentation is
1. '''Accessible''' - GitHub is one of the best places to host it. <br />2. '''Relevant and easily traversable''' - The documentation about every activity will be housed with its GitHub Repository. <br /> Clearly written - Strict format to maintain overall uniformity. <br />3. '''Easily editable (+ VCS)''' - Wiki offers edit history of documents too, but the process to edit a document is cumbersome while GitHub makes it very easy for users to fix issues in it. <br />4. '''More views, more stars''' - GitHub also increases views and popularity of the activity ultimately helping Sugar Labs to get noticed more.<br />
If our documentation has all these features, then everybody would be more inclined towards contributing to Sugar Labs and get to know more about it.
People would contribute more to bug fixing, indirectly helping the Sugar Labs community to grow and expand more. Helping students, developers, activity maintainers, members of the community.
'''Answer 3:''' James Cameron - quozl@laptop.org <br />
(His assessment of project impact from both Github and Email )<br />* "Migration is needed because Wiki is not being used now that GitHub is being used. Maintaining the Wiki in addition to GitHub is not sustainable. So we want documentation to move from Wiki to GitHub. So that when an activity is updated in GitHub, the documentation can be updated in the same place at the same time."<br />* “Originally documentation was separate because we had non-coding developers and tool chains that varied by type of developer. Now we use GitHub the tool chains are combined. With the project as described, documentation will be concentrated in the source code repository for an activity, reducing ongoing maintenance.We have less active Wiki contributors than we ever did, and in the current threat environment a Wiki requires significant monitoring and administration; we recently lost some system administrators and gained new ones; using GitHub allows us to outsource system administration.”<br />
'''Answer 4:''' Tony Anderson - quozl@laptop.org <br />
(His comments clarifying the need of WikiPort and the problems it solves)<br />As always, the question is the impact on our users. The traditional source of information for users is [1] and [2] and, especially the wiki pages. So far the effect of gitHub has been to reduce the value of these two sources. In many cases the activities on ASLO have been superseded by ones on gitHub or Github other git repository but are not available to our users. The documentation of activities on ASLO has never been adequate but now no effort will be made to improve it. This continues the trend toward Sugar being a playground for the technical elite. Complete conversation available here.<br />[1] http://www.sugarlabs.org [2] http://www.laptop.org<br />
'''What will you do if you get stuck on your project and your mentor isn't around?'''<br />
I strive to solve a problem using my own skill base by searching about the problem case and working it out into smaller bits.
#sugar and #sugar-devel channels on IRC, the mailing lists where other community members discuss issues. I am familiar with mailing list etiquettes for open source communities and learn fairly fast.<br />
Google and StackOverflow about the issues, later refer to the documentation of the tool I am using for reference.
'''How do you propose you will be keeping the community informed of your progress and any problems or questions you might have over the course of the project?''' <br />
I would get regular, constructive feedback on the code I will write and the work I would do for the community by posting work samples from Github, on the mailing list and on IRC. [Check Timeline for the same]<br />
Post constant updates on my blog and maintain an update tracker for each week on a public notepad (Etherpad, etc.)
Another Issue that I am currently working is [https://github.com/sugarlabs/sugar-live-build/issues/6 sugar-live-build/issues/]
'''Describe a great learning experience you had as a child.'''<br />Here’s a short story titled “Let me Google that for you.” when I was 14. <br />
From my childhood, web technologies have always piqued my interest for the longest time as I can remember. I been writing HTML since I was 1. But I never liked to read from the old, dusty books. When I code, I had to take help from seniors, teachers, the experience I gained and other sources. Almost everybody was irritated by the constant barrage of long and complex questions, that Vipul Gupta had.
'''Is there anything else we should have asked you or anything else that we should know that might make us like you or your project more?'''<br />
* I think I am well satisfied with the proposal that I have made for WikiPort, the project for Sugar Labs. The template given by Sugar Labs really covers all the corners of my project, and actually helped me think more constructively about the project and the fine aspects of it. <br />* I learned a lot through these few weeks, working on the proposal. MediaWIki API and new skills, contributing to an open-source organisation of this stature, how to interact with people. This has been a memorable experience. I am in awe, I really am.<br />* All open-source communities may not be same, but are run on the same principle that I also try to follow in life. Be humble, and helpful to everybody. The ‘ahaa-moment’ of a beginner is what I live for, when he truly get the point that I am trying to make.<br />* Some Credits - Learned to not contest for issues, but focus on better PR’s, and many new things from James Cameron (@Quozl)’s blog. Jasikrat too for helping people on IRC emailsll mentors who review our proposal and go through all our mails. Thank you all for the opportunities and the help provided to me and my fellow contributors. I hope down the line, I could follow in your footsteps too.<br />
__NOEDITSECTION__
__INDEX__
[[Category:2018 GSoC applications]]

Navigation menu