Grants:IEG/Improve 'Upload to Commons' Android App/Renewal/Midpoint

This project is funded by an Individual Engagement Grant

This Individual Engagement Grant is renewed

Welcome to this project's midpoint report! This report shares progress and learnings from the Individual Engagement Grantee's first 3 months.

Summary

In a few short sentences or bullet points, give the main highlights of what happened with your project so far.

The current version available for download on Google Play and F-Droid is v2.6.7 - this includes a new login screen, improved navigation drawer UI with username included, more information in upload screen, critical bug fixes for the list and map of nearby places that need pictures, optimized app performance, and fewer crashes.
Features that have been implemented in our codebase but have not been released to users (because we are still working on ironing out remaining bugs): new UI and direct uploads for nearby places that need pictures, and two-factor authentication login.
A total of 11,766 distinct images uploaded via the app have been used in Wikimedia articles, and the app has 3477 active installs on Google Play.
Our community of developers continues to grow - in the last month, 21 volunteers submitted successful pull requests (code changes) to our app. Many of the volunteers were new to Wikimedia projects.

Methods and activities

How have you setup your project, and what work has been completed so far?

Describe how you've setup your experiment or pilot, sharing your key focuses so far and including links to any background research or past learning that has guided your decisions. List and describe the activities you've undertaken as part of your project to this point.

Methods

General workflow

As usual, all of the app's source code is available on GitHub, which is also used for version control, collaboration, and issue tracking. Each member of the team submits their work through pull requests, which are then reviewed and tested by a second member of the team before being merged.

New versions of the app containing new features and bugfixes are released to Google Play on a regular basis - first to beta testing, and then if all goes well, to production (all users). Beta testing is open and anyone who wishes can register for it. Releases are also done on F-Droid (a repository for free and open source Android apps) via an automated process from our GitHub repository.

As each release is rather time-intensive (needs to be tested extensively and all known bugs ironed out before it is ready to be rolled out to users, and additional documentation written), we unfortunately are unable to do a release for every single new feature, so there may be some lag time between a feature's implementation on GitHub and its availability to users. However, new major releases (releases that include new features) are rolled out with an average frequency of once every 1-2 months. Minor releases (hotfixes aimed at fixing a bug or crash in the current version) are rolled out as needed, possibly as often as once every few days.

When a new major release is rolled out, a post is made on relevant mailing lists, and our blog updated. Feedback is received from users via Google Play reviews, our Google groups forum, and our GitHub issue tracker.

Project-specific

We are happy to report that we have devised an effective way of working together and communicating as a distributed team with differing timezones. Methods that we used for team collaboration include:

A Gantt chart web app to track the progress of each team member
A GitHub project to organize task assignments and details
Google Hangouts to communicate on a daily basis.

Given that this project was of a substantially larger scale than our previous grant, we also spent a lot more time planning the new features before delving into implementation. For all major new features, we:

Discussed detailed scope and use cases for each feature
Drafted mock-ups for new UI

In all cases, the discussions involved not just the 3 paid team members, but also the larger community of volunteer contributors and users who are interested in participating.

Tasks completed

In production (released to users on Google Play)

The current version available for download on Google Play and F-Droid is v2.6.7.

The new features and bug fixes in that version include:

New privacy policy, About page and store listing, in order to fulfill legal requirements for keeping the app on WMF's Google Play Store account. As requested, instead of using WMF's privacy policy, we drafted an individual privacy policy for our app. The About page of the app and the app's store listing also contain a disclaimer written by WMF's lawyer.

Improved the layout of the navigation drawer and displayed the user's username, so that users know which account they are logged in to at the moment.

Improved the appearance of the login screen based on Material design guidelines.

Improved the upload screen by: (1) Adding 'i' buttons next to upload fields (e.g. title, description, etc) to educate new users on how to appropriately fill in those fields, and (2) reminding user that their picture must adhere to Commons policies (linked).

Improved how the user's location is updated in "Nearby places that need pictures" to prevent issues with users getting empty Nearby maps or lists despite their GPS being turned on, and fixed a bug where place descriptions were showing up as "?" despite a description/label being present for that Wikidata item.

Optimized app performance by reducing battery usage and memory leaks.

Discovered a new, very prevalent issue with Dagger library usage in v2.6 that was causing many users to crash repeatedly. We released several iterations to try to debug and eventually fixed it (hence the latest version being 2.6.7).

New login screen
Navigation drawer with username
New upload screen with info buttons and Commons policies
"About" page with disclaimer and new privacy policy

Pre-release (available in codebase but not in Google Play Store)

These features have been implemented and are available in our code repository but have not been released to users yet. There is a high overhead associated with a Google Play Store release - it requires that all known bugs with the new features be ironed out, that extensive testing has been performed on several different APIs and devices, and a good amount of documentation/administrative work. Due to that, we tend to accumulate a few new features before doing a release. We estimate that the features below will be released to Google Play for beta testing in 1-2 weeks' time, after we have sorted out the remaining known bugs.

Overhauled UI of "Nearby places that need pictures" in preparation for direct uploads. We have overhauled the UI of Nearby Places, utilizing a hybrid list + map view that makes it easier for users to switch between the two views. When the user selects a location on the map, they will be presented with a compact view of the place's details in a bottom sheet, similar to the style used in Google Maps. Swiping up brings the user to a more detailed view, with more options - directions, viewing the Wikidata item, viewing the Wikipedia article, etc. A floating "+" action button on the side of the bottom sheet allows users to upload a picture from their camera or gallery for that particular Wikidata item. The list view also allows users to expand a particular location to view the same options.

Hybrid list + map view
Bottom sheet of map view
Expanded item in list view

Implemented two-factor authentication (2FA) login. Two-factor authentication logins have been implemented and tested. Users should be able to login with both 2FA and non-2FA accounts in the next release.

Two-factor authentication login screen

Improvements for developers

Implemented dummy uploads to allow developers to manually test code without having to submit a real Commons picture each time the upload feature requires a test. This was done by creating a new "beta" build variant of the app, which would send uploads to the beta Commons server instead of the actual Commons server.

Improved documentation for volunteers on our GitHub wiki via various contribution guides and templates.

Other activities

Created a new Facebook page to promote the app and keep users in the loop with new updates.

Created projects for our app for the upcoming GSoC and Outreachy round with the hope of introducing new contributors to free and open source software - Project 1 and Project 2. All 3 grantees and 1 advisor have volunteered as mentors or co-mentors.

Tasks in progress

These features are currently in progress, with some parts of them completed and available in our code repository on GitHub, but other components still being worked on. They are not available on Google Play or F-Droid yet.

Direct uploads from "Nearby places that need pictures"

Building on the new Nearby Places UI, the user can now upload a picture from their camera or gallery for that particular Wikidata item. After a picture has been taken (camera) or selected (gallery), they are taken to the title & description screen as usual, which pre-fills the title and description fields with the title and description of the Wikidata item (the user is still able to edit them).

We are currently working on implementing category suggestions based on the categories associated with that Wikidata item.

Display Commons user talk notifications

We have successfully implemented a "notifications" screen that is accessed via the app's navigation drawer, where users can view their Commons user talk notifications in a list view, from newest to oldest. However, we are still working through a few snags, notably distinguishing deletion nomination messages from other types of messages, and allowing users to tap a notification to view the full message in their talk page.

Pre-filled title and description fields from direct uploads
Prototype of notifications screen

Unable to complete

After several failed attempts and rigorous discussion, it was mutually agreed with WMF's engineers that implementing OAuth for our app was not possible with WMF's current version of OAuth, and thus that requirement was waived.

Midpoint outcomes

What are the results of your project or any experiments you’ve worked on so far?

Please discuss anything you have created or changed (organized, built, grown, etc) as a result of your project to date.

Number of images used in Wikimedia projects

One of the measures of success listed in our proposal was "a total of more than 11,000 distinct images uploaded via the app are used in Wikimedia articles". As of 7 March 2018, the GLAMorous query for the number of images uploaded via the app that were used in Wikimedia articles displayed 25214 total images used and 11,766 distinct images used. We are very happy with this particular result, especially as it is a good indicator of the quality and usability of images uploaded via the app, not just quantity.

Number of active installs on Google Play

The graph below displays the number of active installs (number of Android devices that have been active in the previous 30 days on which the app is currently installed) from Oct 2017 to March 2018. As of 7 March 2018, the number of active installs is 3,477. Growth is fairly regular, which is to be expected as we have been focused on implementing the new features and have not attempted much additional publicity. A dip can be seen from Dec to Jan and at the end of Feb, both of which correspond to an issue that we were experiencing with high crash rates and which has since been fixed (more about that in the "Challenges" section).

Number of files uploaded vs files deleted per quarter

The Commons app stats tool (created by Yusuke) allows us to visualize the number of uploads made using our app, as well as the number of files uploaded via our app that were deleted. The histogram below was taken on 7 March 2018, with the last quarter (2018 Q1) still in progress. The ratio of uploaded files vs deletes has improved slightly over the past few quarters; we are hoping to have encouraging results when we retrieve the numbers at the end of the grant.

New contributors

Alongside the major changes and new features mentioned in this report, there are dozens of other small bugfixes, enhancements, and codebase improvements that have been contributed by volunteers. We are honored to have so many volunteers who have invested their time and effort into our app in various ways, both in terms of non-technical contributions (e.g. helping with our Facebook page updates and beta testing), as well as technical contributions (bugfixes and enhancements to the app via pull requests on GitHub). We have seen the number of technical contributors grow over the past few months, especially when our project was put up for GSoC/Outreachy.

The screenshot below was taken from GitHub's pulse - in the last month alone, 24 people made commits (code changes) that were merged into master (the main development branch). 21 of those people were unpaid volunteers (the other 3 were, of course, grantees). Many of the volunteers were new to Wikimedia projects, so it seems that our app could be a good entry point for bringing budding developers into the Wikimedia ecosystem.

Finances

Please take some time to update the table in your project finances page. Check that you’ve listed all approved and actual expenditures as instructed. If there are differences between the planned and actual use of funds, please use the column provided there to explain them.

Then, answer the following question here: Have you spent your funds according to plan so far? Please briefly describe any major changes to budget or expenditures that you anticipate for the second half of your project.

Yes, we have spent our funds according to plan so far, and we do not currently expect any major budget changes for the second half of the project.

Learning

The best thing about trying something new is that you learn from it. We want to follow in your footsteps and learn along with you, and we want to know that you are taking enough risks to learn something really interesting! Please use the below sections to describe what is working and what you plan to change for the second half of your project.

What are the challenges

What challenges or obstacles have you encountered? What will you do differently going forward? Please list these as short bullet points.

What challenges or obstacles have you encountered?

Quality control of an active open source project. As mentioned above, we are immensely grateful to the volunteer collaborators who have contributed many of our enhancements and bugfixes that are present in our app. However, such a collaborative environment raises some issues - who is responsible for fixing major crashes/bugs that are found? The volunteer who wrote the code that led to it does not always do so, which is fair, as he/she certainly isn't obligated to, but this means that it gets tagged on to the already-full schedule of one of our paid team members. Our team spent many hours isolating and fixing a few bugs that slipped past our tests and caused high crash rates over Dec to Feb, which put us behind schedule for our grant tasks. We also lost roughly a hundred users due to this.

What will you do differently going forward?

We have added GitHub templates for pull requests requesting additional tests from contributors to increase the chances of a bug being found before the pull request is merged, but this in and of itself is unlikely to completely fix the problem, especially due to the many devices and API levels running Android. We have also introduced a new release workflow that makes it easier to apply hotfixes to a release independently of other changes, but while that does reduce the amount of time needed, it still requires finding the source of the bug and implementing a hotfix. If we apply for future grants, I plan to either allocate more time for the main (paid) team to fix not just bugs introduced by their own work but also those which are introduced by volunteers' work, or to add an additional team member who will be responsible for much more frequent/extensive testing and QA than we can afford with our current resources.

What is working well

What have you found works best so far? To help spread successful strategies so that they can be of use to others in the movement, rather than writing lots of text here, we'd like you to share your finding in the form of a link to a learning pattern.

Your learning pattern link goes here

Next steps and opportunities

What are the next steps and opportunities you’ll be focusing on for the second half of your project? Please list these as short bullet points.

Release v2.7 so users are up to date with all the improvements currently in our codebase
Complete the Nearby Places direct upload workflow by adding the image to the associated Wikidata item's P18 property, and logging the action
Finish implementation of user's real-time position on map of nearby places
Fix issues with user talk notifications
Implement showcase of featured images
Allow multiple uploads from within the app, and write corresponding upload tests
Overhaul main UI in accordance with mockup shown in proposal
Introduce additional check during the upload process to prevent file overwrites

Grantee reflection

We’d love to hear any thoughts you have on how the experience of being an IEGrantee has been so far. What is one thing that surprised you, or that you particularly enjoyed from the past 3 months?

Many of the WMF staff have been incredibly helpful to us throughout our journey; we are especially grateful to Janice, Dmitry, Dan, and Corey for going above and beyond the call of duty to assist us with various problems that we faced.

I have never ceased to be amazed by the wonderful community that has built itself around the app, both in terms of contributors/volunteers (technical and non-technical), as well as users who offered suggestions, bug reports, and kind words. We are extremely honored to find that so many people consider this project to be worthy of their time and effort. Thank you all - it means a lot to us.