Community Wishlist Survey 2019/Archive/Low-bandwidth editing app
Low-bandwidth editing app
- Problem: Many people now have access to smartphones and data storage, but not affordable high-bandwidth connections. This is common in the third world. I have also experienced it personally as my work has sometimes taken me to remote areas for periods of a few months. Under these circumstances, I naturally used offline copies of Wikimedia content. Even when I am somewhere with a cheap high-bandwidth connection, I now often browse my offline copies, especially for political content (I don't want tracking of my political interests to be used to target election ads to me) and dictionaries.
The problem is that I then don't edit the wikis. If I see some error, I rarely bother to re-load the page through another interface and fix it, even if I have a cheap high-bandwidth connection right there waiting. If I have an expensive connection (for example, connections via satellite or HF radiophone), I don't even consider editing. If I have no connection, I don't edit, and I rarely remember to go back and fix content when I next have a connection.
- Who would benefit: Any editor or would-be editor with a low-bandwidth or expensive connection, or using offline wiki content for other reasons. A lot of people in developing countries who are underrepresented among editors.
- Proposed solution: A mobile/desktop app allowing me to load wiki content from local storage, but optionally (with a single click) sync a single page with the up-to-date version online. The synced page should then be editable. Ideally, the app would save edits made when there is no connection so that they can be uploaded when the editor next has a connection. Wikipedia edits which use the standard markup are plain text, and thus very low-bandwidth. The app could thus be very cheap to use over even an expensive connection.
Obviously the local storage should be updatable with a sync (download of differences) rather than a full re-download, too.
- More comments: Sorry to submit this at the last moment after withdrawing my former third proposal, but I just thought of it. May overlap with Internet-in-a-Box; Doc James, could you comment?
- Phabricator tickets:
- Proposer: HLHJ (talk) 17:27, 10 November 2018 (UTC)
Discussion
- I think this would be cool. It would be most useful with the offline apps. Could be a little technically complicated to implement. How would one prevent edit conflicts? Doc James (talk · contribs · email) 22:56, 10 November 2018 (UTC)
- Thank you, Doc James. I'm a bit worried that it might be difficult to implement, too, but there's some really good copyleft syncing software out there already... comments from devs on this would be welcome. I think one would deal with edit conflicts in exactly the same way one does now. If you left it too long to upload your changes, you'd be more likely to have a mess you needed to sort out manually. But I've edited plenty of pages which haven't changed for years. HLHJ (talk) 00:24, 11 November 2018 (UTC)
- For those not familiar with the idea of syncing, it goes like this (character counts are made-up):
Anthropomorphized example - App on phone: I have the 10:39, 5 December 2016 version of the article en:Bayo Ojo.
- WM server: That's still the most recent version. You are up-to-date.
- App: And I have the 01:38, 13 October 2016 version of en:Kanu Godwin Agabi
- Server: Oh, I have a more recent version from 02:26, 10 November 2018. Just one more edit than your version; Internet Archive bot rescued some dead links. Please go to character X of your version, delete the next 112 characters, and insert this text: "[text of updated citation]". Then go to character Y, delete the next 117 characters, and replace them with "[text of second updated citation]". Then your local copy will be up-to-date.
- App: Done. I now have the 02:26, 10 November 2018 version of the en:Kanu Godwin Agabi article. I want to update it. Please go to character 1357, remove the next six characters, and add the text "[reference that replaces {{cn}} tag]". Then go to character 40 and add this sentence: "[sentence with new information, followed by ref]".
- Server: Done. The page is updated with your edit of [timestamp]. I am up-to-date.
- App: Oh wait. At character 47 of the [timestamp] version of the en:Kanu Godwin Agabi article, please replace "Agbai" with "Agabi".
- App: Server?
- App: Server, are you there? Are any servers there?
(App decides that the network has gone down again, and saves the edit locally. It also saves another edit to the en:Bayo Ojo article locally)
- Network, three days later: I'm baa-aack!
- App: Good. Wikipedia server?
- Server: Yes?
- App: I now have the [timestamp] version of the en:Kanu Godwin Agabi article.
- Server: That's still the most recent version. You are up-to-date.
- App: At character 47 of the [timestamp] version of the en:Kanu Godwin Agabi article, please replace "Agbai" with "Agabi".
- Server: Done. The page is updated with your edit of [another timestamp, three days later]. I am up-to-date.
- App: I now have the 10:39, 5 December 2016 version of the en:Bayo Ojo article.
- Server: There's been an edit to that at [third timestamp]. Here are the changes you need to make to your copy.
- App: OK, updated. At character 570 of the [third timestamp] version of the en:Bayo Ojo article... Oh wait, I have an edit conflict. Human, please revise this edit.
(human does so, but not until after supper, three hours later)
- App: I now have the [third timestamp] version of the en:Bayo Ojo article.
- Server: That's still the most recent version. You are up-to-date.
- App: Please make the following changes to the the [third timestamp] version of the en:Bayo Ojo article (describes changes)
- Server: Done. The page is updated with your edit of [another timestamp]. I am up-to-date.
- Note that human attention is only required for the stuff after the network comes back if there is an edit conflict. Obviously this is not done in natural human-readable language without data compression, so it's really even more concise. Even in this form, though, it's a lot shorter than sending even one entire copy of both articles, so you transmit less data and pay less to your communications network. HLHJ (talk) 16:18, 11 November 2018 (UTC)
- Seems like WikiFundi/en. Blue Rasberry (talk) 22:52, 11 November 2018 (UTC)
- It does. Thank you very much for the reference, Bluerasberry, I had no idea. "We are working on the next version with the technical support of Kiwix... The most exciting element is that there are discussions ongoing to create a synchronization system... Finalized articles have to be retrieved manually by local facilitators to be imported on Wikipedia or Vikidia, which is obviously not very practical."[1] (quote by Anthere, from last May; Islahaddow is the second co-creator). If I've understood right, it looks as though WikiFundi uses content stored on a local intranet, and (currently manually) syncs the intranet to the WM servers. Target users are schools and similar organizations. So this suggestion is almost exactly a one-person version of WikiFundi. The hierarchical-wiki version makes sense, though, as it would save on hardware costs (although it would sometimes make edit conflicts from more awkward). There are really two problems here; low-bandwidth connectivity (on price or capacity grounds) and intermittent connectivity. HLHJ (talk) 23:58, 11 November 2018 (UTC)
Hello. Indeed, you have a bit reinvented WikiFundi :) Looks like we need to communicate more about it ;)
So, key points
- First version was launched in Jan 2017. We JUST finalized V2 (last successful testing was Friday, so really, just finished).
- The two places where you can find the most info about it is m:WikiFundi and the website http://www.wikifundi.org. I warranty those are up to date because we actually updated those last Friday...
- Dev was done by Emmanuel for the V1 and with Kiwix for V2. We do not do sync but it would be absolutely LOVELY to have a system to do sync... So any technical solution you can come up with... I am interested.
- We plan to communicate about V2... this week or next week. Newsletter to our contacts, the offline mailing list for the Offline Projects and Wikimedians for offline wikis. WMF blog probably. On Twitter and Facebook. Wikimedia-l as well I guess. If you have other interesting venues, please tell me. I would recommend you join the offline mailing list.
- We actually use that WikiFundi stuff... for example, I am currently working on this : https://fr.vikidia.org/wiki/Projet:WikiChallenge_Ecoles_d%27Afrique. It already operated last year and will operate in 2019 again.
- There is a bi-monthly meet up on about async wikis, which is... exactly what you seem to be interested in ;) Next meeting in tomorrow. Please get in touch with David Strine (dstrine BIP wikimedia.org)
- I also invite you (strong recommandation) to check out this project grant request Grants:Project/Rapid/MarkAHershberger/MABS: Multilateral, Asynchronous, Bidirectional Synchronisation (of wikis) and get in touch with Mark ?
Anthere (talk) 11:11, 12 November 2018 (UTC). Motif User:Islahaddow
- Thank you, Anthere. I think I'm a bit late on the monthly meet-up this month. It looks as if MarkAHershberger's grant application for... V3? 4?... is fairly directly what is suggested here. I'd encourage anyone reading this to go endorse the grant proposal, as I have done (it doesn't count towards your votes!). Can this Wishlist proposal be adapted to be useful to this project? I do not know enough about these political structures to know. HLHJ (talk) 03:19, 14 November 2018 (UTC)
- Actually... I was one week ahead :) This is next Tuesday... Let me point the offline people to your wishlist. Anthere (talk)
- This is awesome. I'm glad to see you taking an interest in MABS. With this sort of encouragement, I am more motivated to see if we can produce something that others will be able to use. --☠MarkAHershberger☢(talk)☣ 15:41, 15 November 2018 (UTC)
Hi HLHJ and all: It sounds like you're making friends and connecting to the related work that people are doing. I'm going to take this proposal off the Wishlist Survey, because I think you're already talking with the folks who'll be able to work on these ideas. Thanks for participating in the survey! -- DannyH (WMF) (talk) 19:20, 15 November 2018 (UTC)
- MarkAHershberger, can this proposal be converted into anything of use to your work? Or would it be best for your work to discontinue the proposal? HLHJ (talk) 02:40, 16 November 2018 (UTC)