Community Wishlist Survey 2023/Edit-recovery feature
Auto-save feature now known as Edit-Recovery Feature was the #8 wish in the Community Wishlist Survey 2023. This feature saves wikitext and other edit form information while typing, and allows for restoring it after the browser has been accidentally close, or a power or network outage or browser crash. Requests for this functionality have been made more than once.
|Recovering unsaved edits that are lost while editing.|
|Team members:||Joydeep Sengupta, Dayllan Maza, Harumi Monroy, MusikAnimal, Sam Wilson, Karolin Siebert, Sandister Tei|
|Lead:||Joydeep Sengupta, Dayllan Maza and Karolin Siebert (Product Owners)|
This page outlines Community Tech's approach to addressing the needs expressed in this wish's proposal, as well as considering previous discussions around the request.
Please leave feedback if you have any.
Background & problem space edit
Additionally, in 2012 a ticket was submitted for this auto-save feature in the localStorage and a patch was started. localStorage size was limited to about 5MB and could only contain only strings; the patch was abandoned in 2017.
There has been a discussion that sessionStorage is lost when you switch to a new tab so that localStorage would be an improvement. The trouble with localStorage is that there's a size limit, and multiple autosaves would be likely to exceed it.
Another option is to store the autosave data server-side, where there would be no size limit and recovering data could also be done on a separate device.
In 2022, Community Tech opened an investigation to investigate the StashEdit API as a possible solution. This would mean an autosave lifetime of somewhere in the order of 5 minutes to half an hour. Alternatively, a new database table could be created, in which case the lifetime could be up to 90 days.
Implementation options edit
For this wish, we looked at a few different ways of implementing it. A primary distinction in these was the location of the auto-save data: if it's stored only on the user's device (what we're calling 'client-side'), or transferred to the Wikimedia servers ('server-side'). See below for the legal concerns relating to the former.
- Building upon edit stash — time limit insufficient
- While a user is editing a wiki page, every three seconds the page text is sent to the server as an 'edit stash'. This feature exists so that when the page is finally saved it is quicker to view because the wikitext has already been rendered to HTML. We wondered if we could leverage this feature by making it possible to retrieve the stashed wikitext when re-opening an edit session. This would involve a) increasing the time that the stashed edits are stored for (currently only five minutes); b) changing the stashedit API so that it'd return the wikitext given the unique hash of the content; and c) storing that hash in localStorage in the browser. Apart from not being a great idea to hack an existing feature to do something quite unrelated, this method would be limited to storing the autosave data for something around half an hour, which is not enough.
- New database table and API — complicated, privacy concerns
- A whole new database schema and API could be created to store the autosave data. This would be the most flexible and powerful, allowing autosaves to be linked to users and stored indefinitely (much as Phabricator does it). However, with that comes a lot of complexity in devlopment, as well as concerns about data access (see below for the legal issues), and the worry that this would become a 'drafts' feature by another name. The data retention time would be limited to 90 days.
- Fully client-side — private, guards against connectivity loss, simpler
- Browsers have three main mechanisms for storing data: sessionStorage, localStorage, and IndexedDB. The first has a reasonably large storage quota but applies only to the 'page session' so is deleted after you close a tab. The second has a much smaller quota which is shared between all of a wiki's usages of localStorage and so also isn't suitable. The last, indexedDB, is most useful, as it allows for lots of data to be stored and persists even if the browser is closed or crashes etc. This is the most promising system to use for autosave, although there are still issues to be considered (such as how to reliably delete the data after it's no longer required).
The other aspect of implementation that we considered was where to build the feature: in MediaWiki core, or in an extension (either new or existing). We settled on Core, for a number of reasons:
- There was no existing extension that was an obvious good fit for Edit Recovery, so the option was to create a new one. An extension would be suitable if the codebase will become large or have other dependencies, but we didn't think this very likely. It could also make deployment more complicated, as it'd have to undergo security review and the full new-deployment process.
- MediaWiki Core
- Features added to core should be applicable to every MediaWiki installation (including non-Wikimedia ones), regardless of what extensions it has installed. They may also be used by extensions, and so having them in core means that those extensions don't need to have extra dependencies on other extensions.
Legal considerations edit
In previous work around this topic, concerns have been raised about the possibility of people using a private data storage feature for undesirable purposes (for example, sharing an account username and password and then passing data via the autosave feature).
Community Tech asked WMF Legal to weigh in on this, and their analysis was that there are not any strict legal reasons to not build such a feature, but that a few points need to be kept in mind:
- A time limit on the time that the data is stored, with a maximum of 90 days (in line with other private data retention within Wikimedia).
- A way for Trust and Safety to access the data for a given user, if a court requests it (this is common in other systems with this sort of feature, such as email drafts).
- For all of a user's data to be able to be deleted if they request it.
These concerns do not apply if the data is stored client-side, because the data stays on the user's device and is their own responsibility.
Not a 'drafts' feature edit
One aspect we're very clear that we want to avoid is that it end up being a system for maintaining private drafts of pages. There is a long history around that topic, but the autosave wish is specifically around shorter-term recovery of in-progress edits.
Feature details edit
Summary: It is possible to close an in-progress edit form (via closing the window, the browser crashing, etc.) and have all data restored when you re-open that same page for editing again (at any later time).
- A page is identified by its title (so if a page is moved during an editing session, the recovery happens at the old name).
- A section of a page is identified by its number (so if a section is added or removed by a different user, the recovery might happen for the wrong section, or not happen at all). It's not possible to edit a section of an old revision.
- If a page (or section) is opened for editing and there is saved recovery data, it is loaded into the form.
- The edit-recovery data (for a page and all of its sections) is deleted in a few different situations:
- after a page is published;
- when the Cancel link is clicked;
- when the user logs out;
- when the user switches from wikitext to visual editing (Visual Editor has its own edit-recovery system);
- After edit-recovery data is loaded the page will show the normal 'are you sure' warning when the user tries to navigate away (even if no further changes have been made).
- Still to be decided: If an editing session is underway and the user navigates away and goes to edit an old version of the page, then the recovery data is still loaded.
October 25, 2023 – Edit-Recovery is now available for testing in Beta edit
Hello community, we have some updates. Edit-Recovery wish (formerly known as Auto-save feature) is now available on Beta Cluster, and you are invited to test it.
Start editing any page on any Beta site, for example simple.wikipedia.beta.wmflabs.org, but don't publish your change. Wait 5 seconds and close the tab. Reopen the tab. Your edit should be recovered!
We are working to make the feature more visible with an element like a toast notification when restoring edit data, with the option of discarding the recovered data.
July 4, 2023 – Investigations and your input edit
The CommTech team is reviewing any investigations, discussions, patches that have happened around this wish to determine what is next.
We have would like you to answer some questions as follows:
- How long do we need to save the data for the auto-save functionality (Keeping in mind legal matters, as legal implications are reduced by reducing the amount of time we store edits)?
- What should we store in the database to be able to make autosave functionality work? I.e:
Please leave your responses on the talkpage if you have any.