Community Tech/E-Book Export Reliability
This page documents a project the Wikimedia Foundation's Community Tech team has worked on or declined in the past. Technical work on this project is complete.
We invite you to join the discussion on the talk page.
For years Wikisource users have complained about the unreliability of WSexport, the tool that turns Wikisource works into downloadable e-books. During the 2019 Community Wishlist Survey, Wikisource users rallied behind a wish to improve export of electronic books, making it one of the year's top vote-getters. The wish begins with this description of the problem:
Imagine if Wikipedia pages could not display for many days, or would only be available once in a while for many weeks. Imagine if Wikipedia displayed pages with missing information or scrambled information. This is what visitors get when they download books from...Wikisource.
In April of 2019, the Community Tech team began working on a project to answer Wikisource users' wish by improving the reliability of WSExport. This page tracks our progress toward that goal.
Follow this project and let us hear from youEdit
- Add this page to your Watchlist by clicking the star in the tabs above.
- Watch the project Phabricator board by going to E-Book Export Reliability and clicking “Watch Project”.
- To follow any task on the Phabricator board, click on it to open the task. Then click the “Subscribe” link at top-right (here’s how to get a Phabricator username).
- To tell us what you think, please post comments on project talk page.
June 26, 2019Edit
The team has released a series of improvements to WSexport, which increase the reliability of the tool. These changes resolve many common causes of export failure. To assess the success of this project, we have tracked WSExport reliability over time. Below, we would like to share some of these figures with you.
Here are the uptime stats (June 20th, 2019):
- Past 24 hours: 98.47% uptime
- Past Week: 99.2% uptime
- Past Month: 99.42% uptime
Here are the downtime stats (May 1st to June 15th, 2019):
- June 1-15: 179 minutes downtime
- May 16-31: 50 minutes downtime
- May 1-15 (before improvements): 941 minutes downtime
This project is now marked as complete. If you would like more details on the stats, please visit T226136. We would love to hear your thoughts. Finally, thank you to everyone who contributed to this project!
First priorities (April 3, 2019)Edit
The team has carefully reviewed the winning Wishlist wish and the Phabricator tickets and GitHub issues related to WSexport. As we did, it became clear that our efforts on this project will best be devoted to two main goals:
- #1) Increase WSexport reliability and uptime: The record is full of reports from users that are all variations on the same theme: Conversion to pdf/mobi/txt failed, Server error: 503 on some books, Unable to use wsexport ...due to 502 bad gateway error.... The team’s first job will be to determine the probable causes of these export failures and to address as many of them as possible so that we can keep WSexport up and running. (Here's a list of the reports we've identified as being clearly related to this goal.)
- #2) Make sure e-books are legible: The Wishlist wish for this project references the problem of “pages with missing...or scrambled information.” Our second priority will be to address as many issues as we can that cause the contents of a downloaded book to be unreadable or incomplete.
There are many other improvements to WSexport that readers and Wikisource editors might wish for—new features, improved formatting, new fonts, etc. These are valid goals. But CommTech's time on any one Wishlist wish is limited, since we are tasked with addressing the top-ten wishes from each year’s survey. Given that, I hope you’ll agree that the two goals listed above should take priority.
We want to hear your thoughts about this project, particularly if you have ideas about what’s causing the unreliability of WSexport. We’re listening.
Reaching out to people who voted for the wish to Improve E-Book ExportEdit
- I’ve pinged you because you voted for the Wishlist wish Improve export of electronic books. I wanted to let you know that CommTech has started working on the project. This is the project page, and I’ve just posted to it a summary of what the team sees as project priorities (above). Please watch this page for future updates. We will rely on the Wikisource community for input and advice, so thanks in advance for your help.
- @Consulnico:, @Akme:, @Wuyouyuan:, @Le ciel est par dessus le toit:, @Patrice Dargenton:, @Noé:, @Peter Alberti:, @Viticulum:, @Hector:, @Liuxinyu970226:, @Satdeep Gill:, @ਨਿਸ਼ਾਨ ਸਿੰਘ ਵਿਰਦੀ:, @Harshaan Ghuman:, @Simranjeet Sidhu:, @Jagvir Kaur:, @Kaviraf:, @Zabia:, @Richard Nevell:, @VIGNERON:, @Benoit Rochon:, @Jeb:, @Yodaspirine:, @Amqui:, @Pepe piton:, @Tpt:, @Aristoi:, @Ninovolador:, @Thurs:, @Laurent Jerry:, @Pymouss:, @Wieralee:, @Ankry:, @Bodhisattwa:, @Llywrch:, @Bzhqc:, @Geraki:, @Dorades:, @Derfgiuliano:, @Nalou:, @TheDJ: JMatazzoni (WMF) (talk) 21:17, 4 April 2019 (UTC)
- @Emptyfear:,@Sadads:, @Jmatazzoni:, @Hsarrazin:, @Ernest-Mtl:, @Zdzislaw:, @Lostinlodos:, @Yomomo:, @Acer11:, @Barsetti46:, @Candalua:, @Toto256:, @Shev123:, @Thekidpossum:, @Thibaut120094:, @Lyokoï:, @And.martire:, @Salicyna:, @Pamputt:, @Automatik:, @Alex brollo:, @Vigno:, @CAPTAIN RAJU:, @JAn Dudík:, @Vulphere:, @Conny:, @Novak Watchmen:, @Ayack:,@S1.pl:, @Gce:, @Library Guy:, @SshibumXZ:, @MartinPoulter:, @Balajijagadesh:, @Jayantanth:, @Flo Beck:, @Manseng:, @Siddhant:, @Tpe.g5.stan: —JMatazzoni (WMF) (talk) 21:53, 4 April 2019 (UTC)
- @Chrisric:, @Cantons-de-l'Est:, @*j*jac:, @Sahaquiel9102:, @Slowking4:, @Vieux têtard:, @Sannita:, @Odder:, @Acélan:, @Nahum:, @SyB~Anicium:, @Seb35:, @Yland:, @Micru:, @Ranjithsiji:, @AfroThundr3007730:, @PMG:, @Mathieugp:, @Mx. Granger:, @Trizek:, @Nemo bis:, @ManuD:, @Zyephyrus:, @JLTB34:, @Abecido:, @Bibar:, @Saig Keryann:, @NickK:, @Edhral:, @OrsolyaVirág:, @.mau.:, @Ciampix:, @Pyb:, @Arcyon37:, @Mauricio V. Genta: —JMatazzoni (WMF) (talk) 22:11, 4 April 2019 (UTC)
- This is a very important problem for all of us... The server is almost always too busy and our books can't be downloaded. A reader can try once or twice... but when he got always error 404 he will never come back... Wieralee (talk) 08:44, 10 April 2019 (UTC)
- Mark Graham of internet archive was astonished that the functionality was not available. Slowking4 (talk) 10:00, 12 April 2019 (UTC)
- Wishlist proposal, discussion and votes
- E-Book Export Reliability on Phabricator ticket
- WSexport documentation on Wikisource
- Access the WSexport tool directly on Toolforge (WMFlabs)