Grants talk:PEG/Offline MediaWiki search for NASA and Medicine

Evaluation by the GAC

GAC members who oppose this request

1. MADe (talk) 08:19, 22 December 2014 (UTC), see below

GAC comments

Comments MADe

Can you explain the link with the Wikimedia Foundation? For further discussion, we would also need more info about the costs involved... "Offline Sync extension development: $10,000", that's not really helpful for a discussion by the GAC... MADe (talk) 22:06, 15 December 2014 (UTC)

• The software will be available for all mediawiki installations as an extension. Although Wikimedia does not benefit directly from the other mediawiki installations the first implementation of the extension (outside of WikEM) will be with NASA. The wikimedia Foundation will benefit directly due to the media coverage regarding mediwiki installations being use for organizing information at NASA and the demonstration of an offline wiki for astronaut training. Also as an extension any other mediawiki installation can make use of the code base.

Dan Ostermayer, thanks for your reply. I have some concerns about the grant request. There's no budget table or overview of how the 2x 60h will be spent. How was the wage of 60$/h determined (that is 480$/day, imho on the high side for IT consultants). This is important for us, just look at how the total budget increased with 5k$ overnight...

Secondly, imagine we support the 30 k$ budget request. What's in it for the Foundation? The gains are only indirect. A seperate website is used, a separate dev team, explicitely no meetings with Wikimedia volunteers are planned so also no knownedge sharing ...

Lastly, I see a lot of opportunities to collaborate (Wikimedia is in real need for some skilled EM checks of our articles!). But just not with this proposal MADe (talk) 08:19, 22 December 2014 (UTC)

• MADe, since we now have similarly licensed wiki content we would be glad to have free flow of EM content. Just an FYI, $60/hr is low for a full stack developer these days who also is comfortable with mobile development and familiar with mediawiki code base. http://stackoverflow.com/questions/209170/how-much-does-it-cost-to-develop-an-iphone-application

Community comments

What is the content license?

As far as I can see, there is no license applying to the content indicated on WikEM.org. This is a blocker for any further discussion. May you please clarify the situation? Regards. Kelson (talk) 16:21, 21 December 2014 (UTC)

Kelson, this project does not involve the content on WikEM. However wikem is licensed under. Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License http://www.wikem.org/wiki/WikEM:Copyrights. This application is for developing the offline software the wiki uses and port that to NASA and release as an extension for all mediawiki installations. I hope that answers your question.

First sentence of the proposal is "This grant will expand the current offline mobile sync provided by WikEM.org", if this is not for WikEM.org, for which content do we develop this piece of software? If there is any other stakeholder, he should be clearly identified IMO. The fact that WikEM is not free content is a real problem, an acceptable license would be "CC-BY" or "CC-BY-SA". Wikimedia is a movement which encourages free knowledge and fight about the "non-commercial" clauses (which obviously brake knowledge spreading). Kelson (talk) 20:06, 21 December 2014 (UTC)

Yes would agree that NC and ND are not open licenses. And agree an open license would be a minimum requirement for consideration. Doc James (talk · contribs · email) 21:30, 21 December 2014 (UTC)

Thanks for the feedback. The license is updated to CC-Atribution-Share alike international 4.0 I completely understand. Sorry about the confusion.170.57.140.21 21:59, 21 December 2014 (UTC)Dan Ostermayer (talk) 22:04, 21 December 2014 (UTC)

• In response to the statement This grant will expand the current offline mobile sync provided by WikEM.org". We are taking the functionality we developed privately and expanding it to work on all mediawiki installations. This software will be developed for all mediawiki installations. WikEM is a stakeholder but the larger wiki installation community will benefit from this software which we will modify to work on all installations.Dan Ostermayer (talk) 22:04, 21 December 2014 (UTC)

WikEM Android and iOS apps source code

Where is the source code of the WikEM Android and iOS apps? Where is the code of the "current offline mobile sync" of WikEM? Do we deal here with free software? Kelson (talk) 20:06, 21 December 2014 (UTC)

Right now the mobile app source code is not public and the offline php code is on the server in a non extension format. I believe all the other grants were on community activities an chapters and not funding software. So this grant is a little different than all other proposals. We felt like the mediawiki community could benefit from software development just as much as they do from community activities. Dan Ostermayer (talk) 01:22, 22 December 2014 (UTC)

I don't think, for the same reason like for the content, this is an option for Wikimedia to sponsor software which is not 100% free software. One of the consequences of this code retention is that I still have difficulties to understand how works the current solution and how should work what you want to implement. Kelson (talk) 10:11, 22 December 2014 (UTC)

Technical details about the offline sync

May we have please more details about the offline sync? For example how work the current toolchain? What is the data flow from Mediawiki to the reader app? Which XML format to do you use? What is the storage format? Do you talk about incremental update? How do you deal with videos, pictures and sound? What technical tools exists now and what do you want to develop exactly? What would be the added value to the final user in comparison with the current solution? Kelson (talk) 10:23, 22 December 2014 (UTC)

Kelson, I believe we could offer incremental syncing. As of now it is non incremental. If you would like to look at the code you are welcome to here https://github.com/ostermayer/offlinexml 50.175.186.113 15:56, 23 December 2014 (UTC)

Thank you, this helps. So, you store the HTML generated by the Mediawiki for all articles in a flat XML file, and the project is mainly about migrating this code within a MW extension. Do I'm right? Kelson (talk) 22:47, 23 December 2014 (UTC)

yes. Although we will also make sure it works with complex extension such as semantic media wiki and allow for namespaces and incremental sync.Dan Ostermayer (talk) 21:33, 27 December 2014 (UTC)

Brief explanation for non-experts?

Concepts/terms such as sync are very hard to fathom. Can we please come down to Earth and provide a few sentences of background for people who are unfamiliar with the tech? Exactly what does it all mean?

I see the word "Mediawiki", which has indeed been developed by the WMF, but it's used widely on the internet. How will our readers benefit? Just providing "significant publicity to the WikiMedia Foundation" down the line is not sufficient justification for funding. Benefits for open knowledge generally are insufficient: the justification needs to be in terms of WMF sites and their users.

Why is there discussion on the application page? Tony (talk) 11:36, 27 December 2014 (UTC)

Tony, thanks for your comment. This application is for software that will work with wikipedia or any other wiki and create a easy method for developers to create offline reading on a mobile application. It does not involve content creation but does allow for future advances and ease of taking a wiki offline wherever you travel.

WMF comments

Hi Dan. Thank you for this thoughtful proposal and your timely engagement in the discussion. The project is an important one, but unfortunately not aligned with our grantmaking strategy. As raised in the above discussion, the project does not have a clear impact on either content or participation in the Wikimedia projects. Semantic Mediawiki is not used on the Wikimedia projects themselves, and the additional work needed to support it is not aligned with our focus. Additionally, the proposed software does significantly improve upon Kiwix, which the Wikimedia movement more broadly has been funding for several years. In the interest of optimzing investments already made in funding and time, any content gap (such as an iPhone app) in Kiwix should be addressed by funding that development, not a completely different parallel project.

Again, thank you for your time and effort to provide emergency care knowledge. Please let us know if you have any questions about our feedback. Best, Alex Wang (WMF) (talk) 20:44, 30 December 2014 (UTC)

Kiwix comment

I wanted to give a complete feedback, but I just have remarked that the answer was already "no". Anyway, I still want to invest a little bit time to write it, because this project is still interesting and I think it's important to produce something which might be helpful for this project and other similar ones.

The problems :

• This project (not the grant request) is new (only a few years old) and was started without (at least it looks like) having an analysis of the state-of-the-art software offer. Although Kiwix/openZIM is a lot older and recognized to be the most advanced solution in that field, we never were in touch AFAIK.
• The content of the project is not free (as free-licensed content) and although a quick "overnight move", I'm still uncertain about the real legal status of both text&pictures.
• The software is not open-source and of course not transparently developed.
• The offline format is a lot worth than the ZIM one (rendering limitation, bigger file sized, ...)
• The way this grant request was done, does not allow to see exactly what would be the technical outcomes.

That said, I think we should do our best to help this project having a working solution and I think they might win to switch to Kiwix and our toolchain and I'll do my best to help them if they are interested going that way. Here is how this might work:

• Content has to be free-licensed, so this needs to be clearly checked (I can help here if legal questions) against all contributors.
• ZIM file of the wiki (Parsoid/VE needs to be installed) should be created on time a month (I can help to do that)
• About the iOS support, we have a prototype of Kiwix for iOS, and we plan to release in the next months (we need testers and developers) Done 46.253.186.2 18:48, 22 September 2015 (UTC)
• About the fulltext search engine, this might be done in a few days of work (we just need to integrate (Kiwix related) Xapian C++ code in the app. and write the glue code - c++ developers needed)

I think with a lot less resources (than 30.000 USD) we might be able to make all this working before summer.

Kind regards Kelson (talk) 15:00, 21 January 2015 (UTC)