Talk:Wikidata/Notes/Caching investigation

API and memcached

One solution I toyed around with for an extension that accessed a very slow external API, was to replace some part of the content on the rendered wikipage with stored content from memcached. If the content wasn't available it was replaced with a placeholder. This placeholder had a link for those users had not enabled Javascript. If Javascript was enabled it would fire a request repeatedly to query for the missing content until it was available, and then replace the part of the content that was missing. When the external data was available the extension would store it in memcached until next time it was requested or it expired. If a request was made after the memcached content was available the extension would not render the placeholder but replace it with the content from memcached immediately and before it was delivered to the browser.

This was made as a solution for an extension that accessed external sites with very long response times, and where efficiency wasn't really an issue. It was although necessary to lower the number of requests to the external site, ie. to be able to say to external siteowners that the extension provided sufficient caching. In practice caching will be sufficient to catch editing in the wiki, but it will not provide any safeguards against rendered pages with expired data from the external sites. That is it provides caching for repeated rendering but not validating rendered wikipages against inconsistency with the external site (data integrity with the external site, ie. that it is an accurate and complete reproduction of data available on the external site).

This is probably not a good solution for Wikipedia accessing Wikidata, but it could be a solution for external wikis accessing (and using) Wikidata.

The extension is not available for reuse, but the basic idea is simple enough to reimplement as an extension with open license. — John Erling Blad (WMDE) (talk) 22:58, 17 April 2012 (UTC)

Handle affected pages in client wiki similar to pages affected by template change

As to re-rendering of affected Wikipedia pages, a change in Wikidata is similar to a change of a Wikipedia template. I don't really know how MediaWiki handles template changes (job queue etc.), but maybe that process could be re-used for Wikidata changes. Chrisahn (talk) 21:21, 20 April 2012 (UTC)