Wikibase Community User Group/Reports/2020

2020 was the third year of existence of the Wikibase Community User Group. 🤝 Meetings and notable 💬 Telegram and 📧 mailing list discussions are noted below. You can also see the 2018, 2019 and 2021 reports.

While there were good discussions and help was provided in a majority of cases, a few queries fell through the cracks, both on Telegram and the mailing list. Some cases may have been because the right people did not see them, or have time to respond; others may reveal gaps in feature or documentation. Regular meetings were not maintained during the first wave of the COVID-19 pandemic, although some resumed afterwards.

Activities

Below is summarised the relevant activities and discussions of members within the user group's official channels, and at organized meetings and events. Telegram discussions may be read via date search after joining the channel.

January

(Jan 3) 💬 Identifying the WDQS SPARQL endpoint (for use in Python's SPARQLWrapper)
(Jan 9) 💬 Use of WikidataIntegrator fast-run with Wikibase
(Jan 15-16) 💬 The need to increase $wgAccountRequestThrottle in the ConfirmAccount extension if requesting multiple accounts for workshops.
(Jan 18) 💬 The Wikibase quality constraints gadget's interaction with eval-unsafe Content Security Policy, and how to import/configure constraints
(Jan 20-21) 💬 Questions about a) why http: is used as the Concept URI for WDQS, and how to redirect to the appropriate HTTPS page, b) why wd/wdt are used as prefixes, and whether this will cause problems for federated queries (answer: probably), and c) differing use of prefixes on PersonalData.IO and Wikilex35, the former allowing non-prefixed queries by specifying a default in query service config
(Jan 22) 💬 Visit of Swiss French newspaper Le Temps to PersonalData.IO Wikibase instance for collaborative investigations
(Jan 25) 💬 How to do RDF dumps and imports
(Jan 26) 💬 Use of $wgWBRepoSettings['propertyOrderUrl'] to order properties on the page (Wikidata), and the difference with StatementGrouper (e.g. zones for statements vs. identifiers vs. constraints)
(Jan 26) 💬 Using the Wikibase Registry for documentation, and recording of extension/gadget usage
(Jan 27) 💬 Letting users save URL values when blocked by the ConfirmEdit extension, by setting $wgGroupPermissions['autoconfirmed']['skipcatcha'] = true (works by itself as autoconfirm defaults are 0)

February

(Feb 1-2) 💬 Creating on-wiki item lists by property using Lua ({{For}}/Module:For) vs. Module:Wikidata/WikidataIB vs. Lingua Libre's QueryViz, used for lists of speakers and records - see also generating html lists)
(Feb 2) 💬 Talk of creating a Dutch version of DerDieDas (from the second of two Wikibase/Wikidata presentations by Lydia Pintscher at FosDem 2020); the original French version UnOuUne was by Auregann.
(Feb 5-6) 💬 Issue running the WDQS updater from wikibase-docker (ended up as task T244367)
(Feb 10) 💬 Performance issues with scaling Wikibase API inserts; use of multithreaded bots
(Feb 12) 💬 Loading RDF dump into a new Wikibase instance
(Feb 18) 💬 Solving an issue with Wikibase\DataModle\Entity\ItemId not being found in WikibaseLib.entitypes.php - the solution was to run composer install twice (task T197868 and task T228593)
(Feb 18) 💬 Configuring default WDQS prefixes different to those of Wikidata ({{phab|T196042)
(Feb 18-22) 💬 Installation and use of quality constraints; question about items suggested use of Entity Schemas, like Cradle (WBStack version)
(Feb 20) 🤝 Online meeting of the user group, discussing status and organization (to be steered by Andra Waagmeester and briefly Envel Le Hir - WMDE representatives expressed bystander position); Telegram as backbone supplemented by mailing list and recurring meetings with presentation; use of the Wikibase Registry for documentation; and OpenRefine's lack of Wikibase support and alternatives.
(Feb 21) 💬 Resolving "newbie" confusion over a property that only allows other properties to be selected (the property must be of datatype Item, not Property)
(Feb 21) 💬 QuickStatements batch mode not working in wikibase-docker; suggestion was to use WikibaseImport
(Feb 23) 💬 Scrubbing information from Wikibase item history; suggested RevisionDelete (enabling) for "oversight" equivalent
(Feb 24) 💬 Further debate about performance from Feb 10 (VM with 128Gb RAM, 20 cores, 2TB NVMe SSD -> 30 item edits/sec; Wikidata peaks ~1000/sec) - MediaWiki suggestions
(Feb 24) 💬 Explanation of how to enter "no value" as a value (usability issue: user expected to be able to just save with an empty field)
(Feb 25) 💬 Questions about editing the sidebar, using Special:ListProperties and Special:AllPages (with appropriate namespace) for listings, and moving to a different domain
(Feb 25) 💬 Deleting an item (need to use Action API and set a title like "Item:Q1" if not in the default namespace)
(Feb 26) 💬 Duplicating the stream.wikimedia.org architecture to provide a server-sent event stream of recent changes (requires EventBus extension, Kafka and EventStreams)
(Feb 28) 💬 Managing the sites table to create sitelinks, and having the search box autosuggest foreign results in the user's language (unanswered, possibly requires the CirrusSearch extension)

March

(Mar 4-5) 💬 Best practices for mapping local properties to Wikidata using SKOS mapping properties and WikidataIntegrator
(Mar 9-10) 💬 Getting Quickstatements authentication to work with wikibase-docker - issues with entrypoint.sh
(Mar 10-15) 💬 Enriching a local wikibase with Wikidata content: federated queries via mapping identifier (avoids duplicating effort and going out of data, requires nuance over mapping) vs. importing data and using OpenRefine (simpler to query, better for full-text search)
(Mar 12) 💬 Using the API Sandbox to list properties using action=query&list=allpages&gapnamespace=XXX (and optionally labels with prop=pageterms&generator=allpages&wbpatterns=label; namespaces can be found with action=query&meta=siteinfo&siprop=namespaces)
(Mar 13) 💬 Changing the datatype of a property after creation can be done iff they have the same value type; use maintenance script extensions/Wikibase/repo/maintenance/changePropertyDataType.php (this topic repeated May 5)
(Mar 17-18) 💬 Discussion of mapping open-source COVID-19 projects and capturing of the Special Announcement schema
(Mar 21) SPARQL query optimization
(Mar 22-23) 📧 Accessing Wikidata over an API - suggested use of SPARQL endpoint via Semantic MediaWiki LinkedWiki extension rather than Wikibase Client
(Mar 26-28) 💬 Identifying resynchronization issue after failed upgrade to wikibase-docker bundle 1.34, and using WikibaseQualityConstraints with 1.33

April

(Apr 5) 💬 Discussion of COVID-19 Biohackathon 2020 (noted on Apr 16 wikibases were started to capture taxonomy descriptions)
(Apr 6) 💬 Unanswered OAuth-related problems logging into Quickstatements distributed with wikibase-docker
(Apr 7-11) 💬 Maintenance status of the WikibaseImport extension
(Apr 13) 💬 Making the formatter URL property work in your own Wikibase (datatype must be "external identifier")
(Apr 14) 📧 Discussion of import performance - suggestion to try wikibase-cli; future boost to query service noted (related blog post)
(Apr 15) 💬 Memory requirements for wikibase-docker; 4GB resulted in OutOfMemory exceptions, 8GB recommended
(Apr 15-16) 💬 Examples of Wikibase used for scientific datasets: https;//blog.factgrid.de/archives/1591 FactGrid], Gene Wiki, Biodiversity Observations
(Apr 16-17) 💬 Creating EntitySchema via the API (can re-enable in extensions/EntitySchema/src/MediaWiki/Content/EntitySchemaContentHandler.php; must use valid JSON)
(Apr 17) 💬 Shape expression/entity schema editor YASHE supports autocompletion on all Wikibases
(Apr 20) 💬 Documenting Wikibase backup and restore
(Apr 21) 💬 Possibility of generating radial barpltos using d3js in the Query Service
(Apr 24) 💬 Unanswered question about examples of quality control workflow for a multi-user wikibase
(Apr 28) 📧 Explaining that the wbeditentity API call requires the entire statement to replace (as compared to wbsetqualifier/wbremovequalifiers), and the same statement ID
(Apr 29) 📧 Question about duplicate references (also unanswered)

May

(May 7, May 22) 💬 Sorting properties, and raising the size limit of monolingual text to store abstracts
(May 8) 💬 LocalSettings.php required to be created/mounted in wikibase-docker container to prevent wikibase-docker from performing startup tasks, especially CirrusSearch|Elastic/CirrusSearch indexing
(May 11) 💬 Duplicating items and using qualifiers to annotate statements
(May 15) 💬 Suggestion to hardcode conceptBaseUri if changing the wikibase-docker language with MW_SITE_LANG (but it only changed @prefix wd:, not @prefix data:)
(May 16-17) 💬 Changing the title of random pages, and making them Item; as it defaults to only content namespaces, add it to them (suggested as a default) or make MediaWiki:Randompage-url link to Special:Random/Item
(May 18) 💬 Dealing with WDQS updater reporting that the last update time is before the minimum safe poll time (by deleting the data.jnl file and restarting WDQS and the updater)
(May 18-19) 💬 Adding the Wikidata merge gadget (not entirely successfully)
(May 19) 💬 Adapting Cradle to WBStack
(May 19) 💬 Unanswered question about how to use UPDATER_OPTS with wikibase-docker
(May 20) 💬 Quickstatements consumerKey and consumerSecret empty in oauth.ini; resolved by creating a new consumer with extensions/OAuth/maintenance/OAuthConsumer.php
(May 22) 💬 Getting the revision history of a wikibase item through the API
(May 23) 💬 Updating qualifiers of a statement using the API; tools may require the full value of the snak, not just the string to change
(May 30-31) 💬 De-duplicating references via script

June

(Jun 1) 💬 API query for Wikidata entity search like the search box (use &origin=* to resolve CORS issues); WBStack and its infrastructure, comparison of SPARQL and Cypher, ontologies, and combating vandalism
(Jun 2) 💬 Use cases where Wikidata federation may not be suitable; retrieving a wikibase item with properties and qualifiers using WikidataIntegrator; getting entities via the API in one language; upgrade documentation
(Jun 3) 📧 Issue regarding maintenance of the list (owned by absent founders)
(Jun 3) 💬 Unsupported content-encoding when WDQS updater fetches RDF; identified Jun 8 as needing php-zip installed prior to MediaWiki to support HTTP compression
(Jun 4) 💬 (Not being able to) link to anything other than Wikimedia Commons URLs for the "commons media" type - later supported by the Wikibase Local Media extension
(Jun 5-6) 💬 Desire to programmatically delete a Wikibase instance
(Jun 10) 💬 Identifying an ideal number of threads for the WDQS updater (Wikidata settings: --import-async -b 500 -T 1200)
(Jun 7-12) 📧 Follow up from April 14 import performance discussion reveals new wikibase-insert tool (doesn't update CirrusSearch DB) and highlights use of MySQL transactions (and newer MySQL JDBC driver) as key to import speed
(Jun 12) 💬 Query Service URIs and prefixes on WBStack (e.g. <http://X.wiki.opencura.com/entity/Q1> <http://X.wiki.opencura.com/prop/direct/P1> ?item)
(Jun 14) 💬 'Q' as a synonym for 'thing'; other prefixes such as (P)roperty, (L)exeme, (F)orm, (S)ense, (E)ntity Schema and (M)ediaInfo; (non-)configurability of the prefix (task T202676, previously 'entityPrefixes')
(Jun 15) 💬 Portable install of Wikibase by generating tar files of the images; (not being able to) generate properties or lexemes using Quickstatements
(Jun 16) 💬 Installing Quickstatements using Docker
(Jun 18) 💬 Low-level migration of SQLite Wikibase install to MySQL didn't work; reported that exporting via XML and importing having set allowEntityImport works
(Jun 20-22) 💬 Desire to hook item creation and updates in Wikibase Repository using PHP
(Jun 24-Jul 7) 📧 Discussion of new Wikibase Live Sessions leads into organization of data modelling in August
(Jun 25-26) 💬 Institutional requirement to restrict write access to specific groups of items
(Jun 26-27) 💬 Explanation of Wikibase and considering its value vs. Semantic MediaWiki or DynamicPageList3/Engine for a set of voting databases currently storing data in MediaWiki pages; Cradle suggested for form entry

July

(Jul 2) 🤝 Wikibase Live Session: Data Modeling in the context of Libraries presented by Jens Ohlig. (notes)
(Jul 3-29) 📧💬 Follow-up discussion on proposed meetings and channels of communication; Libraries and Wikibase Models Telegram group and Cultural Wikiprojects mentioned
(Jul 4-5) 💬 Jupyter notebooks from Andra Waagmeester and Magnus Sälgö to configure and load data into Wikibase; discussion of visulizations of Illuminati exchanges using Wikibase
(Jul 6) 💬 Getting sorted (statements via) properties working; unanswered questions on additional {{#statements}} parser function documentation beyond that of Wikidata, and accessing item labels/descriptions without Lua
(Jul 8-9) 💬 Problem with WDQS returning nothing despite population of entities (probably task T182394)
(Jul 8-9) 💬 Inability to import RDF graph using the Wikibase model via e.g. an "Merge"/"Upsert"-style SPARQL query on the WDQS endpoint (vs. using the Wikibase API)
(Jul 13) 💬 Loading from a dump to initialize WDQS with data beyond the 90-day window
(Jul 14) 💬 Avoiding a redirect loop for HTTPS-only services by setting $wgServer appropriately
(Jul 14, Aug 12-13) 💬 Adjusting the X-BIGDATA-MAX-QUERY-MILLIS header (and potentially proxy_read_timeout) in the wdqs-proxy nginx template to increase the WDQS timeout
(Jul 16) 💬 Assisting with setting an appropriate public scheme, host and port
(Jul 17) 💬 Need to use extra-install.sh to prepare the OAuth Consumer configuration for QuickStatements
(Jul 18-20, Oct 7) 💬 Creating an equivalent of mw:Professional development and consulting for Wikibase; later implemented as Wikibase/Consultants and Support Providers
(Jul 20) 💬 Syntax for adding qualifiers to existing items using WikidataIntegrator
(Jul 22) 💬 Ontological interoperability between Wikibase and Protégé (related: task T226150); a query to export a Wikibase vocabulary annotated with owl:equivalentProperty/Class and owl:sameAs properties was proposed by Peb Aryan
(Jul 24) 💬 Setting php.ini's sendmail_path or $wgSMTP to fix errors when resetting passwords in the Docker distribution
(Jul 29-30) 💬 Generating visualizations of Wikibase graphs via e.g. wikidata-graph-builder or Extension:Graph
(Jul 31-Aug 2) 💬 Renewal via API required for expiry of editing CSRF token

August

(Aug 5-8) 💬 Discussion of means to get Wikidata-enabled infoboxes on non-WMF wikis; possibility of using LinkedWiki
(Aug 6) 🤝 Wikibase Live Session: Focus on data modeling presented by Lozana Rossenova. (notes)
(Aug 7) 💬 WikidataIntegrator, sparql-client and internal Requests module as alternatives to qwikidata library for non-Wikidata wikibases
(Aug 9) 💬 Potential for for-profit hybrid scraper/miner + user contribution service to restrict access to data download and limiting querying
(Aug 10-11) 💬 Jobs queueing up due to failure of WDQS updater, over minimum safe poll time (stop WDQS service and updater, delete WDQS data.jnl (or volume), turn back on if no data loaded; else reload from dump)
(Aug 15) 💬 Need for WikibaseCirrusSearch extension in addition to CirrusSearch for haswbstatement: P1 searches to work
(Aug 16, 23, 29-30) 💬 Using the OpenRefine reconciliation service, initially to convert a Google Sheet to QuickStatements
(Aug 17-18) 💬 Inserting RDF triples from data modelled in Protégé and Karma to Wikibase via the wikibase-api and custom Python scripts
(Aug 24-25) 💬 Migration from Windows to Ubuntu Linux triggered error referencing Class 'Wikibase\\DataModel\\Entity\\ItemId' not found (task T197868#4324109) - Composer update required
(Aug 26) 💬 Fix for QuickStatements authentication failure due to MagnusTools update

September

(Sep 3) 🤝 Wikibase Live Session: Focus on data modeling – continued. An open discussion led by presented by Lozana Rossenova. (notes)
(Sep 29) 📧 Notice regarding the IFLA Wikidata Working group, supporting Wikibase in libraries

October

(Oct 1) 🤝 Wikibase Live Session: Use of batch data editing tools in Wikibase. Short presentation by a few people about their workflow followed by a group discussion to compare experiences working with importing data into Wikibase. (notes)
(Oct 6) 📧 Using wiki files inside Wikibase with the Wikibase Local Media extension
(Oct 6-8) 📧 Discussion of the Wikibase Consultants and Support Providers list

November

(Nov 12) 🤝 Wikibase Live Session: Participants share-out anything new or exciting that’s been happening with their Wikibase instance; what they've been doing recently, what is working well, what has been challenging for them, etc. (notes)
(Nov 20) 📧 Looking for tutorials on Entity Schema creation (unanswered on list)

December

(Dec 17) 🤝 Wikibase Live Session: Federation discussion, SPARQL, Wikibase vs. Semantic MediaWiki and tabular data, poll on Wikibase discussion platforms (notes)