Wikibase/Wikibase.cloud
Project edit
Wikibase.cloud is a cloud-based platform developed by Wikimedia Deutschland (Wikimedia.de) to host instances of Wikibase. Wikibase is an open-source software suite for creating collaborative knowledge bases. Originally developed for Wikidata, the Wikibase project is being actively developed and maintained by the team at Wikimedia Deutschland. With Wikibase.cloud, we want to make it easier than ever to create, connect and grow a linked open database. In so doing, we aim to liberate data from silos and increase access to knowledge around the world.
Wikibase.cloud is now in Open Beta -- feel free to sign up now! Consider adding your instance to wikibase.world, an invaluable community-run site for Wikibase and the various instances you can find across the net.
We'd like to say a special thanks to Adam Shorland for building WBStack on which Wikibase Cloud is based, and Rhizome, for their support, innovation and leadership.
Timeline edit
2019: WBStack.com was launched by Adam Shorland (Addshore)
2020-2021: Wikimedia Deutschland investigates creating a hosted Wikibase service to replace WBStack.com
2021: Wikimedia Deutschland announces decision to start Wikibase.cloud at WikidataCon 2021 (video)
2022: Wikibase.cloud launched, with primary focus on migrating users from WBStack.com to Wikibase.cloud
June 2022: WBStack.com users should all be migrated, service will be sunsetted
Resources edit
Connect with the Wikibase.cloud community:
Documentation:
- User documentation for Wikibase.cloud can be found on mediawiki.org
Contact us edit
As with all Wikimedia projects, we use the public ticketing system Phabricator (phabricator.wikimedia.org). You can keep up with our progress via the Wikibase.cloud Phabricator board. Otherwise, reach out to us at swe_partnerships@wikimedia.de.
Updates edit
February 9th 2024
- Status of cloud access
- 879 Wikibases on cloud total
- News from engineering:
- Errors from MediaWiki are now ingested in our error reports
- More details:
- https://phabricator.wikimedia.org/T350394
- QuestyCaptcha is here! See full announcement.
- Anything else worth mentioning:
- New Wikibase Cloud PM, more info soon!
- Development Plan 2024 status update online here: https://www.wikidata.org/wiki/Wikidata:Development_plan
- Knowledge Equity Research, participants for interviews needed. See full announcement here: https://lists.wikimedia.org/hyperkitty/list/wikibaseug@lists.wikimedia.org/thread/DPHZNRODTNGS7EXHRKKLPHVVNGLNKJBC/
January 10th 2024
- Status of cloud access
- 824 Wikibases on cloud total
- News from engineering:
- Solving issues with ElasticSearch
- Find out about it here: https://phabricator.wikimedia.org/T354048
- Continue to work on Spam protection (using QuestyCaptcha)
- More details here: https://phabricator.wikimedia.org/T342143
- Solving issues with ElasticSearch
- Anything else worth mentioning:
- Next Wikibase Live Session on 25th of January
- End of the year survey share-out coming soon!
- Wikidata & Wikibase office hours on 17th of January
December 15th 2023
- Status of cloud access
- 779 Wikibases on cloud total
- News from engineering:
- The new homepage was released! https://www.wikibase.cloud/. We wanted to give it a more professional look, to show our intend of continuing to build out Wikibase Cloud for the existing community, give more spotlight to Wikibases on the platform and make expectations more clear, also with the goal of growing the user base.
- We continue to improve the QueryService performance in terms of speed as well as to avoid data being skipped. Relevant tickets to keep an eye on:
- With the growth of the platform, we also see an increase in spam accounts being created on Wikibases. We are introducing an additional Captcha, managed by Wikibase owners, to prevent these spam accounts from successfully being created or requested.
- This the main ticket: https://phabricator.wikimedia.org/T342143 the sub-tickets display the process
- We also noticed a bug in the existing Captcha which allowed spammers to circumvent it, and deployed a fix
- Finally, we continue to improve our deployment workflow for more efficiency: https://phabricator.wikimedia.org/T351412
- Anything else worth mentioning:
- Survey results coming out soon, stay tuned!
November 27th 2023
- Status of cloud access
- 750 Wikibases on cloud total
- News from engineering:
- Wrapped up the ElasticSearch version updates and maintenance, no more ElasticSearch on our schedule for a while
- Ongoing optimizations to the QueryService, latest fixes we shipped:
- And work in progress on the QueryService:
- https://phabricator.wikimedia.org/T352009
- https://phabricator.wikimedia.org/T352008 (investigation overarching ticket)
- We noticed a steady share of Wikibases remaining empty or not even being opened (roughly 20% of all Wikibases we host), and thus deployed an automatic notification to the owners to remind them about it / offer help: https://phabricator.wikimedia.org/T344689
- New homepage is a work in progress https://phabricator.wikimedia.org/T339332
- Working on introducing an additional, self-managed Captcha to avoid the amount of spam accounts some people are experiencing on their Wikibases: https://phabricator.wikimedia.org/T342143
- Started work on automating our deployment workflow to make us more efficient in the future: https://phabricator.wikimedia.org/T351412
- Anything else worth mentioning:
- Wikibase presentation at Kuwiki: tips & tools https://de.wikipedia.org/wiki/Wikipedia:Arbeitsgemeinschaft_Kunstwissenschaften_%2B_Wikipedia/tips_%26_tools
- Upcoming Data Modelling days https://www.wikidata.org/wiki/Wikidata:Events/Data_Modelling_Days_2023
- Upcoming Wikibase Live Session on the 30th of November at 17:00 UTC
November 3rd 2023
- Status of cloud access
- 697 Wikibases on cloud total
- News from engineering:
- All Wiki’s are now running on the new ElasticSearch version, we have yet to remove the old cluster to finalise the update process
- It’s now possible to create a Wikibase name and domain using special characters https://phabricator.wikimedia.org/T345139
- First improvements to the QueryService performance were rolled out https://phabricator.wikimedia.org/T333164, further ones are in progress
- Rolled out the newest MW 1.39 security release https://phabricator.wikimedia.org/T347553
- Anything else worth mentioning:
October 16th 2023
- Status of cloud access
- 636 Wikibases on cloud total
- Fully open access!
- News from engineering:
- Discovery tool got published, check this link
- New Homepage to be released in the coming weeks
- There is a working ElasticSearch 7 image we can use to finally update our cluster. We are still looking into some details of building it in CI T335891
- We improved our alerting setup for logical backup failures T348148
- Users that created empty wikibases shall soon be notified after 90 days of inactivity T344689
- Retention statistics will soon be availabe to the product team T345434
Wikibase.cloud now suppports special characters in custom domain names T345139
- Anything else worth mentioning:
September 7th 2023
- Status of cloud access
- 574 Wikibases on cloud total
- 12 new invite codes sent
- News from engineering:
- Discovery tool about to be published!
- Major ElasticSearch update being rolled out soon
- Completed multiple storage and database upgrades
- More fallbacks in place for ElasticSearch outages to reduce the impact on users, and increase the team’s ability to work anything that is not ElasticSearch and infrastructure
- New Homepage to be released in the coming weeks
- Anything else worth mentioning:
- Valerie WMDE gave a Wikibase presentation at Femnetz' monthly discussion group
- Wikibase presentation at ICOM conference in Valencia
August 7th, 2023
- Status of cloud access
- 556 Wikibases on cloud total
- 8 new invite codes sent
- News from engineering:
- ElasticSearch roadworks are ongoing. We’re very close to kicking off the large scale version upgrade for all Wiki’s, which might affect search experience in the form of downtime, re-ordering of results, indexing and such, but for the noble cause of more stability and reducing incidents caused by ElasticSearch going forward
- Guardrails are put in place to safeguard stability when we remove the invitation code. With this, we’re ready to move out of closed beta in the coming weeks
- Redesigned the homepage including a section to feature homepages with examples, which will be released together with the Discovery feature where people can browse through Wiki’s hosted on Wikibase Cloud
- Minor fixes such as the empty license statement and broken pop-ups
- Anything else worth mentioning:
- Wikibase default sidebar with helpful links, thread created for community’s input: https://phabricator.wikimedia.org/T343686
- Documentation needs, thread created for community: https://phabricator.wikimedia.org/T343710
- Wikimania is coming soon! Join us for our presentation about ‘the wondrous world about wikibase’ on Friday 18th!
July 12th, 2023
- Status of cloud access
- 523 Wikibases on cloud total
- 43 invite codes sent out since 16th of May
- News from engineering:
- Work to update all Wikibases to a newer version of ElasticSearch is progressing. This should hopefully bring more stability to the overall search experience
- Started with the introduction of sign up throttling to safeguard the platform stability when we remove the invite code requirement and sign up flow
- Prepared work on the discovery feature to make Wikibases hosted on Cloud more discoverable for new users. Tickets are on Phabricator. This was the main outcome of the Q1 survey on data modelling
- Anything else worth mentioning:
- Wikibase Live Session, docs created to summarize topics of interest
- New blog posts in the pipeline!
- Preparing additional refinements to the documentation for new joiners (please share your thoughts on what’s still missing!)
June 12th, 2023
- Status of cloud access
- 503 Wikibases on cloud total
- News from engineering:
- Encountered outages and ElasticSearch issues caused by infrastructure maintenance over the last 2 weeks, now recovered
- Finishing up automatically running through backlogged ElasticSearch jobs to quickly recover results during (maintenance) outages https://phabricator.wikimedia.org/T330389
- Moving forward with ElasticSearch updates https://phabricator.wikimedia.org/T330998
- We are working on a feature to discover other Wikibases, as uncovered in our Data modelling research
- Anything else worth mentioning:
- We attended the re:publica conference on Tuesday June 16th to talk about Wikibase
May 16th, 2023
- Status of cloud access
- 490 Wikibases on cloud total
- 8 new invite codes were sent out in the last weeks
- News from engineering:
- Further ElasticSearch scaling stabilisations in order to unblock ourselves from opening up the platform to a wider audience (version updates based on new MW versions, fallback solutions etc.)
- Fixed an issue where verified, non-admin users couldn’t add references with URI’s (https://phabricator.wikimedia.org/T327752)
- Anything else worth mentioning:
- Data Modelling report is on commons now!
- New blog post is out!
- LOD MeetUp at re:publica! Stay tuned!
May 3rd, 2023
- Status of cloud access
- 483 Wikibases on cloud total
- 6 new invite codes were sent out in the last weeks
- News from engineering:
- Successfully finished migrating all Wikibases on Wikibase Cloud to MediaWiki version 1.39
- In the process of upgrading ElasticSearch
- Reducing impact of ongoing ElasticSearch issues with re-introducing a fallback as well as reducing data recovery time in https://phabricator.wikimedia.org/T330389
- Anything else worth mentioning:
April 11th, 2023
- Status of cloud access
- 475 Wikibases on cloud total
- 11 new invite codes were sent out in the last weeks
- News from engineering:
- Upgraded production environment to MediaWiki 1.38
- ElasticSearch major incident, see mail. The fix takes up more time than expected, a more stable long-term solution is needed, this is currently a work in progress.
- Anything else worth mentioning:
- Data modelling user interviews are done, we’ve gathered great insights and want to thank all participants for their valuable time and input
- Preparing for Open-Beta Launch, as part of this ElasticSearch issues will be resolved and we will move to MediaWiki 1.39
- Suite Survey insights, soon to be shared via Commons and Live Session
- Last Live Session on Documentation, see notes
- Newbie Docs will be launched this week
March 6th, 2023
- Status of cloud access
- 436 Wikibases on cloud total
- 27 invite codes were sent out in the last weeks
- News from engineering:
- ElasticSearch has been rolled out to all Wikibases on Cloud and re-enabled for new Wikibases. There are ongoing challenges with our scaling efforts that result in Search sometimes not being available or not indexing all items. If you encounter any (similar) issues, please add them here: https://phabricator.wikimedia.org/T330069
- Added new Wikibase Cloud logo and favicon to main page
- Anything else worth mentioning:
- Uploaded Wikibase Cloud logo onto Commons: https://commons.wikimedia.org/wiki/File:Wikibase_Cloud_Logo.svg
- Research Report https://commons.wikimedia.org/wiki/File:Wikibase_Cloud_UX_Research.pdf
- Soon to come:blog post via tech-news blog, interviews with Wikibase community members and their projects. If you’d like to be part of this write to valerie.wollinger@wikimedia.de
- Suite Survey is out! Please fill it in via: https://wikimedia.sslsurvey.de/Wikibase_Suite-User-Research-02-23/
- Last Live Session about Data modelling using Wikibase was great! Check out notes via https://etherpad.wikimedia.org/p/WBUG_2023.02.23
January 17th, 2023
- Status of cloud access
- 407 Wikibases on cloud total
- 10 new invite codes were sent out this week
- News from engineering:
- https://phabricator.wikimedia.org/T316093
- https://phabricator.wikimedia.org/T322665 is resolved
- Contact page was released!
- Bumped up some resources to deal with more incoming activity on Cloud
- Onboarding new team members
- Full focus on ElasticSearch exclusively from now on
- Anything else worth mentioning:
- Soon community insights to be presented at user group meeting on 26th of January
- Presentation at University of Applied Sciences in Mainz!
- LD4 presentation about Wikibase Cloud on Tuesday 17th by Evelien
- Stay tuned for logo release on Commons (just ironing out the final details)
December 12th, 2022
- As of today:
- a total of 383 Wikibases on cloud.
- 8 new invite codes were sent out this week.
- News from engineering:
- Some frontend adjustments will be released this week and the next (minor adjustments to the sign-up flow and the release of the contact page).
- We’ll be starting with Search until the end of the year, and continue into next year to release a solution for everyone. This is not expected to finish this year.
- Anything else worth mentioning:
- Wikibase Cloud logo will soon be released.
- The survey closes tonight. We will need some time for processing and generating insights, so the results will be presented early next year.
November 30th, 2022
- As of today:
- there are 378 Wikibases on cloud total
- 23 new invite codes were sent out this week
- News from engineering:
- Captcha issue for Wikibases with a custom domain has been solved: https://phabricator.wikimedia.org/T323498.
- Kubernetes upgrade is completed, we’re now running on a stable, supported version: https://phabricator.wikimedia.org/T311205
- Working ofinishingup the new contact page and aiming to release in the coming weeks so that we can end the year with full focus on Search
- Anything else worth mentioning:
- check out last Wikibase Live Session (see notes)
November 15th, 2022
- As of today:
- there are 351 wikibases on cloud total
- 50 new invite codes will be sent out this week
- News from engineering:
- we are done with secret rotation.
- wbaas-deploy is officially open-source!
- Alerts were modified to make them less sensitive and spamming emails.
- We switched to ingress API version to GA v1 from v1beta1 in a few charts
- Kubenetes upgrade is still blocked by other tasks
- Anything else worth mentioning:
- Wikibase/Wikidata office hour took place last week on Wikidata telegram channel.
- Wikibase Survey is out! Check: https://wikimedia.sslsurvey.de/Wikibase.cloud-User-Research-11-22/
November 2nd, 2022
- As of today
- there are 344 wikibases on cloud total
- codes are sent out regularly to applicants for closed beta
- News from the engineering team
- tackling some user requests who want to change account details
- Not yet everything open-source, getting very close
- Kubernetes updatesare due
- Anything else worth mentioning:
- Wikibase live session recap and link to notes
- Qichwabase created an introduction video to explain more about their Wikibase.cloud instance
- Save the date: Office hour on Telegram next Wednesday, 9th of November. Event announcement will come soon!
- A survey for Wikibase.cloud is being drafted, scheduled to be sent out this month. Please participate, we’re excited to hear your feedback!
October 11th, 2022
- As of today
- 334 wikis created
- Since September we sent out 101 codes to people on the waiting list
- We sent reminders to 48 people who have not yet used their requested invite codes
- What has our engineering team worked on this week? What challenges did we face?
- Working on open-sourcing the deployment code of wikibase.cloud
- Running maintenances on Wikibases to correct a bug from migration from all Wikibases task T312256
- Some work on the front end for the contact page
- Anything else worth mentioning?
- Our new Wikibase ComCom Manager, Valerie, introduced herself to the community. In her own words she will support constructive information flows to enable co-creation among the Community and the Software Development team.
- This week 3 people in the Wikibase.Cloud team will present Wikibase at the Kubernetes community event
- Upcoming user group meeting will take place on October 27th
September 27th, 2022
- As of today
- We have generated 105 invite codes
- We have sent out 92 invite codes to new users
- Dashboard stats
- 304 wikis created
- 193 have been actively maintained
- 69 have not yet seen any data added
- 42 have been deleted
- What has our engineering team worked on this week? What challenges did we face?
- We made some UI improvements
- Worked on Bug report for unexpected values in labels for units
- Existing metrics dashboards were added to terraform and can be re-created
- Reworked and deployed Wikibase.cloud password reset flow
- Anything else worth mentioning?
- The team welcomed a new Community Communications manager for Wikibase. They are currently being onboarded and will introduce themself to the community at the start of October.
September 09th, 2022
- As of today there are 283 wikis created of which 180 have been actively maintained over the last 3 months. 62 have not yet seen any data added and 41 have been deleted.
- We sent out 75 invite codes to new (non-Wbstack) users from the early access waiting list. If you're interested in receiving an invite code, please can fill out this application form.
August 25th, 2022
- We are still preparing for onboarding. We now have metrics in place and are working on further extending Wikibase.Cloud documentation and FAQs.
- Dashboards with metrics are in place that will help us monitor and analyse before onboarding.
- The Wiki limit per user will be set to 6 task T311121. We encourage you to delete empty wiki's to free up space as to not hit the limit so fast. This would help to reduce the load on our resources.
- ElasticSearch will be disabled for new users task T314937
August 04th, 2022
- We are still in the process of setting up monitoring and metrics to analyse our readiness and ensure a smooth onboarding for future users.
- We added a waiting list link to the wikibase.cloud landing page.
July 21st, 2022
- Wrapping up the past sprint by completing bug-fixing and other migration-related topics to tie off loose ends.
July 14th, 2022
- We have started brainstorming setting up monitoring and metrics to analyze our readiness and ensure a smooth onboarding for future users.
- Fixing user-reported bugs such as task T312747 and task T311602
- To reduce migration complexity, in mid-June we temporarily put in place a Wikibase creation limit of 1 for Wikibase.cloud users. We’ve since analysed the amount of Wikibases our current user base have, and decided to increase the limit to 6, and will continuously analyse the optimal limit. This will be implemented in the near future
- task T312801: An issue related to search indexing newly created items was reported this week. We suspect this might be related to multiple other issues, as mentioned in the ticket, and need some time to find the root cause.
July 07th, 2022
- During migration issues and bugs arose, so the team is taking some time to get ready for access by users from the waiting list.
- task T311602: We discovered that log in to Wikibase.Cloud fails for accounts registered using task-specific emails.
- task T311495: Wikibase.cloud instance does not update triple store
June 29th, 2022
- task T311495: We had an outage of the queryservice-updater on June 23rd. This has since been resolved.
June 21th, 2022
- The 3rd and final migration batch was successfully completed on Friday 17.06.2022
- Our new Wikibase Cloud Product Manager, Evelien Zandbergen, introduced herself to the community. In her own words, she will be “...focusing on understanding what Wikibase users need, finding what adds real value for you, and turning this into a beautiful roadmap so that I, together with the team, can improve Wikibase.cloud for you and we all can fulfill our vision in the long run.”
June 14th, 2022
- Batch B is concluded and the final Batch (C) migration has started. We hope to finish the migration by June 17th.
- This week we limited the number of new wikis that can be created on Cloud. That means users with no wikis yet (only an account) can only create one wiki. All others cannot create more wikis than what they already have.
- Due to the overhead of Wikis on ElasticSearch we have disabled the feature. We want to prioritize migrating people in.
- The deadline to accept Wikibase.Cloud Privacy Policy and Terms of Use (ToS) for Wbstack users ended on June 14th. If you are a Wbstack user who did not agree to the ToS or a new user, you can apply for early access to Wikibase.Cloud using this form.
June 7th, 2022
- task T303852: We have paused the migration of batch B users due to issues with Elasticsearch. Batch B users have been informed, and we expect to have more clarity around when migration will resume this week.
- task T309019: The focus has been on increasing the stability and observability of Elasticsearch. For instance, in the future, we would like to gather more data on the resources required to run Wikibase.cloud.
- task T309692: We discovered an issue related to the query service. Unfortunately, this means that Wikibase instances that have already been migrated to Wikibase.cloud are not updating as they usually would.
- Once we have completed the migration of all WBstack.com users, we will begin to contact early access applicants. If you’ve not yet submitted an application for early access, you can do so here.
- We published a post on the Wikimedia Deutschland tech news blog.