Talk:Lingua Libre/Archive 2020

Media

• https://www.bbc.com/future/article/20200414-the-many-lanuages-still-missing-from-the-internet

Blue Rasberry (talk) 15:55, 16 April 2020 (UTC)

Question on Lingua libre

Does lingualibre share its recordings with Wikidata? If so, is there any way to connect lexeme on Wikidata to lingualibre’s recordings? I feel this could be considerably useful. Thank you! Weird green frog (talk) 07:34, 20 July 2020 (UTC)

User:Weird green frogL: The recordings are safe in Mediawiki Commons. AFAIK Lingua Libre Bot was planned to add entries to Wikidata, but I'm not sure how it ended. Olaf (talk) 07:59, 12 March 2021 (UTC)

Temporary forum

Hi everyone. This section is opened as a contengency plan. You can continue or open conversations here, store informations and anything We will also periodically gather relevant status information here and ping everyone when we start to be back online. Please add your username below if you wish to be notified of our progress. cc @Pamputt, Olaf, Poslovitch, WikiLucas00, Poemat, Eihel, Titodutta, सुबोध कुलकर्णी, Subodh (CIS-A2K), Lyokoï, LoquaxFR, DSwissK, and Vami: Yug (talk) 13:07, 11 March 2021 (UTC)

• Prayers. Yes, I'll join. -- টিটো দত্ত (কথা) 13:09, 11 March 2021 (UTC)
• Thanks for update. Marathi LL community is hoping for the best! Everything will be alright soon, our prayers to all at LL management. - --सुबोध कुलकर्णी (talk) 04:56, 12 March 2021 (UTC)

Server fire and Backup?

Millions of websites offline after fire at French cloud services firm --Reuters.com

See official Wikimedia France message on #Lingua_Libre 2.1 - fire edition ǃ.

TL;DR: All recorded audios are already and safely on Wikimedia Commons. 3 weeks required to restore website, database into proper shape. Some documentations, lists, translations may be lost.'

Is any backup in another place?. BoldLuis (talk) 18:12, 11 March 2021 (UTC)

Hello BoldLuis. Not "full backups" that I know of but:

All recorded audio data are already on Wikimedia Commons.

User accounts there too.

The software codes are on GitHub.

While.... the documentations, discussions, files' and speakers' metadata together with some gadgets, js and css are on Lingualibre.org mediawiki database.

I assume(d) OVH to have multi-physical-sites backups, but I got no confirmation of this so far. We know the site is down, but I don't know if our server was in the SGB2 (physically burnt) or in associated centers (SGB1, SGB3, SGB4). There are "hints" that we may be ok but nothing solid yet. Nothing much to do this week, more visibility early next week.

We have enough to rebuild anyway but the cost-time depends on the situation. Yug (talk) 18:45, 11 March 2021 (UTC)

I wish the best 🙏 BoldLuis (talk) 18:58, 11 March 2021 (UTC)

According to Michael on Discord, the wikibase server was in SGB1, and a third of this datacenter is destroyed (I don’t have more details).

Also, the up-to-date Blazegraph database is on another server, so if the wikibase itself cannot be recovered it’s probably possible to reimport from the Blazegraph to the Wikibase, although with a loss of the history in this case. ~ Seb35 [^_^] 19:01, 11 March 2021 (UTC)

Seb35: Only for Qids or for wikipage as well ? Yug (talk) 20:25, 11 March 2021 (UTC)

@Yug: Blazegraph only contains wikibase pages (Qid and Pid) and their content. Cheers, VIGNERON * ^discut. 07:46, 12 March 2021 (UTC)

As a Plan C I saved Google, Yahoo-Bing, Yandex cached pages for our English help pages. All on a github repository. Most pages are from late february, html and without wikicode. But if required, we have the raw texts. Yug (talk) 12:23, 13 March 2021 (UTC)

@Yug: Good to think of a plan C. Could you give me the link to this repository please? I downloaded the Technical Board page, probably one of the last versions (March 9th at 10:25pm, but only the html and images). Best — WikiLucas (🖋️) 14:42, 13 March 2021 (UTC)

@WikiLucas00 and Olaf: there Github Plan C. If you have text, create a folder at your username and upload. Or add file to the root it's ok too ;) Olaf, I will need your github. Yug (talk) 16:34, 13 March 2021 (UTC)

@Yug: To grant access rights? https://github.com/olafmat Olaf (talk) 11:08, 14 March 2021 (UTC)

@Olaf: Access granted. If you saved some content, create a folder at your name and paste there. Yug (talk) 19:56, 14 March 2021 (UTC)

@Yug: Done. I think the Special:Version may be useful, because it contains version of every MediaWiki extension installed. And a snapshot of Special:ActiveUser may be useful when inviting people back. Olaf (talk) 21:15, 14 March 2021 (UTC)

@Olaf: thank you ! :) I think WikiValley has a MediaWiki with all our extensions updated on his/their computer(s). Nice to see we have Special:ActiveUser !

Plan A is getting all back when the servers are turned on (26 March?): within hours or a day we could be back on track.

Plan B/C is rebuilding Wikibase from Blazegraph, documentation from /lilidown repository, and few js script from scratch.

• By now we hopefully saved +90% of our documentations. Only one week of recent posts and sections would be missing and to rewrite. Mos recent forums issues could be dropt, since most are exchange of info for team training. Ex: NLPTK discussion, Ratelimit feedbacks are non-critical. What was written the week prior to the crash ? Kurdish and Catalan users defined their wikt structures and I wrote a "Workshops" documentation which could be to rewrite, but it's minimal.
• The gadgets and css would be missing and are more critical. We must consider the possibility to have to recode those.
• the « core Lili contributors » would have to lead and organize works, wikification by experienced wikimedians but non-Lili folks. Lili devs will have give some love to JS & gadgets (language importer, list bots). Something like that.

This Plan B-C worse case senario could take us one Month + of code and Wikiedits and will likely require WikiValley's formal help to reverse the data from Blazegraph to Wikibase. But doable, and we may likely ask for help on various bistro if necessary. We shouldn't plan this too much as of now, but starting to identify potential helps to call over is wise. Yug (talk) 22:07, 14 March 2021 (UTC)

It's a pity, there was no backup of speakers' profiles. Probably the only data left about the origin of each speaker is the information put by the bot in audio templates on fr-wiki, isn't it? And the data about the language level are not preserved anywhere outside of the damaged datacenter. So we don't know which recordings were done by natives. Olaf (talk) 22:16, 14 March 2021 (UTC)

We had talks about better bots and to add metadata into audio files and Commons with P. But not before Mai~August. Yug (talk) 22:30, 14 March 2021 (UTC)

Ok, but do we still have the data about speakers, or the only copy was in lingualibre.org? Olaf (talk) 22:38, 14 March 2021 (UTC)

I asked the question. No answer at the moment. Yug (talk) 17:26, 15 March 2021 (UTC)

@Yug and Olaf: According to Vigneron, the BlazeGraph contains all the elements from the Wikibase (Q-ids and P-ids). Speakers' profiles were elements in the Wikibase, so they should be in the BlazeGraph. No worries there --Poslovitch (talk) 10:15, 16 March 2021 (UTC)

Yes, it should (sadly, I need to emphasis on should, we will know more soon). Cheers, VIGNERON * ^discut. 10:29, 16 March 2021 (UTC)

Wikimedia France, VIGNERON, Poslovitch and myself will have an online meeting Friday to share last understanding of the OVH servers' status and discuss the coming actions to takes. Yug (talk) 20:23, 16 March 2021 (UTC)

English Wikipedia

A (small) good news among the recent bad news: Lingua Libre now has an article on the English-speaking Wikipedia (help us improving it!). For the moment, LiLi has an article on 4 differents Wikipedias (es, eu, fr, en), which means we still need to write some in new languages, and improve existing ones (it can be a good way to wait while LiLi is down)! All the best — WikiLucas (🖋️) 15:06, 12 March 2021 (UTC)