Research:Language-Agnostic Topic Classification/Outlink model performance/All wikis

This is a data dump of how the model does when evaluated on every single Wikipedia (based on applying English labels to other languages per Wikidata). In particular, "Had Groundtruth" means that an article existed in that language for which there was an English equivalent with groundtruth topic labels (and therefore is included in the micro/macro precision/recall/f1 statistics). "No Groundtruth" generally means that that article had no English equivalent per Wikidata (though in a very small number of cases, it would have an English equivalent with no groundtruth topic labels). Generally I would include average precision statistics as well, but that is much more memory-intensive and so has not been done yet for all wikis. This is actually based on a model with the following characteristics:

• Code: https://github.com/geohci/wikipedia-language-agnostic-topic-classification
• Model architecture: multi-label fastText supervised model
• Number of articles by language in training data (% of training data). This was an attempt to balance convenience and coverage. The languages were included in rough proportion to their number of articles with labels (with some dampening of English Wikipedia):
  • Arabic (ar): 662,021 (10.5%)
  • Spanish (es): 854,992 (13.6%)
  • English (en): 2,363,824 (37.5%)
  • French (fr): 1,138,743 (18.1%)
  • Hindi (hi): 62,720 (0.1%)
  • Russian (ru): 707,924 (11.2%)
  • Chinese (zh): 513,292 (8.1%)
• Epochs: 5
• Learning rate: 0.1
• Window size: 20
• Min count (under which QID is not retained in vocab): 20
• No pre-trained embeddings used
• Embeddings dimension: 50
• Total number of model params: 3200 (50 x 64)
• Vocab size: 2,829,260
• Total number of embeddings params: 141,463,000 (vocab size * embeddings dimension)
• Model size on disk: 588.0 MB

Outliers and Anecdotal Observations

These observations are based on making topic predictions for every Wikipedia article and then looking at languages that are outliers in their distribution of topics -- e.g., no biographies or a lot of history articles. Some of these are real -- e.g., small wikis can have strong skews introduced by highly active editors; Lsjbot and other such bots can lead to a very strong skew in article topic towards science or geography or other topics for which it is easy to generate large numbers of articles based on standard templates. Some outliers though, are a combination of bot skew and data bias -- i.e. a wiki is heavily skewed towards a certain type of article such as articles about dates but each of these articles has a similar distribution of outlinks that are different than what the model was trained on and thus it consistently predicts the wrong topic for all of these articles. To verify trends, I checked specific predictions for a language as well as repeatedly used the simple process of visiting https://<lang-code>.wikipedia.org/wiki/Special:Random and then checking the Wikidata item to see what the article was about. Some examples:

• Skew towards articles about settlements and rivers that are predicted to be about History and Society.Society. Many of these articles seem to originate with EmausBot (see contributions by wiki here)
  • Chechen Wikipedia (ce) -- e.g., Q15363845
  • Tatar Wikipedia (tt) -- e.g., Q4517052
  • Bashir Wikipedia (ba) -- e.g., Q1065110
  • Chuvash Wikipedia (cv) -- e.g., Q6946367
• Skew towards biographies that is in part a large number of articles about dates (e.g., Q2065)
  • Western Armenian (hyw)
• Skew towards Culture.Linguistics and History_and_Society.Business_and_economics that is actually asteroid articles with very few outlinks (e.g., Q209053)
  • Yoruba (yo)
• All languages, regardless of content, have zero recall for Central Africa as a result of my not implementing the topic labeling changes described here: https://github.com/wikimedia/drafttopic/blob/master/drafttopic/utilities/add_central_africa.py
• Common outliers that are appropriate predictions:
  • Articles for dates that are predicted to be related to History: Turkmen (tk), Nahuatl (nah), Greenlandic (kl)

Performance by Wiki

wiki_db	Had Groundtruth	No Groundtruth	micro precision	macro precision	micro recall	macro recall	micro f1	macro f1
amwiki	8031	6572	0.652	0.615	0.348	0.287	0.434	0.367
sqwiki	52302	24798	0.778	0.656	0.608	0.379	0.666	0.456
mlwiki	52489	15146	0.807	0.717	0.600	0.423	0.675	0.513
emlwiki	8624	3814	0.623	0.405	0.427	0.232	0.473	0.267
kiwiki	1206	100	0.747	0.361	0.545	0.160	0.596	0.199
cywiki	102396	27825	0.806	0.732	0.654	0.407	0.692	0.493
pihwiki	737	26	0.702	0.448	0.304	0.189	0.395	0.243
hsbwiki	10095	3414	0.793	0.525	0.554	0.251	0.624	0.308
bxrwiki	1948	164	0.723	0.557	0.314	0.255	0.407	0.326
dsbwiki	2763	424	0.712	0.468	0.439	0.227	0.508	0.282
kowiki	303220	184175	0.818	0.772	0.667	0.533	0.725	0.614
ilowiki	14930	213	0.828	0.619	0.717	0.295	0.739	0.355
crhwiki	2387	2945	0.770	0.385	0.512	0.181	0.578	0.222
lnwiki	2124	852	0.634	0.558	0.333	0.247	0.417	0.321
ltgwiki	644	256	0.656	0.315	0.425	0.125	0.464	0.155
mswiki	243883	95045	0.907	0.817	0.807	0.587	0.847	0.662
hywiki	147969	122191	0.837	0.785	0.603	0.466	0.679	0.560
abwiki	4203	1693	0.563	0.142	0.407	0.133	0.432	0.104
nywiki	423	47	0.656	0.399	0.204	0.148	0.288	0.199
chrwiki	548	39	0.604	0.331	0.192	0.103	0.261	0.139
vowiki	120989	3565	0.890	0.424	0.751	0.262	0.800	0.295
nrmwiki	3981	447	0.589	0.398	0.387	0.172	0.429	0.202
eewiki	303	9	0.532	0.300	0.305	0.170	0.362	0.199
thwiki	100687	36381	0.822	0.760	0.669	0.540	0.730	0.615
bawiki	19864	32130	0.809	0.682	0.337	0.305	0.431	0.386
glkwiki	1193	4557	0.670	0.221	0.381	0.126	0.479	0.149
elwiki	126848	50989	0.805	0.766	0.669	0.529	0.720	0.607
ltwiki	113098	83717	0.826	0.766	0.670	0.540	0.727	0.615
maiwiki	10081	3199	0.787	0.561	0.604	0.256	0.653	0.322
minwiki	33712	188086	0.932	0.370	0.908	0.227	0.913	0.252
pdcwiki	1213	550	0.615	0.379	0.356	0.201	0.427	0.243
urwiki	127581	25176	0.852	0.735	0.748	0.498	0.785	0.567
cuwiki	671	31	0.705	0.368	0.318	0.161	0.399	0.197
mtwiki	3092	326	0.756	0.543	0.530	0.293	0.598	0.348
ugwiki	2528	1471	0.741	0.555	0.408	0.243	0.489	0.300
astwiki	86124	20724	0.858	0.792	0.648	0.461	0.718	0.557
scnwiki	20145	3939	0.791	0.615	0.589	0.302	0.657	0.379
be_x_oldwiki	39412	30822	0.807	0.743	0.589	0.443	0.662	0.531
srnwiki	841	181	0.541	0.311	0.329	0.149	0.361	0.176
afwiki	72733	18495	0.794	0.763	0.509	0.399	0.581	0.486
etwiki	121204	85314	0.797	0.740	0.615	0.494	0.680	0.575
angwiki	2889	111	0.714	0.585	0.386	0.267	0.478	0.340
nnwiki	120378	32000	0.844	0.737	0.733	0.509	0.773	0.580
mrwiki	36219	16129	0.824	0.695	0.603	0.392	0.682	0.478
rowiki	256232	149010	0.862	0.796	0.747	0.546	0.790	0.627
newiki	20311	11265	0.767	0.627	0.593	0.333	0.651	0.412
zh_yuewiki	57260	27158	0.802	0.728	0.623	0.452	0.690	0.537
azwiki	101834	53357	0.827	0.767	0.620	0.456	0.690	0.551
papwiki	1665	238	0.657	0.499	0.271	0.200	0.348	0.258
stqwiki	3500	462	0.684	0.492	0.484	0.282	0.543	0.325
barwiki	19842	10734	0.726	0.593	0.604	0.343	0.644	0.408
liwiki	9950	3000	0.733	0.603	0.554	0.332	0.606	0.403
sahwiki	7071	4709	0.707	0.558	0.310	0.232	0.397	0.301
jbowiki	1110	46	0.658	0.459	0.340	0.194	0.415	0.248
tyvwiki	981	1081	0.603	0.323	0.198	0.119	0.265	0.152
lezwiki	2343	1627	0.611	0.435	0.350	0.223	0.388	0.264
euwiki	283636	74972	0.908	0.807	0.781	0.499	0.828	0.593
aswiki	5266	1252	0.729	0.537	0.525	0.229	0.580	0.291
suwiki	10993	44595	0.741	0.590	0.571	0.310	0.615	0.375
ruwiki	852420	764561	0.891	0.861	0.741	0.674	0.794	0.736
azbwiki	228938	5454	0.825	0.621	0.587	0.347	0.655	0.395
ladwiki	3135	391	0.633	0.448	0.394	0.242	0.454	0.284
miwiki	2954	4192	0.588	0.276	0.416	0.132	0.430	0.158
biwiki	1112	71	0.563	0.387	0.170	0.115	0.227	0.159
bjnwiki	1309	1261	0.628	0.464	0.316	0.178	0.379	0.225
fawiki	609314	116590	0.872	0.788	0.721	0.570	0.778	0.638
kbpwiki	1494	99	0.565	0.428	0.177	0.136	0.242	0.184
gomwiki	794	2162	0.482	0.170	0.117	0.027	0.174	0.043
newwiki	22800	49051	0.877	0.588	0.618	0.265	0.658	0.308
pswiki	7010	3288	0.731	0.558	0.466	0.255	0.539	0.313
nds_nlwiki	5004	1921	0.646	0.430	0.470	0.223	0.512	0.266
vlswiki	5523	1597	0.638	0.458	0.539	0.257	0.549	0.307
simplewiki	156742	5559	0.821	0.757	0.655	0.547	0.722	0.620
orwiki	12196	3178	0.741	0.437	0.610	0.262	0.643	0.297
lowiki	2171	771	0.683	0.537	0.371	0.204	0.450	0.264
fiwiki	330250	153521	0.822	0.770	0.684	0.571	0.738	0.641
lgwiki	633	208	0.611	0.304	0.249	0.113	0.310	0.142
kuwiki	12405	14302	0.717	0.618	0.493	0.308	0.563	0.384
glwiki	102666	61259	0.828	0.803	0.645	0.482	0.707	0.577
tumwiki	548	9	0.861	0.357	0.418	0.125	0.462	0.163
lmowiki	34801	4774	0.844	0.595	0.708	0.302	0.746	0.362
smwiki	681	110	0.691	0.412	0.370	0.218	0.454	0.268
zuwiki	1977	150	0.748	0.406	0.496	0.145	0.559	0.193
zeawiki	4275	417	0.777	0.413	0.631	0.225	0.665	0.261
pagwiki	2284	144	0.849	0.256	0.752	0.117	0.781	0.118
dtywiki	2259	836	0.756	0.498	0.413	0.150	0.499	0.212
pamwiki	7901	566	0.784	0.438	0.635	0.260	0.685	0.300
huwiki	324058	144786	0.862	0.788	0.760	0.550	0.799	0.627
arcwiki	1428	248	0.727	0.498	0.428	0.266	0.515	0.321
oswiki	7795	3911	0.692	0.483	0.452	0.274	0.513	0.307
siwiki	9544	4406	0.736	0.593	0.505	0.297	0.579	0.375
zh_classicalwiki	7704	2335	0.715	0.590	0.516	0.286	0.570	0.354
csbwiki	4377	607	0.615	0.350	0.389	0.159	0.435	0.188
fowiki	11315	1853	0.757	0.637	0.538	0.286	0.592	0.356
furwiki	2730	529	0.634	0.427	0.414	0.217	0.476	0.262
xhwiki	749	112	0.643	0.352	0.443	0.169	0.497	0.210
alswiki	20743	6382	0.769	0.635	0.659	0.373	0.693	0.448
lijwiki	3140	592	0.691	0.483	0.359	0.203	0.440	0.262
tnwiki	540	77	0.692	0.180	0.319	0.098	0.431	0.121
enwiki	5764873	297117	0.868	0.826	0.794	0.685	0.825	0.735
arzwiki	432317	146835	0.871	0.693	0.578	0.273	0.667	0.349
snwiki	1670	2742	0.662	0.492	0.255	0.211	0.347	0.274
omwiki	503	223	0.622	0.344	0.322	0.144	0.387	0.178
diqwiki	11771	5821	0.757	0.607	0.560	0.320	0.616	0.381
udmwiki	3744	1112	0.700	0.380	0.300	0.168	0.398	0.214
mwlwiki	2931	445	0.716	0.532	0.385	0.224	0.471	0.288
brwiki	56010	11856	0.745	0.654	0.600	0.400	0.647	0.468
lawiki	115712	17117	0.853	0.728	0.681	0.436	0.747	0.521
cebwiki	562536	4193784	0.879	0.547	0.802	0.331	0.822	0.362
guwiki	8568	20068	0.739	0.512	0.571	0.226	0.615	0.285
scwiki	5687	789	0.737	0.626	0.414	0.276	0.505	0.363
quwiki	18212	3905	0.796	0.670	0.588	0.362	0.655	0.439
ndswiki	27322	15962	0.727	0.606	0.621	0.390	0.654	0.443
tkwiki	4860	482	0.749	0.502	0.466	0.182	0.528	0.239
rnwiki	251	202	0.434	0.255	0.128	0.130	0.181	0.155
iewiki	4574	238	0.716	0.467	0.457	0.267	0.532	0.310
gagwiki	1787	864	0.701	0.388	0.455	0.219	0.518	0.251
mywiki	16595	27724	0.747	0.625	0.531	0.320	0.587	0.392
dvwiki	1994	837	0.701	0.400	0.410	0.174	0.476	0.211
bewiki	92051	99095	0.836	0.787	0.657	0.516	0.720	0.601
stwiki	579	52	0.727	0.324	0.500	0.151	0.559	0.186
nsowiki	5213	2945	0.903	0.349	0.863	0.224	0.872	0.246
pawiki	24216	7494	0.747	0.665	0.470	0.306	0.555	0.398
bhwiki	6843	189	0.775	0.616	0.595	0.307	0.648	0.365
anwiki	27892	8435	0.855	0.763	0.653	0.371	0.713	0.466
bowiki	2491	3045	0.736	0.523	0.358	0.260	0.447	0.309
kbdwiki	1189	374	0.746	0.377	0.389	0.145	0.456	0.188
lbwiki	38341	19681	0.778	0.670	0.629	0.376	0.678	0.450
novwiki	1612	36	0.691	0.400	0.461	0.232	0.519	0.273
kawiki	97184	36816	0.822	0.778	0.654	0.515	0.713	0.596
itwiki	1079796	527130	0.857	0.806	0.759	0.612	0.798	0.677
fiu_vrowiki	3286	2225	0.684	0.606	0.328	0.232	0.409	0.306
olowiki	2298	851	0.548	0.457	0.238	0.188	0.294	0.235
wawiki	5353	7420	0.643	0.542	0.447	0.300	0.511	0.359
mgwiki	77707	14584	0.686	0.487	0.195	0.158	0.255	0.186
cawiki	431728	215389	0.856	0.795	0.742	0.584	0.786	0.657
cowiki	4541	1150	0.660	0.410	0.404	0.232	0.476	0.273
szywiki	575	16	0.650	0.449	0.225	0.115	0.308	0.168
zh_min_nanwiki	339214	63327	0.936	0.747	0.816	0.487	0.865	0.555
eowiki	191856	88212	0.819	0.748	0.692	0.511	0.739	0.588
frrwiki	9092	1510	0.742	0.543	0.539	0.321	0.585	0.369
sswiki	420	40	0.570	0.332	0.202	0.081	0.268	0.117
ruewiki	6478	965	0.744	0.521	0.540	0.246	0.593	0.308
pcdwiki	3640	1006	0.708	0.345	0.564	0.201	0.602	0.218
krcwiki	1877	128	0.565	0.340	0.296	0.145	0.351	0.181
shwiki	220895	230251	0.849	0.766	0.739	0.523	0.778	0.599
wuuwiki	28790	2181	0.760	0.696	0.357	0.262	0.467	0.356
scowiki	54561	2066	0.840	0.794	0.604	0.474	0.688	0.568
hewiki	200222	65956	0.825	0.796	0.653	0.536	0.719	0.624
vewiki	233	105	0.820	0.374	0.723	0.239	0.749	0.273
kvwiki	3784	1512	0.686	0.381	0.325	0.210	0.404	0.245
vepwiki	5491	943	0.772	0.617	0.318	0.316	0.423	0.391
hrwiki	127557	69825	0.811	0.754	0.661	0.518	0.719	0.598
roa_rupwiki	1031	118	0.647	0.381	0.346	0.186	0.414	0.227
cswiki	291075	164120	0.828	0.784	0.697	0.557	0.748	0.633
bat_smgwiki	6078	10610	0.689	0.598	0.412	0.239	0.483	0.311
fjwiki	811	17	0.652	0.399	0.259	0.197	0.324	0.226
bnwiki	71238	14609	0.818	0.732	0.655	0.445	0.715	0.534
extwiki	2793	324	0.642	0.538	0.347	0.240	0.420	0.299
rwwiki	1576	167	0.764	0.445	0.353	0.201	0.435	0.252
iowiki	28385	823	0.762	0.691	0.502	0.334	0.582	0.419
pnbwiki	41271	11055	0.794	0.667	0.577	0.349	0.645	0.426
szlwiki	11406	40270	0.753	0.550	0.673	0.293	0.683	0.344
map_bmswiki	3337	9913	0.768	0.458	0.691	0.191	0.693	0.233
mdfwiki	836	298	0.718	0.391	0.157	0.126	0.221	0.166
bgwiki	186520	76175	0.822	0.762	0.696	0.534	0.744	0.609
trwiki	253790	97492	0.825	0.768	0.683	0.569	0.741	0.639
bclwiki	8717	996	0.748	0.496	0.582	0.222	0.625	0.273
cvwiki	19213	23460	0.781	0.630	0.368	0.324	0.459	0.396
kabwiki	4052	539	0.753	0.525	0.487	0.257	0.558	0.312
crwiki	81	5	0.546	0.141	0.299	0.083	0.368	0.096
bugwiki	13956	67	0.994	0.305	0.982	0.226	0.985	0.239
hifwiki	5472	4049	0.728	0.562	0.486	0.232	0.540	0.284
mhrwiki	5579	4470	0.726	0.435	0.346	0.217	0.423	0.257
bmwiki	532	54	0.675	0.330	0.266	0.152	0.323	0.189
ganwiki	5122	1249	0.720	0.474	0.538	0.260	0.589	0.303
mrjwiki	9123	1012	0.693	0.279	0.422	0.190	0.496	0.204
gawiki	49403	3230	0.824	0.746	0.571	0.337	0.639	0.426
srwiki	266936	365258	0.841	0.768	0.729	0.519	0.768	0.597
gvwiki	4693	288	0.717	0.514	0.534	0.306	0.579	0.352
dzwiki	172	6	0.527	0.257	0.328	0.154	0.345	0.168
akwiki	669	19	0.803	0.380	0.241	0.131	0.310	0.172
myvwiki	4374	1695	0.724	0.372	0.265	0.173	0.337	0.209
yowiki	18282	13459	0.711	0.592	0.499	0.353	0.555	0.405
ttwiki	30684	58295	0.793	0.665	0.347	0.360	0.434	0.435
nlwiki	882260	1129948	0.873	0.791	0.785	0.599	0.821	0.665
gorwiki	398	2180	0.741	0.182	0.604	0.141	0.639	0.141
tetwiki	912	546	0.576	0.324	0.387	0.172	0.427	0.204
ptwiki	735185	294143	0.861	0.800	0.749	0.603	0.795	0.672
hiwiki	75110	60232	0.872	0.825	0.724	0.558	0.782	0.650
ocwiki	77503	8050	0.855	0.703	0.740	0.398	0.777	0.483
koiwiki	2308	1043	0.740	0.252	0.320	0.156	0.405	0.159
warwiki	309820	946155	0.965	0.583	0.887	0.305	0.909	0.351
jamwiki	1379	14	0.742	0.541	0.326	0.195	0.418	0.258
nowiki	362132	170168	0.830	0.767	0.715	0.556	0.760	0.630
tawiki	78575	48093	0.823	0.711	0.638	0.415	0.706	0.506
knwiki	16050	8460	0.729	0.595	0.479	0.256	0.559	0.334
tpiwiki	1514	32	0.693	0.399	0.322	0.165	0.383	0.196
napwiki	13422	1045	0.784	0.328	0.752	0.215	0.749	0.224
cdowiki	10707	4443	0.826	0.624	0.539	0.292	0.612	0.354
plwiki	867196	538486	0.861	0.803	0.755	0.596	0.797	0.667
avwiki	1709	475	0.643	0.340	0.343	0.170	0.396	0.204
lvwiki	67662	33783	0.828	0.756	0.658	0.509	0.719	0.588
gnwiki	3509	234	0.701	0.603	0.402	0.286	0.482	0.346
tcywiki	569	542	0.552	0.282	0.312	0.094	0.363	0.126
igwiki	1032	97	0.611	0.378	0.349	0.158	0.422	0.203
sewiki	6548	961	0.730	0.533	0.592	0.273	0.612	0.320
svwiki	842490	2888049	0.864	0.777	0.769	0.570	0.806	0.640
hakwiki	8088	959	0.829	0.653	0.597	0.335	0.671	0.407
klwiki	713	8	0.642	0.321	0.405	0.135	0.454	0.166
chywiki	562	26	0.598	0.358	0.274	0.178	0.352	0.210
mznwiki	10576	2406	0.799	0.568	0.588	0.293	0.657	0.353
tewiki	21806	46667	0.802	0.595	0.621	0.285	0.681	0.356
roa_tarawiki	8733	511	0.927	0.356	0.868	0.142	0.880	0.179
tiwiki	101	18	0.482	0.237	0.110	0.140	0.158	0.152
bpywiki	19448	5554	0.899	0.401	0.823	0.216	0.833	0.225
rmywiki	603	41	0.674	0.326	0.460	0.176	0.514	0.207
tlwiki	58722	12949	0.764	0.689	0.486	0.385	0.551	0.453
piwiki	316	96	0.632	0.259	0.581	0.239	0.603	0.245
ikwiki	223	25	0.627	0.216	0.173	0.084	0.247	0.109
chwiki	421	8	0.526	0.240	0.180	0.116	0.250	0.150
tywiki	797	148	0.773	0.276	0.245	0.109	0.347	0.144
hawiki	4680	225	0.756	0.526	0.504	0.251	0.579	0.317
swwiki	36588	21867	0.767	0.664	0.533	0.412	0.608	0.482
zhwiki	606632	505069	0.907	0.875	0.810	0.691	0.850	0.755
sawiki	7904	3181	0.687	0.444	0.537	0.194	0.558	0.233
nawiki	1300	50	0.695	0.393	0.307	0.173	0.392	0.221
frpwiki	3471	364	0.667	0.386	0.504	0.182	0.531	0.207
inhwiki	978	184	0.699	0.492	0.295	0.168	0.380	0.228
sowiki	3719	1493	0.699	0.573	0.402	0.279	0.489	0.358
iswiki	33927	15283	0.788	0.744	0.562	0.431	0.643	0.527
dawiki	175503	83775	0.820	0.775	0.693	0.522	0.740	0.605
kgwiki	1042	79	0.717	0.428	0.354	0.165	0.437	0.210
fywiki	26924	16810	0.740	0.682	0.538	0.381	0.606	0.465
lbewiki	914	236	0.718	0.264	0.361	0.107	0.436	0.123
xalwiki	1825	42	0.565	0.300	0.162	0.094	0.184	0.109
pflwiki	2351	251	0.858	0.292	0.779	0.111	0.797	0.147
viwiki	520274	723339	0.911	0.793	0.817	0.584	0.855	0.655
gdwiki	13512	1485	0.773	0.620	0.587	0.338	0.642	0.408
twwiki	618	31	0.590	0.365	0.259	0.192	0.337	0.233
kshwiki	2200	609	0.506	0.380	0.277	0.151	0.327	0.197
slwiki	111852	56157	0.844	0.758	0.723	0.512	0.769	0.591
towiki	694	942	0.725	0.297	0.498	0.097	0.532	0.129
bswiki	53571	29053	0.823	0.726	0.686	0.442	0.734	0.526
aywiki	4426	231	0.768	0.441	0.474	0.163	0.551	0.214
pmswiki	57898	6823	0.838	0.578	0.790	0.268	0.791	0.313
tgwiki	61288	37891	0.838	0.644	0.554	0.321	0.626	0.392
iuwiki	275	20	0.614	0.337	0.236	0.143	0.314	0.182
dewiki	1178972	1243500	0.822	0.776	0.730	0.610	0.767	0.668
frwiki	1379793	836635	0.893	0.864	0.805	0.696	0.841	0.755
ckbwiki	20315	5392	0.758	0.672	0.507	0.372	0.592	0.453
yiwiki	10528	4365	0.727	0.672	0.494	0.392	0.572	0.477
rmwiki	3141	459	0.756	0.392	0.619	0.199	0.661	0.245
kaawiki	1417	160	0.660	0.454	0.278	0.174	0.364	0.231
ukwiki	597011	418505	0.854	0.803	0.742	0.599	0.783	0.666
acewiki	2484	7744	0.686	0.432	0.324	0.174	0.405	0.221
kkwiki	103091	113705	0.852	0.685	0.686	0.391	0.739	0.467
jvwiki	25358	31377	0.790	0.687	0.571	0.392	0.650	0.479
shnwiki	3583	3302	0.925	0.161	0.803	0.145	0.851	0.131
tswiki	491	134	0.567	0.339	0.328	0.153	0.390	0.186
arwiki	762928	281507	0.891	0.844	0.752	0.596	0.804	0.679
kywiki	38817	40221	0.771	0.623	0.476	0.289	0.553	0.367
dinwiki	99	1	0.345	0.128	0.130	0.055	0.170	0.067
lfnwiki	3409	209	0.694	0.527	0.372	0.247	0.472	0.318
sdwiki	7703	4745	0.593	0.388	0.380	0.197	0.431	0.223
cbk_zamwiki	2645	56	0.823	0.333	0.651	0.181	0.677	0.207
xmfwiki	12402	1410	0.780	0.686	0.519	0.369	0.594	0.445
htwiki	36776	21028	0.801	0.541	0.517	0.224	0.601	0.272
mkwiki	71377	33508	0.826	0.751	0.659	0.492	0.720	0.572
sgwiki	207	12	0.449	0.241	0.263	0.109	0.302	0.138
pntwiki	422	21	0.640	0.259	0.223	0.122	0.290	0.149
nvwiki	14291	839	0.868	0.362	0.770	0.191	0.798	0.219
hywwiki	4674	3010	0.589	0.446	0.355	0.165	0.392	0.213
kswiki	182	26	0.604	0.208	0.331	0.069	0.400	0.094
mnwiki	14857	3750	0.783	0.689	0.531	0.356	0.609	0.441
jawiki	527883	673018	0.835	0.792	0.716	0.575	0.762	0.648
nahwiki	5874	931	0.726	0.559	0.502	0.260	0.564	0.325
adywiki	338	52	0.770	0.453	0.281	0.209	0.365	0.268
atjwiki	649	452	0.562	0.128	0.319	0.038	0.349	0.046
kwwiki	3848	136	0.747	0.487	0.475	0.247	0.539	0.300
iawiki	18193	4011	0.804	0.648	0.629	0.346	0.682	0.417
kmwiki	2646	4405	0.735	0.536	0.466	0.213	0.537	0.285
zawiki	1597	154	0.637	0.361	0.325	0.148	0.389	0.182
vecwiki	55953	1503	0.919	0.507	0.887	0.301	0.896	0.353
lrcwiki	1096	3239	0.743	0.366	0.353	0.139	0.446	0.181
idwiki	288752	235923	0.853	0.794	0.708	0.571	0.767	0.649
skwiki	154422	77274	0.851	0.728	0.738	0.503	0.782	0.578
cewiki	67020	186091	0.900	0.551	0.434	0.262	0.495	0.311
ffwiki	213	7	0.689	0.273	0.239	0.097	0.301	0.125
wowiki	1095	139	0.634	0.427	0.307	0.186	0.383	0.237
eswiki	1015429	531111	0.900	0.868	0.801	0.682	0.842	0.748
hawwiki	2504	777	0.794	0.438	0.515	0.179	0.577	0.233
uzwiki	97534	36344	0.847	0.674	0.597	0.322	0.660	0.393

Performance by Number of Outlinks

To be precise, outlinks that were mappable to Wikidata IDs. They may or may not have been in the model's vocabulary.

# outlinks	Had Groundtruth	No Groundtruth	micro precision	macro precision	micro recall	macro recall	micro f1	macro f1
1	229821	222672	0.587	0.611	0.217	0.182	0.299	0.262
2	407078	499090	0.569	0.565	0.388	0.308	0.443	0.379
3	500734	434597	0.635	0.610	0.487	0.401	0.539	0.467
4	602630	490467	0.705	0.646	0.591	0.473	0.629	0.527
5	589201	542381	0.717	0.658	0.602	0.496	0.644	0.549
6	675404	679716	0.757	0.693	0.643	0.529	0.687	0.584
7	804155	907277	0.784	0.713	0.652	0.542	0.704	0.601
8	765228	1127214	0.800	0.729	0.679	0.562	0.726	0.620
9	749086	1166286	0.811	0.743	0.700	0.576	0.745	0.635
10-19	6751252	7412010	0.852	0.786	0.749	0.611	0.791	0.673
20-29	3988634	3230575	0.869	0.816	0.761	0.623	0.804	0.692
30-39	2557281	1751152	0.874	0.830	0.760	0.629	0.806	0.701
40-49	1740596	687020	0.877	0.844	0.759	0.635	0.806	0.708
50+	9606934	3373909	0.903	0.874	0.759	0.635	0.816	0.717