Image filter referendum/Results/en

The referendum ended 30 August 2011. No more votes will be accepted.
The results were announced on 3 September 2011.

Image filter referendum

Organization

Introduction

edit

The election committee for the personal image filter plebiscite publishes and distributes this preliminary and interim report, and will publish a full report once final analysis is done. The data for this analysis was provided by Software in the Public Interest, a neutral and trusted third party, who held the secure keys for encryption and did not decrypt the results until the conclusion of the election. The totals shown here were certified by Michael Schultheiss, who serves as the treasurer of Software in the Public Interest.

The committee thanks Michael Schultheiss of SPI for his diligent and timely assistance, and Andrew Garrett of WMF for his technical assistance and expertise.

Results

edit

There were 24,146 votes cast, making this the largest exercise of its type for the Wikimedia Foundation. Of those, 123 votes were struck by the committee, including three test votes by committee members, leaving 24,023 votes accepted. The votes that were struck were disallowed due to confirmed instances of either intentional or accidental double voting. The voting total does not include ballots automatically cancelled when users revisited their votes.

There were two sections to this ballot: the first a series of six statements asking that voters rate their level of agreement with the statement on a scale of 0–10 (10 means strong support), and the second a free text comment field. The two sections were analyzed separately and are discussed below.

Numerical results

edit

For the first question:

 
Important to offer this feature.
It is important for the Wikimedia projects to offer this feature to readers.
Total ? Total 0 Total 1 Total 2 Total 3 Total 4 Total 5 Total 6 Total 7 Total 8 Total 9 Total 10 Votes Votes with a preference Average of those with a preference Median
269 3763 790 1163 978 715 2819 1800 2670 2957 1308 4791 24023 23754 5.7 6
1.12 % 15.66 % 3.29 % 4.84 % 4.07 % 2.98 % 11.73 % 7.49 % 11.11 % 12.31 % 5.44 % 19.94 %
 
Logged in and logged out.
It is important that the feature be usable by both logged-in and logged-out readers.
Total ? Total 0 Total 1 Total 2 Total 3 Total 4 Total 5 Total 6 Total 7 Total 8 Total 9 Total 10 Votes Votes with a preference Average of those with a preference Median
627 3394 630 852 715 572 2393 1115 1921 2682 1947 7175 24023 23396 6.4 8
2.61 % 14.13 % 2.62 % 3.55 % 2.98 % 2.38 % 9.96 % 4.64 % 8.00 % 11.16 % 8.10 % 29.87 %
 
User flags images.
It is important that individuals be able to report or flag images that they see as controversial, that have not yet been categorized as such.
Total ? Total 0 Total 1 Total 2 Total 3 Total 4 Total 5 Total 6 Total 7 Total 8 Total 9 Total 10 Votes Votes with a preference Average of those with a preference Median
614 2809 617 829 684 697 2159 1790 2602 3405 2137 5680 24023 23409 6.4 7
2.56 % 11.69 % 2.57 % 3.45 % 2.85 % 2.90 % 8.99 % 7.45 % 10.83 % 14.17 % 8.90 % 23.64 %
 
Culturally neutral.
It is important that the feature be culturally neutral
as much as possible, it should aim to reflect a global or multicultural view of what imagery is potentially controversial.
Total ? Total 0 Total 1 Total 2 Total 3 Total 4 Total 5 Total 6 Total 7 Total 8 Total 9 Total 10 Votes Votes with a preference Average of those with a preference Median
1399 2048 350 466 394 507 2134 895 1441 2271 2116 10002 24023 22624 7.4 9
5.82 % 8.53 % 1.46 % 1.94 % 1.64 % 2.11 % 8.88 % 3.73 % 6.00 % 9.45 % 8.81 % 41.64 %
 
Reversible.
It is important that hiding be reversible
readers should be supported if they decide to change their minds.
Total ? Total 0 Total 1 Total 2 Total 3 Total 4 Total 5 Total 6 Total 7 Total 8 Total 9 Total 10 Votes Votes with a preference Average of those with a preference Median
471 548 40 63 61 70 433 253 551 1383 1812 18338 24023 23552 9.3 10
1.96 % 2.28 % 0.17 % 0.26 % 0.25 % 0.29 % 1.80 % 1.05 % 2.29 % 5.76 % 7.54 % 76.34 %
 
Ease of choice.
It is important that the feature allow readers to quickly and easily choose which types of images they want to hide (e.g., 5-10 categories), so that people could choose for example to hide sexual imagery but not violent imagery.
Total ? Total 0 Total 1 Total 2 Total 3 Total 4 Total 5 Total 6 Total 7 Total 8 Total 9 Total 10 Votes Votes with a preference Average of those with a preference Median
595 2193 359 442 407 434 1804 1260 2399 3662 2890 7578 24023 23428 7.2 8
2.48 % 9.13 % 1.49 % 1.84 % 1.69 % 1.81 % 7.51 % 5.24 % 9.99 % 15.24 % 12.03 % 31.54 %

Analysis

edit

Of those who expressed a preference in their votes (that is, they voted anything other than “not enough information”), the community expressed the overwhelming desire that the hiding feature be reversible (median 10), strong support for the suggestion that readers could easily and quickly choose categories (median 8), that the feature be culturally neutral (median 9), and that both logged in and logged out users have access to the feature (median 8). Voters expressed slightly less support for the notion that readers could flag images (median 7). Support for reversible hiding was completely clear, while the other vote distributions were bimodal to varying degrees, most notably for overall importance.

With regard to the assessment of the importance of the filter by respondents, some further analysis was done. The average vote on this question (among those who held an opinion) was 5.74, and the median vote was 6, indicating that a greater number of respondents considered this to be important than unimportant, though the difference is not overwhelming. Of the 8554 voters (36.0% of the total 23754) who rated this either a 10 or a 0, 4791 voters (56.0%) rated it a “10” and 3763 votes (44.0%) rated it a “0”.

A further analysis of the responses to the “importance” question, comparing votes at the extremes of the spectrum, working towards the middle, revealed the following results:

Selected results Total
"10" "0"
Votes 4791 (56.0 %) 3763 (44.0 %) 8554
"10 + 9" "0 + 1"
Votes 6099 (57.2 %) 4553 (42.8 %) 10652
"10 + 9 + 8" "0 + 1 + 2"
Votes 9056 (61.3 %) 5716 (38.7 %) 14772
"10 + 9 + 8 + 7" "0 + 1 + 2 + 3"
Votes 11726 (63.6 %) 6694 (36.4 %) 18420
"10 + 9 + 8 + 7 + 6" "0 + 1 + 2 + 3 + 4"
Votes 13526 (64.6 %) 7409 (35.4 %) 20935

The committee also notes the significant number of “0” votes cast for all but one question (ranging from 9% to 16%), which suggests the presence of a sizable number of voters who wished to express their opposition to the basic idea of the image hiding feature. This voting pattern coincides with the anecdotal evidence of both the comments to the Meta page on the feature and the sampled comments of the plebiscite itself.

Results of the comments section

edit

A total of 6956 users added free text comments to their votes (29% of all voters). Of these, a representative sample of 1298 comments (19%) underwent a preliminary review in order to gauge and analyze the tone and content of the comment. The remaining comments will continue to be analyzed by the committee. In addition, word frequency charts were run on the full set of comments.

Methodological note: the sample to be analyzed was created by numbering the comments sequentially in the order in which they were received and then generating a list of random numbers from atmospheric noise using http://www.random.org/integers/. The numbers generated by the randomization process were matched to comment numbers and those comments were analyzed. Translations were obtained by asking native or high fluency speakers for translations. Initial issues with importing unicode characters were resolved in time to include some of those comments in analysis. Because multiple evaluators participated in the review process, a small set of comments was evaluated several times by multiple people to ensure consistency in rating. That set was removed prior to reporting these numbers.

 
Referendum comments wordcloud

141 words were used more than 500 times in the full set of comments. Below is a table showing most frequent word use and count. Note that differences using modifiers such as “not” were analyzed as well. For instance, the word “controversial” appears 1143 times in the comments, with 19 instances of “non-controversial”, 11 instances of “uncontroversial”, 2 of “noncontroversial”, 1 of “not-controversial”, 1 of “not-so-controversial”, and 7 of “not controversial”. These numbers are representative of other modifiers sampled. These modifiers make up a small but significant percent of the words and should be evaluated more fully for the committee’s final report.

The committee was heartened to see that many non-English words were represented in the table of most frequently used words.

 
Tone

Of the 1298 comment samples rated for tone, 387 (29.82%) had a positive tone, 376 (28.97%) had a neutral tone, and 535 (41.22%) had a negative tone.

The committee was interested in a particular subset of those who used a positive tone in their comments, in particular whether the commenter made reference to whether they personally intended to use the image filter or whether they believed it was a valuable tool for others. Of that set of “positive tone” comments, 32 (18% of the subset, 2.05% of the full set) made reference to the tool as though it would be of use to the respondent personally. Another 143 (82% of the subset, 9.16% of the full set) referenced the tool as though it would be of use to others.

Comments were also reviewed to identify references to voting problems or issues with the design of the voting process/plebiscite. There were 2 references (0.13%) to voting problems in the reviewed set, and 45 (2.88%) references to concerns with the referendum process itself.

The reviewers further looked for comments that provided particular input as to implementation details, such as suggestions for categorization schemes or other ideas to be provided to the design team. There were 389 such comments in the comment set that was analyzed (24.92%).

In the context of analysis of the comments, one trend emerged very clearly: the negative comments more frequently mentioned objections on a philosophical basis, while positive comments more frequently mentioned practical concerns.

Discussion

edit

The committee is pleased overall with the use of the free text comment field. Although text-based comments provide a challenge for evaluative purposes, the value gained from this field in informing the design process is substantial. The committee wishes to also clearly state that it is fully committed to reading and evaluating every comment, and design/implementation suggestions will be anonymized and passed on to the design team.

Some comments include personally identifiable data, so the committee regrets to advise that a full dump can not be released at this time. It is our hope that a more full and scholarly analysis can be done at a later date.

What comes next?

edit

The numeric results detailing responses to the six statements are now complete and will not be amended in future reports.

The committee will continue to review and analyze the free-text comments, and will issue a final report once this task is complete.

For the Personal Image Filter Referendum Committee,

Risker 01:56, 4 September 2011 (UTC)[reply]

Appendix 1

edit
Table of word frequency
Word Occurences
the 16335
to 15770
10 12434
a 10437
of 9791
' 8908
be 8825
is 8400
and 6977
i 6971
that 6757
it 6275
0 6139
this 5851
in 5756
for 5378
not 5176
images 4274
as 3660
should 3613
or 3484
8 3253
on 2986
if 2973
would 2688
an 2663
5 2583
de 2533
wikipedia 2523
will 2414
are 2397
7 2389
9 2305
by 2279
image 2190
feature 2155
but 2143
have 2133
with 2081
they 1748
people 1690
users 1678
can 1652
6 1572
think 1572
see 1539
all 1500
what 1491
filter 1476
que 1462
content 1462
  
Word Occurences
you 1454
do 1376
some 1323
there 1321
2 1315
such 1314
no 1309
la 1288
from 1268
so 1249
their 1203
1 1200
any 1166
its 1161
important 1157
controversial 1143
-1 1132
like 1130
censorship 1127
which 1091
could 1085
dont 1083
about 1083
die 1051
who 1029
more 1027
we 1017
may 1015
at 1013
want 1012
user 987
only 986
also 972
one 967
3 934
me 902
my 891
very 885
en 879
categories 874
es 868
idea 865
neutral 865
wikimedia 817
other 806
im 800
use 793
r 781
4 779
pictures 773
them 771
  
Word Occurences
hide 763
les 736
offensive 724
even 721
good 720
view 720
way 717
because 683
make 681
information 674
n 665
am 664
just 652
un 651
culturally 648
cultural 647
nicht 645
des 644
might 644
example 639
then 633
than 628
s 626
has 616
default 603
und 601
those 596
possible 595
children 593
der 592
ich 590
able 578
being 577
sexual 577
le 576
m 571
how 568
f 553
must 551
ist 547
when 534
e 533
filtering 533
wiki 524
o 520
question 519
without 510
pas 510
certain 503
option 502
d 500

Appendix 2

edit
Vote results by project
language project votes avg_age_days avg_age_years q1votes q1avg q2votes q2avg q3votes q3avg q4votes q4avg q5votes q5avg q6votes q6avg
en wikipedia 9245 1632.84 4.47 9176 5.89 9056 6.43 9091 6.38 8810 7.27 9090 9.35 9077 7.27
meta wikipedia 381 1175.94 3.22 376 5.51 373 6.06 370 6.02 365 7.22 376 8.83 368 6.95
ja wikipedia 313 1197.4 3.28 308 7.76 308 7.48 303 7.5 303 8.4 310 9.24 309 8.56
ru wikipedia 1627 1004.81 2.75 1591 6.46 1587 6.96 1583 7.38 1535 7 1600 9.49 1592 7.87
de wikipedia 3923 1326.99 3.63 3863 4.22 3787 5.49 3781 5.35 3573 7.21 3824 9.42 3772 6.24
zh wikipedia 316 1102.78 3.02 309 7.69 308 7.32 308 8.08 307 8.47 310 9.36 308 8.16
fr wikipedia 2441 1215.31 3.33 2411 5.73 2341 6.88 2374 6.44 2281 7.97 2385 8.84 2384 7.21
ar wikipedia 94 1002.58 2.74 94 8.17 94 7.89 91 7.68 90 7.29 94 9.02 89 8.64
commons commons 220 1147.81 3.14 220 5.31 216 6.04 215 5.67 207 7.3 216 9.34 215 6.73
es wikipedia 1278 1173.2 3.21 1260 6.44 1235 6.47 1233 6.6 1218 8.11 1250 9.39 1247 7.65
vi wikipedia 28 798.91 2.19 27 7.11 24 8 25 8.4 27 7.7 25 8.84 23 8.43
en wikibooks 30 917 2.51 29 6.41 30 6.8 30 6.77 27 7.04 30 8.9 30 7.17
ca wikipedia 86 1197.43 3.28 84 6.29 85 7.25 84 5.96 80 7.61 85 9.13 84 7.37
nl wikipedia 695 1272.32 3.48 689 5.24 684 6.12 680 6.38 665 7.03 677 9.17 679 6.77
ko wikipedia 118 933.23 2.56 117 7.81 114 7.64 113 7.46 113 8.5 117 9.36 115 8.16
it wikipedia 779 1274.01 3.49 778 5.96 763 6.61 762 7.96 737 8.14 767 9.44 760 7.59
id wikipedia 49 980.52 2.68 49 8.04 48 7.42 48 7.77 47 8.53 48 8 49 7.88
pl wikipedia 642 1284.17 3.52 631 5.8 622 6.31 623 6.91 602 6.86 634 9.19 622 7.34
no wikipedia 93 1350.91 3.7 91 5.49 91 6.51 91 6.85 90 7.28 91 8.96 91 7.02
hu wikipedia 108 1201.16 3.29 107 6.6 105 7.16 100 6.14 98 6.84 106 8.88 105 8.02
he wikipedia 171 1098.57 3.01 170 7.27 169 7.58 166 7.1 153 6.48 163 8.94 163 7.8
fa wikipedia 65 953.01 2.61 64 7.56 65 7.4 64 7.2 64 7.22 64 8.98 64 8.25
bg wikipedia 21 1212.47 3.32 21 5.1 21 5.57 21 6.52 21 6.57 20 9.7 21 6.95
uk wikipedia 108 915.94 2.51 107 6.81 106 5.91 107 7.07 101 6.99 108 8.99 107 7.66
et wikipedia 21 1114.1 3.05 21 3.38 20 5.4 19 4.63 18 5.5 21 8.62 20 6.3
fi wikipedia 62 1347.23 3.69 62 5.06 60 5.8 62 6.23 60 7.83 61 9.66 61 7
pt wikipedia 226 1179.38 3.23 224 6.9 225 6.56 211 7.19 209 8.33 222 9.6 224 8.11
sv wikipedia 147 1288.26 3.53 146 3.34 140 5.34 141 4.93 136 7.76 143 9.36 140 5.88
mediawiki mediawiki 28 998.87 2.73 28 5.29 28 6.36 28 6.64 26 7.46 27 8.59 27 7.52
da wikipedia 34 1359.06 3.72 34 3.47 33 5 33 4.39 33 7.21 33 8.64 33 5.79
en wiktionary 38 1037.41 2.84 37 5.89 37 7.49 36 6.39 33 6.36 35 9.6 35 7.74
ro wikipedia 25 1297.14 3.55 25 4.84 25 5 24 6.5 24 5.75 24 9.17 25 6.68
tr wikipedia 43 1201.11 3.29 43 6.93 42 7.38 40 7.2 38 8.58 41 8.9 40 8.55
cs wikipedia 142 1297.05 3.55 140 4.69 141 6.06 138 6.29 136 6.67 139 9.71 137 7.32
consolidated multiple 426 1005.31 2.75 422 6.41 413 6.7 414 6.69 397 7.78 416 9.35 412 7.57

Note: All votes from projects which had a total vote count of less than 20 were folded into the 'consolidated projects' line item in the table above to protect the confidentiality of votes. Those projects include: ptwikisource; simplewiki; tewiki; enwikisource; sawiki; dawikisource; cywiki; cuwiki; frwikisource; specieswiki; mlwiki; itwikisource; gawiki; frwikibooks; gvwiki; ltwiki; bswiki; sqwikinews; lvwiki; orwiki; ndswiki; dewiktionary; ocwiki; swwiki; nlwikibooks; frwikinews; srwiki; incubatorwiki; slwiki; eowiki; enwikinews; dewikinews; be_x_oldwiki; simplewiktionary; lawiki; itwikiquote; hewikisource; tawiki; viwikisource; elwiki; knwiki; ptwikibooks; ttwiki; svwikisource; oswiki; hrwiki; hewikibooks; kuwiki; frwikiversity; ruwiktionary; svwiktionary; mkwiki; mswiki; eswiktionary; suwiki; frwiktionary; angwiki; enwikiversity; iawiki; enwikiquote; mtwiki; afwiktionary; ptwiktionary; plwiktionary; alswiki; kshwiki; skwiki; hewiktionary; nds_nlwiki; ckbwiki; dewikiversity; ruwikisource; cswiktionary; cawikisource; nnwiki; dewikibooks; iswiktionary; iswiki; gawiktionary; kawiki; fiwiktionary; pflwiki; hiwiki; hywiki; frwikiquote; glwiki; lmowiki; huwikisource; bewiki; lbwiki; elwikiversity; itwiktionary; svwikinews; shwiki; eowikibooks; thwiki; zhwikisource; bnwiki; lnwiki; eswikibooks; huwiktionary; viwiktionary; elwikisource; tlwiki; emlwiki; kywiki; klwiki; ladwiki; ruwikibooks; anwiki; svwikibooks; itwikibooks; ptwikinews; roa_tarawiki; arzwiki; barwiki; bgwiktionary; sowiki; afwiki; nowiktionary; huwikiquote; frrwiki; astwiki; dewikisource; lvwiktionary; zhwikiquote; betawikiversity; itwikiversity; acewiki; kywikiquote; yiwiki; vecwiki; sahwiki; cawikibooks; siwiki; aswiki; zh_yuewiki; cawiktionary; bgwikisource; kmwiki; bgwikiquote; euwiki

Appendix 3

edit
Average vote on Q1 by age of account
Age of account Average vote % of votes represented
90 days or less 6.645614035 1%
91 - 180 days 6.32183908 2%
181 days to 1 year 6.372141372 5%
1 - 2 years 6.223976109 11%
2 - 3 years 6.081090407 14%
3 - 4 years 5.844198175 15%
4 - 5 years 5.809265537 22%
5 - 6 years 5.524947784 21%
6 - 7 years 5.439243924 5%
7 + years 5.331649832 3%