Learning patterns/Vote Matching Tool (Digital-o-Mat)

A learning pattern foroutreach

Learning Pattern Library

Creating a Vote Matching Tool on digital policy

problemHow to engage the general public in digital policy topics that are relevant for Wikimedia projects.

solutionCreate a vote matching tool, i.e. a voting advice application, with questions that focus specifically on digital policy questions to raise awareness around these issues.

creator• John Weitzmann (WMDE)

created on08:23, 9 October 2017 (UTC)

What problem does the tool solve?

There are increasingly numerous vote matching tools before general elections. The general type of these tools usually doesn't cover digital policy issues at all or only on the side. Thus, the Digital-o-Mat was developed by Wikimedia Deutschland, the German Wikimedia Chapter, as a special interest vote matching tool that can be used to reach out and engage the general public around digital policy topics that are especially relevant for Wikimedia projects. The Digital-o-Mat is a tool similar to the Wahl-o-Mat which for the first time was launched in 2002 by the German Federal Agency for Civic Education (Bundeszentrale für politische Bildung, bpb) as a website in preparation for the federal elections. This general purpose Wahl-o-Mat of the bpb is an election information platform for which the political parties are asked to answer questions about their political views covering the whole political spectrum.

Afterwards, the user is asked to answer the same questions in order for the platform to identify similarities and differences between the users’ preferences and those of the political parties. These results are then presented to the user as percentage of agreements.

The intended aim of the interactive tool is to inform the users about different political agendas of the competing parties. In addition, it is possible to examine whether or not they correspond with the user’s own political views.

So why a new tool?

In the recent version of the Wahl-o-Mat, 38 questions are asked in order to cover the political spectrum. This means, considering the range of political subfields, that a single topic can not be represented in detail - if at all.

Therefore, the most influential political field for the civil engagement of Wikimedians - digital policy - is barely represented on the platform. Even the two questions concerning digital policy do not closely relate to the interests of most Wikimedians or other digital volunteers. Additionally, the topic of digital policy does not seem to be a notable part of the electoral program for most of the parties. All of this indicates that the interests of Wikimedians and other digital volunteers are largely excluded from the mainstream public discourse and, as a result, from the decision-making process before the election.

The lack of public discourse on digital policy in the run up to elections is the reason why WMDE decided to join forces with further NGO's working in the field of digital policy in the early days of 2017 to establish a Vote Match platform for digital policy. The so-called Digital-o-Mat, a tool analogous to the original Wahl-o-Mat, was designed to exclusively focus on issues of digital policy.

We have two different approaches to the Digital-o-Mat. One rather prospective, which was used for the state and federal elections in 2017 and 2018 and one retrospective approach, which was developed especially for the European election 2019.

The goal of both versions is to increase the public profile of certain digital policy issues and to give further information on the party's positions.

Despite the somewhat difficult measurability of the impact of such a tool, we can affirm that the Digital-o-Mat reached a broad audience during the election campaign (~65k pageviews/ ~40k complete answers). At the same time we are aware that the coverage could have been wider.

This is the issue of this pattern and the reason why, in order to help improving future projects, we would like to share our learnings with this learning pattern. Please note that this pattern is not dealing with the public relations work necessary to increase the outreach of an online tool, which would be a great topic for further learning patterns.

The prospective tool

This tool is useful for state or federal elections. The main focus are the different positions of the parties towards different public policy topics. It also helps to compare the answers to their election program. An advantage of this version is that fairly new or very small parties can participate as well.

1) The first step to initiate the project was to informally discuss the idea with various NGOs concerning digital policy. Discussing the issue generated a loose coalition willing to realize a Digital-o-Mat focussing on digital policy only. The catchy title “Digital-O-Mat” was somewhat key in getting everybody on board.

2a) In the time following this first step, the focus of the coalition was 1) to negotiate structure and layout of the website 2) to divide the political issues to be dealt with between NGOs and 3) to decide which political parties should be integrated into the vote match.

2b) On the logistics side, questions needed to be answered regarding budget for the project and regarding the choice of contractor to program the tool – or whether it could be developed by volunteer coders instead. It was decided, that in any case, the software must in the end be open source and available on GitHub or a similar platform.

3) Afterwards, every member of the coalition was asked to formulate a list of questions to be put to the political parties, which were based on the respective electoral programs as well as the partner NGO’s policy interests. The collaboration on creating this questionnaire took place through a etherpad, which helps everybody to stay updated about the total progress.

3b) In the 2018 version for the regional elections in Bavaria and Hesse all members of the coalition were asked to send their questions in separately. Also, the number of questions was reduced to 2 per NGO/member in order to keep the total number of questions down.

4) Then, the questionnaires were sent to those political parties which were most likely to enter the German parliament after the election. Also, make sure that you emphasize that it is important for the parties to answer the questions with yes/no/no comment (or neutral) and then give their respective explanation/proof for the answer (see learning). As a yardstick for this, we resorted to a collection of poll data from all polling institutes and selected the parties that were above the parliamentary threshold of 5 percent continuously for the entire past year. This included a controversial right-wing party, which led to intense debate amongst the coalition partners. We decided to include them nonetheless in order to not give merit to claims of any political bias of the project.

5) After receiving responses, the answers had to be checked for evidence to prevent popular but ambiguous as well as misleading statements. Without this step, we reckoned, the vote match might be abused by the participating political parties to gain extra matches by answering in ways that fit mainstream opinion but are in fact not what they propose in their electoral programs. Fortunately it turned out that no party tried this.

6) When this was done and all responses were collected, they could be transferred to the website, which is when the period of testing began. After implementing the feedback gathered and completing the final test the website was ready to go public. Initially after that, additional feedback swept in from the first batch of users, which led to another update regarding mostly clarity of the website’s explanatory texts. The retrospective tool The main focus in this version is to compare how the different parties have voted in parliament over a certain period of time. So we did not reach out to the parties and sent them a list of questions that they were supposed to answer. But instead we chose parliamentary votes concerning matters of our interest (such as data protection or e-commerce). That way we were able to look at how parties voted which could the user then help to decide whether or not a party represented his or her interests sufficiently. The reason why this version was used in order to build a Digital-o-Mat for the European Election in 2019 is that it gives the chance to compare more parties over a longer period of time. We chose this version because it simply would have been too complicated to get answers to our question from all the German parties. 1) The first steps are similar to those of the prospective tool: find partner organisations to team up with and decide on a layout, number of topics and statements, time frame, budget and deadlines. It is maybe worth mentioning that the project was a group effort and that weekly calls between all participants were initiated. 2) The second step was to match the chosen statements or questions to their respective votes in parliament (this is the crucial difference between the prospective and the retrospective tool: for the former parties are asked to answer to questions sent to them by us. The latter focusses on past votes and how the different members of parliament voted). The next step was to extract the data. We used Votewatch for this. We covered name, party in the European Government, party in Germany, vote (yes/no/abstained/or absent, if necessary) of 10 past votes. 3) The extracted data were transmitted to the website and the testing period begun. After some feedback and further testing, the website went public. Learnings 1) Team up This one should be easy to understand for Wikimedians who are used to working collaboratively. Teaming up with a coalition of NGO's gave additional reach and weight to the tool and made it a lot easier to draft the questionnaire because each NGO was in charge of supplying one or two questions dealing with their own focus area. Surely it requires additional work to coordinate with partners, but besides adding further expertise and increasing the reach of your tool, it also decreases workload through division of labour. 2) Selecting the political parties (and making it transparent) Another problem you have to deal with, and the second learning: selecting the parties to be included in the tool based on universal, abstract criteria to avoid bias. For example, 48 parties ran for the German parliament during this year’s federal election. Since there was no capacity to include all of them into the Digital-O-Mat, the partners had to make a decision about which political parties to include. It seemed necessary that, in order to be transparent and unbiased, this had to be a universal and self-explanatory selection algorithm, abstract from political preferences of the NGO coalition. The partners decided to include every party constantly ranking above the German election threshold of 5% in the polls during the last year prior to the election, and therefore had a good chance of getting elected into the new Bundestag. In that way, six parties were selected to be included on the platform and received the questionnaire. This selection algorithm (and also whether any party did not answer the questions) is a point, which should be made transparent for the users. Feedback from the community indicated that this should be highlighted directly on the webpage with the questions, since many users do not take a look for background information on the website. 3) Small parties vs. big parties When using the retrospective tool small parties might have a slight disadvantage. This is due to the fact that if there is only one member that is able to vote it means that this person decides whether the party supports a proposal or is against it whereas big parties do not have that problem. There is no proper solution for this matter but it is something that we got feedback on from party members. 4) Advantages and disadvantages of the retrospective tool The main point here is that it is possible to cover more parties: because you don't have to reach out to them but instead focus on their past votings - this makes it easier to cover more data. However, this is also a problem: since the base are past votes, newer parties can not be taken into account. Another learning concerning the parties is: some parties are very small and if their respective MEP is absent this can quickly lead to a distorted image. Also, if you launch the website, explain in detail how the retrospective tool works since this has - in our experience - led to some confusion. 5) Simplicity and precision of questions Third learning: formulating the questions. It will most likely be impossible to formulate questions that appear appropriate to everyone, but at least two contrary attributes stick to our minds considering the feedback of the community. First, the questions should be as simple as possible. But, at the same time, they have to be as explicit as possible, not giving the parties a chance to knowingly avoid unpopular statements by formulating answers very vaguely. Make sure that you emphasize that it is important for the parties to answer with yes/no/no comment (or neutral) and then give their respective explanation/proof for the answer (see no. 6). Also, think about if you want to formulate the questions in a positive, negative or neutral way and how your choice might effect the outcome and the respective answers of the parties. 6) Make sure to check the answers Fourthly, we had to learn that the parties sometimes do not check their answers again so we had to do it for them which took up a lot of time. So make sure to check whether the initial answer (yes/no/neutral) matches the following explanation and that the general answer again matches the political program of the party. 7) “We apologize for the delay” We also learned to send the list of questions to the political parties early - very early. WMDE experienced plenty of deadlines passing without any reaction by the parties. This may differ by country, but it still seems to be a good idea to plan some extra time for the parties to answer your questions and to nag them again and again if necessary. It turned out it really helps to phone up and tell them you need the answer otherwise their party unfortunately might not be included - just to make sure they take your request seriously. Also, make sure to send the questions to the respective member of the party who is assigned to topics like Public Policy or Media and Politics and not a general email address. This may help you to get the answers a bit quicker. 8) Do not be a campaign assistant This learning is quite similar to the third and fourth one, do not let the political parties fool you. It seems necessary to check if the answers provided by the political parties fit their political program and actions. That is why we asked all parties for evidence that the answers provided are actually in line with their party position, e.g. excerpts from their programs, passed resolutions or official statements. 9) Test, and then test again Last but not least: For most of the people a vote match is a tool for one-time usage only, they don’t pop by twice for any given election. This means that the website should be finished and feedback from testing should already be included when making the vote match public to prevent wasting a part of the valuable public attention to a still imperfect tool. 10) How to handle the GDPR Something that you want to keep in mind is data protection. So think about which kinds of data you will collect with you tool, if they serve your purpose and how you can protect them. Also, there are several possibilities to evaluate the collected data so maybe think about what you aim at when using the tool. Things to consider timing simple language short but detailed questions transparent choice of parties testing and quick implementation of feedback checking answers (generally) and for evidence When to use prior to national or regional elections Endorsements For further information about this, please feel free to contact: politik@wikimedia.de See also Related patterns How to campaign on a political issue External links https://digital-o-mat.de/ Sourcecode Digital-O-Mat https://www.votewatch.eu/ References http://www.votematch.eu/ http://www.bpb.de/veranstaltungen/netzwerke/nece/155659/vote-match-europe Members of the coalition Wikimedia Germany Bündnis Freie Bildung Der Chaos Computer Club Digitale Gesellschaft Freifunk Free Software Foundation Europe Open Knowledge Foundation Deutschland