Grants:Programs/Wikimedia Alliances Fund/The missing link:Incorporating policy reports into the the free knowledge ecosystem/Midpoint Report
Report Status: Accepted
Due date: 2023-01-15T00:00:00Z
Funding program: Wikimedia Alliances Fund
Report type: Midterm
This is an automatically generated Meta-Wiki page. The page was copied from Fluxx, the grantmaking web service of Wikimedia Foundation where the user has submitted their midpoint report. Please do not make any changes to this page because all changes will be removed after the next update. Use the discussion page for your feedback. The page was created by CR-FluxxBot.
General information edit
This form is for organizations receiving Wikimedia Community Funds (General Support) or Wikimedia Alliances Funds to report on their mid-term learning and results. See the Wikimedia Alliances Fund application if you want to review the initial proposal.
- Name of Organization: Analysis & Policy Observatory (APO)
- Title of Proposal: The missing link: Incorporating policy reports into the the free knowledge ecosystem
- Amount awarded: 41234 USD, 58214.83 AUD
- Amount spent: 20802 AUD
Part 1 Understanding your work edit
1. Briefly describe how your strategies and activities proposed were implemented and if any changes to what was proposed are worth highlighting?
- Step A: Establish the project - a volunteer advisory group has been established. It was difficult to find someone to commit to the Wikimedia Facilitator role, so this is now being carried out by the Project Lead with assistance from the advisory group. The Wikimedians in Residence have completed their Wikimedia editor training.
To date, we have been unable to find a First Nations lived experience expert to provide advice and direction due to the demands on relevant experts in this field. Step B: Verifying policy organisations as reliable sources - the first step involved creating a spreadsheet of nominated policy organisations to include and noting whether they have been mentioned in the Reliable Sources Noticeboard. Only one of the organisations had been so we had to work out a way to determine if the rest would be considered reliable sources. We engaged with the Reliable Sources Noticeboard, WikiProject Source MetaData, Village Pump, and Tiers of Reliability. The feedback we received was that we shouldn’t seek to determine the reliability of each organisation. Instead, we developed principles to apply to the selection of organisations and their reports to include in the upload to Wikidata. We also sought feedback at the Worlds of Wikimedia conference in Sydney. Some participants argued that we should just upload all content from APO, and we shouldn’t bother with the filtering. However, we don’t think it’s useful to upload metadata of reports that no-one will use because they are not reliable. Also, editors may need assistance in determining what is a reliable source, especially when it comes to grey literature where the guidelines aren’t always clear-cut. While we are filtering, our criteria are not overly restrictive. The principles we have agreed are: 1. Advocacy organisations will be excluded. 2. The reports are based on research, or for policy documents they are published by organisations that have a remit for or governance over the policy area.
2. Were there any strategies or approaches that you feel are being effective in achieving your goals?
- We have come to this project recognising that we have a lot to learn about the world of Wikimedia and this has proven to be a useful approach. While some of the feedback has come across quite abrupt, we have continued to keep an open mind and draw out the main points being communicated. For example, the feedback appeared to be that we didn't have the right approach and it was delivered as “you can’t do this”. So we took on board the main objections and found an alternative approach that was agreeable. In return, the feedback we received was: “Your candor and willingness to apply fact-based principles is refreshing. Best wishes” 69.203.140.37 (talk) 00:46, 1 October 2022 (UTC).
So the two main “do’s” we would recommend are: 1) be open to feedback; and 2) be flexible and the main “don’t” is don’t take it personally!
3. What challenges or obstacles have you encountered so far?
- It has been difficult to determine what constitutes a reliable source as there is a range of guidance out there that isn’t aligned. For example we have found two different reliability scales on the Reliable Sources Noticeboard and Tiers of Reliability page. As well as other further overlapping guidance around on verifiability, Reliable sources, Perennial sources, and No original research (Reliable sources).
We have found the best approach is to determine which part of the citation we are trying to determine the reliability for: the work itself; the author; the type of publication; or the publisher which can fall into two categories of commercial publisher or non-commercial publisher which is often the organisation.
We have found it challenging to identify the appropriate forums to engage Wikimedians – as there is no list of the various options. Our strategy has been to start with a couple of forums we are aware of and then Wikimedians have suggested more appropriate forums for us to engage with. However, we don’t know if there are any other forums out there we are missing.
In Australia, there are is an overwhelming demand on the time and input of First Nations people – particularly as an accepted principle in academic and policy contexts is “nothing about us, without us”. We had some people in mind who would be ideal in contributing their First Nations input to the project, however, they are already busy with another Wikimedia project. And there doesn’t seem to be many First Nations people engaged with Wikimedia in Australia. Our new strategy is to engage with First Nations academics at Swinburne University to see if there are students who would be willing to get involved.
4. Please describe how different communities are participating and being informed about your work.
- The Wikimedia community has been engaged in the project with presentations given at a Wikimedia Australia meeting, the Worlds of Wikimedia Conference, and the ESEAP Conference. This was a good opportunity to meet other Wikimedians and to get advice on various aspects of the project. APO also tweeted about these presentations and participation in these forums.
Our volunteer advisory group is made up of members from Wikimedia Australia.
APO published the WikiProject Policy Reports page on Wikidata. This is being used to publicly document our progress on the project. We have engaged in various Wikimedia forums (Reliable Sources Noticeboard, WikiProject Source MetaData, Village Pump, and Tiers of Reliability) to facilitate participation in the project.
A blog about the project was published on APO’s blog and this was promoted to our 15,000 newsletter subscribers, as well as on LinkedIn and Twitter.
5. Please share reflections on how your efforts are helping to engage participants and/or build content, particularly for underrepresented groups.
- In this phase of the project, the participants we have actively engaged are Wikimedians who can share their knowledge and advice. Everyone we have engaged with in person (i.e. not Wikimedians on online forums) have been really helpful and have offered to continue to provide help. This includes the three members of the volunteer advisory group and one person who has provided direct help and advice on Wikidata.
APO’s blog and social media engagement has contributed to building awareness of the value of Wikimedia and the free knowledge ecosystem.
We have committed to, in the second half of the project, more targeted efforts to engage with new and existing participants directly to build content, such as direct emails and invitations.
6. In your application, you outlined your learning priorities. What have you learned so far about these areas during this period?
- One of our learning priorities was to determine if material published by organisations be an important component of Wikimedia and the free knowledge ecosystem. President of Wikimedia Australia, Amanda Lawrence, has been an advocate for this and we collaborated on our presentations to Worlds of Wikimedia and the ESEAP conference. There has been a positive response to this, however, it is not clear how this could be progressed. It appears it would require overhauling (and ideally streamlining) the guidance on reliable sources.
The remaining learning priorities were centred on understanding the impact of using policy reports to create and update Wikipedia content. As we have not yet reached the stage of creating and updating Wikipedia content, there is little we can say about this.
It is worth noting that, to date, we have received a very positive response from the local Wikimedia community to our efforts to build links between Wikimedia and APO.
7. What are the next steps and opportunities you’ll be focusing on for the second half of your work?
- For the second half of the project the focus will be on Step D: Building awareness and use of policy reports in Wikipedia.
Adaptations include: ᐧ Instead of a Wikimedia facilitator, we will be seeking support from the volunteer advisory group in holding the edit-a-thon. ᐧ Instead of a First Nations lived experience expert we will seek First Nations input in other ways such as recruiting students to the project. ᐧ We have revised the number of publishing organisations from 100 to 60.
Part 2: Metrics edit
8a. Open and additional metrics data.
| Open Metrics | Description | Target | Results | Comments | Methodology |
|---|---|---|---|---|---|
| N/A | N/A | N/A | N/A | I cannot see the metric description. | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| Additional Metrics | Description | Target | Results | Comments | Methodology |
|---|---|---|---|---|---|
| Number of editors that continue to participate/retained after activities | N/A | N/A | N/A | N/A | |
| Number of organizers that continue to participate/retained after activities | N/A | N/A | N/A | N/A | N/A |
| Number of strategic partnerships that contribute to longer term growth, diversity and sustainability | N/A | N/A | N/A | N/A | N/A |
| Feedback from participants on effective strategies for attracting and retaining contributors | N/A | N/A | N/A | N/A | N/A |
| Diversity of participants brought in by grantees | N/A | N/A | N/A | N/A | N/A |
| Number of people reached through social media publications | N/A | N/A | 1177 | 101 views of the blog post
711 views of all Twitter posts about Wikimedia (not including retweets) 365 views of LinkedIn posts about Wikimedia |
N/A |
| Number of activities developed | N/A | N/A | N/A | N/A | N/A |
| Number of volunteer hours | N/A | N/A | N/A | N/A | N/A |
8b. Additional core metrics data.
| Core metrics | Description | Target | Results | Comments | Methodology |
|---|---|---|---|---|---|
| Number of participants | Stage B: We are hoping to engage approximately 20 returning participants through the Reliable Sources Noticeboard and the volunteer advisory group.
Stage C: We are hoping to engage approximately 10 returning participants who have expertise in Wikidata including the volunteer advisory group. Stage D: We are hoping to engage approximately 10 new and returning participants in the edit-a-thon. In addition to the Wikimedians in residence (2), the Facilitator (1), and the lived experience expert (1). |
44 | 20 | New participants:
2 Wikimedians in residence 1 project Lead 1 Systems developer Returning participants: 3 volunteer advisory group members 3 other Wikimedia Australia members 1 Wikidata advisor 9 editors on Wikimedia forums. |
|
| Number of editors | We are proposing to hold an edit-a-thon for which we expect to attract 5 returning editors and 6 Newly Registered Users (new editors) in total. This includes the lived experience expert who may or may not already be a registered user.
There will be an additional three editors who attend the edit-a-thon: the Wikimedia Facilitator (existing users) and the two APO Editors who will be trained as Wikimedians in Residence (Newly Registered Users). In total this is 6 returning editors and 8 new editors. |
14 | 12 | Newly registered users:
2 Wikimedians in residence 1 project Lead Returning editors: 9 editors on Wikimedia forums. |
|
| Number of organizers | The organisers include the Project Lead, Wikimedia Facilitator and the two Wikimedians in Residence.
It is expected that there will be 4 members of the advisory group. The Wikimedia Facilitator will train the lived experience expert so that they can recruit new editors from their respective communities to participate in the edit-a-thon. |
9 | 7 | 2 Wikimedia Australia communications and support staff
1 Swinburne admiinistrator 1 Project lead 3 Advisory Group members |
| Wikimedia Project | Description | Target | Results | Comments | Methodology |
|---|---|---|---|---|---|
| Wikidata | It’s expected that approximately 100 organisations and 1,000 reports will be added to Wikidata. | 1100 | N/A | N/A | N/A |
| Wikipedia | Approximately 6 Wikipedia pages will be created and 12 Wikipedia pages will be improved in the edit-a-thon. | 18 | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
| N/A | N/A | N/A | N/A | N/A | N/A |
9. Are you having any difficulties collecting data to measure your results?
- No.
10. Are you collaborating and sharing learning with Wikimedia affiliates or community members?
- Yes
10a. Please describe how you have already shared them and if you would like to do more sharing, and if so how?
- The project lead has presented the project to a community meeting of Wikimedia Australia, as well as co-present with Wikimedia Australia President at the Worlds of Wikimedia Conference and the ESEAP Conference in Sydney in November.
We’d like to continue sharing the project learning with any forums that are held in 2023.
11. Documentation of your work process, story, and impact.
- Below there is a section to upload files, videos, sound files, images (photos and infographics, e.g. communications materials, blog posts, compelling quotes, social media posts, etc.). This can be anything that would be useful to understand and show your learning and results to date (e.g., training material, dashboards, presentations, communications material, training material, etc).
- Below is an additional field to type in link URLs.
- The missing link in the free knowledge ecosystem: https://blog.apo.org.au/2022/10/25/free-knowledge-ecosystem/
WikiProject Policy Reports: https://www.wikidata.org/wiki/Wikidata:WikiProject_Policy_Reports
Part 3: Financial reporting and compliance edit
12. Please state the total amount spent in your local currency.
- 20802
13. Local currency type
- AUD
14. Please report the funds received and spending in the currency of your fund.
- Upload Documents, Templates, and Files.
- Provide links to your financial reporting documents.
15. Based on your implementation and learning to date, do you have any plans to make changes to the budget spending?
- Yes
15a. Please provide an explanation on how you hope to adjust this.
- No major changes. But the spending on personnel has altered - due to not being to recruit a Wikimedia Facilitator and Lived experience expert. This budget allocation will be used for the alternative strategies we put in place. Any underspending in the first three steps of the project will be allocated to the final step so that we can put more effort into building awareness and recruiting new participants to the project.
16. We’d love to hear any thoughts you have on how the experience of being a grantee has been so far.
- It's been a good experience for the team so far. The alignment between Wikimedia and APO has become even more clear and obvious - so we are really grateful for the opportunity. We are keen to spend any excess resources on promoting the project to create more public policy content in Wikipedia.
The challenging aspect has been finding information about Wikimedia to do our work - and the dispersed range of information sources on a similar topic.