Grants talk:Programs/Wikimedia Alliances Fund/Wikicredibility Initiative: Wikimedians Strengthening Knowledge and News Credibility on the Internet

Preliminary staff feedback

@SuperHamster and Sj: Hello Ahmed, Jennifer, Kevin, and SJ, and thanks for your proposal to extend the Wikicredibility Initiative. Before the Regional Committee offers its own feedback and decisions regarding the proposal, I wanted to share some initial staff feedback that we collect as part of our due diligence process. Having reviewed this staff feedback already, the committee would like you to review, respond, and make potential revisions to your proposal here on Meta accordingly to support their final review later this week. Below, I will provide a summary of staff feedback:

• In terms of its organizational values, experience, and general community, Hack/Hackers has strong mission alignment with the Foundation and the broader Wikimedia movement, and are a good fit for the Wikimedia Alliances Fund.
• The challenges this proposal seeks to address, such as enabling more discussion over reliable source guidelines for underrepresented communities, are very relevant to several movement strategy principles.
• There are a number of significant concerns related to the proposed activities:

• In terms of context, it would be helpful to understand more about the WikiCredibility initiative and its relation to this proposal. To do this, it will be helpful to summarize what the WikiCredibility Initiative is and why it has been successful to better frame the rationale for this proposal.
• The details of how this work will be implemented and what outcomes are expected are generally unclear. For example, the proposal notes that workshops will be carried out. How will these activities be delivered, and what is their final objective? What does Hack/Hackers hope to achieve through these workshops and how will they measure this? As a second example, what will result from the discussion of reliable sources guidelines and new tools and prototypes developed thanks to WikiCred microgrants?
• In terms of evaluation, there are some points of disconnect between your learning questions and specific measures. For example, ...is there a standardized format for engaging different diverse communities on the topic of credibility and reliable information that we can pass on to organizers? is an interesting question, but there is no proposed evaluation plan that clearly responds to the question. Similarly, Can we engage a larger community of technologists into the conversation around credible information, the offerings of wikimedia projects for the overall information ecosystem? also maps clearly onto the general purpose of this proposal. While the evaluation plan includes an intention to collect new organizations engaged in the field of media literacy and open-knowledge ecosystem that will choose to interact with this project..., a more in-depth analysis of these organizations would be more robust, such as understanding what needs they have to engage in the coalition, or what would motivate them to participate.
• Some clarity is needed around the quantitative metrics, such as whether the metrics represent absolute totals or totals per activity (e.g. 20 per workshop). In general, absolute totals across all activities/the length of the grant period are preferred where possible. Some more clarification is also needed for what "Wikipedia" means as a metric in relation to the 100 metric associated with it.
• The current budget is not sufficiently detailed, and precludes meaningful review. A more specific breakdown of the budget is requested, including specific staff costs and descriptions of those roles, and what expenses G&A is supporting separate from staffing. There is also some inconsistency in the budget summary in that G&A is described as 25% of the total budget, but only $15,000 is requested from the $100,000 total.
• Related to staffing and G&A, expenses that include staffing compensation should be accompanied by a clear rationale for why each role is necessary for achieving your proposed goals or supporting critical organizational operations.

As I noted earlier, you are welcome to respond here and make changes here on Meta directly related to any of the above feedback. The committee will begin its final review on 23 February (Wednesday) next week to support a final decision on 28 February (Monday). Please let me know if you have any questions or needs for clarity. I'm also happy to meet early next week to discuss any of this initial staff feedback. I JethroBT (WMF) (talk) 16:57, 19 February 2022 (UTC)

Hi Jethro, thanks! I reckon a discussion next week would be welcome. We will discuss and try to clarify (I take it the description on the page can be revised?) Warmly, –SJ talk  22:03, 19 February 2022 (UTC)

@Sj: Sounds good. I'll follow up by e-mail for us to schedule meeting next week. Thanks! I JethroBT (WMF) (talk) 23:49, 19 February 2022 (UTC)

@Sj: (And sorry, yes, the description can be revised.) I JethroBT (WMF) (talk) 22:51, 20 February 2022 (UTC)

Added clarification to metrics + budget sections based on chat today; Ahmed & @SuperHamster: feel free to correct + improve :) –SJ talk  00:10, 25 February 2022 (UTC)

Added some sample projects and links to the overview section, along with other minor tweaks throughout the grant. Thanks, ~SuperHamster Talk Contribs 19:18, 26 February 2022 (UTC)

We made an attempt to update the grant proposal as well

Request for feedback from Regional Committee

@SuperHamster and Sj: Hello Ahmed, Jennifer, Kevin, and SJ. The Regional Committee is requesting feedback on one final set of questions to support their review and decisionmaking:

• Are there any specific communities or audiences you will encourage or invite to apply for funding from this phrase of the WikiCred program?

• For this second phase of WikiCred, please provide an overview of how your proposal review and decisionmaking processes will operate. Will there be any changes from the general procedures used for the initial phase of WikiCred? Please specify by either copy pasting or answering from fresh: who is the target audience (applicants) for the subgrants, who makes the decisions on which subgrants to award and what criteria will they use (are there any budget caps/guidance, metrics)?

• For this second phase of WikiCred, would the program support both activity-specific microgrants and the development of new or existing tools, or will the program support a more specific set of activities?

• Clarity is needed around the main objective of the proposal. The main concern is that the objective is focused more on activities rather than a statement of purpose: The stated objective is to ...continue this work with a new series of grants, webinars, and workshops to support new editors, readers, technologists and other ecosystem members who rely on Wikipedia for verified information, sourcing, understanding and creating better processes and frameworks for content creation and information provenance with the help of Wikimedians.
  • Part of this statement -- the intention to distribute grants to a certain number of projects or initiatives, prepare webinars and workshops -- represents activities or tasks, not an objective that conveys why you want to do this work, or why the work is important.
  • The latter part of the statement -- to create better processes and frameworks for content creation and information provenance – provides more context related to proposed activities, but is still somewhat unclear. Why are better processes needed for content creation? Why are better frameworks needed for information provenance? One way of framing an objective for this proposal could be to consider what broad problems or barriers that exist with respect to content creation and information provenance in the communities you work with, and why sub-grants will be an effective way to address them.

In addition, the committee has also noted a number of strengths in the proposal and your organization. First, it is clear the project team has good experience with grantmaking in the movement through the WikiCred program, and has been able to support projects and initiatives that have broad benefit for the movement. The examples provided are helpful in understanding areas where WikICred has been successful. Your team has demonstrated skill and capacity to develop effective partnerships with organizations that support WIkimedia movement values. Finally, the committee appreciates the effort to work on a challenging need in the movement around the topic of reliable source guidelines that can harm the ability of some communities to represent and communicate their knowledge.

Given the depth of questions and feedback requested, the Regional Committee is requesting your responses to these questions and/or possible revisions to the proposal by 11 March 2022 (Friday). The committee will review and make a final decision on funding by 15 March 2022 (Tuesday). You are welcome to contact me (Chris) if you have any further questions or needs for clarity. On behalf of the Regional Committee, I JethroBT (WMF) (talk) 22:04, 1 March 2022 (UTC)

Hello, thanks for these questions + comments. Catching up ... the team has been discussing, will update presently. (1-2: yes to specific target audiences, particularly those excluded last round or working on bridging / translating / extending existing solutions that are already well-used to new audiences; 2: similar judging process + metrics [will repaste] w/ new judges, 10k cap for reusable tools + lower cap for activities; 3: mostly reusable outputs, incl expanding feature + audience of existing ones, and initiatives that build persistent communities of practice around credibility work (which could include a mix of social templates, implementations of them, and related tools; compare WikiGap & gapfinder); 4: we'll clarify the objective! encouraging / raising the visibility and status of {{sofixit}} approaches to rs+ cred helps communities prioritize and identify their own barriers, and find others working on solutions. [filing bugs that don't get addressed can be disempowering; seeing a stream of supported proposals in contrast can be empowering.] –SJ talk  23:43, 11 March 2022 (UTC)

Hello! We highly appreciate the feedback and the noted strengths. The team has met and come up with answers for the questions. Vast majority of this has been added to the grant proper, but I have also placed our collective answers here for easier referencing.

Are there any specific communities or audiences you will encourage or invite to apply for funding from this phrase of the WikiCred program?

The following communities and audiences will be encouraged to apply for funding during this phase of the WikiCred program:

• Both English and non-English audiences
• Educators, academics, and researchers in the misinformation space (and related fields)
• Technologists and software developers

For this second phase of WikiCred, please provide an overview of how your proposal review and decisionmaking processes will operate.

For this second phase of WikiCred microgrants, there will be no changes to general procedures from the initial phase of WikiCred. The proposal review and decision-making processes will operate as follows:

• Target audience: Existing and new target audiences include Wikimedia DC, Wikimedia Mexico, Wikimedians of the Levant, Wikimedian academics at the University of Sydney, Arizona State and Wikimedian librarians that WikiCred has partnered with in the past.
• Subgrant budget cap: $7,500
• Subgrant requirements: We provide direction for project scope and budgeting; applicants to provide target metrics and outcomes, and then will report against results.
• Review process: As with last time, we will have a review committee that consists of experienced Wikimedians and misinformation experts from Misinfocon and CredCo. The review committee regularly reviews applications, provides initial ratings, and then meets to come to consensus on decisions (which may include engaging with the applicant for more information or adjusting their grant plan).

For this second phase of WikiCred, would the program support both activity-specific microgrants and the development of new or existing tools, or will the program support a more specific set of activities?

We are open to any proposal that works towards our goals, but we will be focusing more on the development of new or existing tools, processes, and frameworks, as we have found these can have long-lasting, measurable impact for Wikimedia editors. Proposals for activities and grants will of course also be considered. We may also redirect grant requests as needed (for example, a proposal for a small activity may be redirected to the Wikimedia Foundation's rapid funds process if we believe it will be a better fit).

Clarity is needed around the main objective of the proposal...

Ongoing conflicts around the world show the need to curate and surface quality information; this focus on building resources to address disinformation has been a key strategic activity since our first MisinfoCon in Kyiv in 2019.

Our proposal is focused on addressing the challenge of credibility online in a scalable, sustainable way by continuing to build the capacity of Wikimedia communities. The new initiatives we are proposing will continue to support and incubate existing credibility tools and also connect them to new audiences, and thereby expand their impact.

The objective of this grant is to continue this work by with a new series of grants, webinars, and workshops to support new editors, readers, technologists and other ecosystem members who rely on Wikipedia for verified information, sourcing, understanding and creating better processes and frameworks for content creation and information provenance with the help of Wikimedians.

While Wikipedia has several frameworks and processes in place to ensure reliability, we hope to improve them and help fill in gaps. One problem is the difficulty editors can face in distinguishing reliable sources from unreliable ones, in a world where new websites are popping up every day. Another example is that existing processes are gameable to introduction misinformation; Wikipedia typically does a good job of keeping out uncited statements, but once someone adds a citation it becomes more difficult to identify misinformation and/or unreliable sources and to act accordingly. It is also possible for citations to be manipulated to appear more reliable than they are; for example, someone may cite self-published sources across Wikipedia, and if they stick that can increase their perceived value and builds a positive feedback loop. By studying, improving, and building new processes, we hope to make it easier for editors to tackle misinformation and improve sourcing. Looking back at the first round of grants, the Vaccine Safety Project established a list of reliable sources related to vaccines; RefB provided an automated way to add reliable sourcing to biomedical topics; and Glassbox provides a way to identify changes in existing sources. Continuing to fund and support projects like these are how we plan to continue improving reliability on Wikipedia.

Based on our success so far with the WikiCred initiative, we will also continue to encourage new WikiCred projects proposals that address gaps in the credibility space, as well as projects that will serve new communities. We are also proposing to continue to move towards comprehensive coverage of active credibility needs by launching and supporting community-maintained tools.

Finally, thanks to improving pandemic conditions, we intend to meet the demand for in-person events by planning and facilitating “WikiCredCons” where research, technology, activist and Wikimedia communities can come together to discuss, surface, and collaborate on new solutions and tools that address credibility issues.

On behalf of the WikiCred team, ~SuperHamster Talk Contribs 08:39, 12 March 2022 (UTC)

Alliances Fund Program Funding approved in the amount of 100,000 USD

@SuperHamster and Sj: Hello Ahmed, Jennifer, Kevin, and SJ, Congratulations! Your grant is approved in the amount of 100,000 USD with a grant term starting 22 March 2022 and ending 31 December 2023. For clarity, the start date was adjusted in order to support the requirements of our grant agreement process. If more time is needed to complete the proposed work, we are happy to extend the proposed end date during your grant period.

The United States and Canada Regional Committee appreciates your responses and proposal revisions addressing their requests and suggestions. After reviewing the final proposal, the committee offers the following comments and requests:

• The committee supports the continuation of grant processes used for the previous WikiCred program.
• Overall, the objectives of the proposal have only become more important in the past weeks with respect to international affairs and conflicts involving misinformation.
• There is greater clarity about the procedures and overall objectives for your work. Furthermore, it is now understood that there will be greater focus to use funding to develop technology relative to supporting event organizing. One suggestion would be to consider a more intentional split of funding between organizing grants and technology grants within the total $75K budget. These splits do not need to be exact amounts, and could be approximate proportions of the budget (e.g. about ⅔ toward technology, ⅓ towards event organizing).
• There are some further considerations around who may be eligible for funding in this second phase. For example, the committee does not recommend funding to Wikimedia affiliates already receiving grants from the Foundation, as this may circumvent or complicate the Foundation’s own funding processes. For similar reasons, the committee also requests that applicants should not be eligible for funding through the WikiCredibility Initiative if they are not eligible for funding from the Wikimedia Foundation according to its eligibility criteria.
• Future applications from Hack/Hackers should provide about this level of detail about your programs and overall strategy for the initial application to support the committee to provide more detailed and nuanced support for your programmatic work.
• Finally, to support greater transparency, the committee requests more information around the general process and procedures used for subgranting and providing information about who is on the grantmaking committee. This information can be provided during your midpoint and final reports for this grant.

We appreciate your engagement with us and look forward to learning more about the outcomes and new opportunities that Hack/Hackers will support in this second phase of the WikiCredibiltiy Initiative. On behalf of the committee, I JethroBT (WMF) (talk) 21:56, 16 March 2022 (UTC)