Grants talk:APG/Proposals/2017-2018 round 1/Amical Wikimedia/Staff proposal assessment

We want to answer back the staff proposal assessment. This text has been agreed by the community. We provide the original Catalan version, which addresses the main critics and concerns of the Grant Page--Barcelona (talk) 10:01, 16 November 2017 (UTC)

Amical Statement (Catalan)

Acabem de rebre el primer comentari sobre la nostra proposta d’APG i voldríem contestar perquè creiem que no és just ni en el contingut (afirmacions no ajustades als fets ni a les passades interaccions amb l’avaluadora) ni en la forma (per exemple la frase «Making fun of us all» al resum creiem que no toca ni per respecte ni per la formalitat del procés que després exigeixen).

Agraïm el temps dedicat a avaluar la nostra proposta i el reconeixement als nostres punts forts. A continuació adrecem algunes de les crítiques que inclou la pàgina i que no considerem ajustades:

Amical som una organització única, com constaten, ja vam néixer així, posant en dubte el model de capítols estatals. Periòdicament hem alertat del malbaratament de recursos econòmics que detectem en el moviment i, si bé entenem que no tots els contextos tenen les mateixes característiques, pensem que part d’aquest excés de dependència dels diners i de staff és justament per no aplicar una estratègia consistent, que se centri en bastir comunitats i projectes externs de qualitat i a mig termini. Podem equivocar-nos, però sí tenim estratègia i és evident que ens funciona (les mètriques que tant agraden com nombre de cursos educatius podrien ser un exemple o, dins els nostres paràmetres, les converses amb professors que fan possible que repeteixin any rere any). Ens sembla que no s’accepta la diversitat de models ni agrada la crítica, malgrat es dediquin conferències monogràfiques al respecte. Volen que fem exactament el que fa la resta, sense tenir en compte el context. Tanmateix, si cometem coses que no veiem clares del moviment, se’ns respon que no prenem en consideració els contextos diferencials. Una doble vara de mesurar?

Falta d’avaluació

Se’ns acusa repetidament de no avaluar els nostres programes o no facilitar mètriques. Ens hem queixat sovint que només ens demanen mètriques quantitatives que no reflecteixen l’essència dels nostres programes i no aporten informació útil de millora. Tot i així, les omplim cada any: https://meta.wikimedia.org/wiki/Grants:APG/Proposals/2015-2016_round1/Amical_Wikimedia/Impact_report_form/Global_Metrics_program_details També es poden deduir dels reports mensuals on indiquem el nombre i dates dels actes offline (i que ningú contesta mai). Hem argumentat a diversos llocs perquè aquestes mètriques no ens encaixen, la darrera vegada en l’auditoria presencial del juny. Entenem que les xifres són importants per explicar que quelcom existeix però no pot ser l’única mesura.

Hem explicat que duem a terme avaluacions periòdiques més qualitatives i que ens interessa molt comptar amb partners estables en els projectes, que puguin dur a terme actes pel seu compte i que canviïn de mica en mica la mentalitat cap als projectes Wikimedia. Dins la visita d’auditoria vam incloure una reunió en profunditat amb un d’aquests partners per vetificar el que comentem sobre incidència dins els sectors amb els quals treballem (reunió per la qual vam rebre elogis).

Ens importa més que els membres de la comunitat quedin satisfets i amb ganes de seguir, que siguin autònoms i incorporin els principis wiki, que no pas el nombre d’articles que surtin d’un acte (que de totes maneres afegim i són força alts). Preferim invertir temps en articular un discurs coherent sobre per què fem les coses que tenir personal que compti el nombre d’octets, que no incloem perquè creiem que desdibuixa panorama. Com comptem les contribucions que fan tots els nostres socis? Comptem les esborrades? Posem més palla a la redacció per augmentar quantitat? No són mètriques rellevants i afavoreixen editcountitits: https://en.wikipedia.org/wiki/Wikipedia:Editcountitis De vegades les fites numèriques van bé per animar a la participació (a veure si doblem la llargada d’aquests articles, a veure si completem aquesta llista o arribem als X editors), com a reptes, però no poden ser l’essència de l’avaluació. Preferim una avaluació qualitativa i que consideri com a centre la gent, el nostre recurs més preuat, la comunitat de voluntaris que fa possible que es creïn tots els articles. Un comentari elogiós d’un nou partner, un nou projecte que comença sense la nostra intervenció, un editor que torna després de missatges d’ànim són indicadors més precisos de la bona salut a mig termini del projecte que tots els documents que ens demanen. Però per això no ens pregunta ningú mai. Hem inclòs algun apunt sobre la qüestió a https://www.wikimedia.cat/2017/03/28/how-do-our-partners-see-us/

De fet, quan demanem justament avaluació, no se’ns dona. Si tot el seguiment que pot fer la WMF és una lectura dos cops l’any d’un informe basat en quadres de xifres, comprenem que no s’entengui el que fem i com ho fem. Voldríem que el FDC fos un recurs que es fixés en el que anem enviant cada mes, que establís un diàleg real amb les comunitats (se suposa que les visites sobre el terreny tenen aquesta finalitat però la informació després es perd o s’adequa al que apareix als informes públics). Voldríem que ens ajudessin a millorar amb comentaris sobre programes o debats, però solament demanen «quant» i res més. Ens sembla injust que se’ns acusi de manca d’avaluació quan és una de les nostres queixes, que volem més avaluació real. Es confon quantificació amb avaluació i, tot i fer els quadres que ens demanen, ens diuen que boicotegem el procés global. Incomprensible.

Falta d’estratègia

Hem explicat repetidament que totes les nostres accions estan emmarcades dins el pla estratègic 2014 – 2018 i que dediquem molts recursos a comentar les línies generals per guanyar en coherència global. Considerem que és un dels nostres punts forts, però no sembla clar des de fora veient les acusacions al respecte (i en un to gens constructiu). Voldríem pistes de com explicar-nos millor, consells útils per fer arribar el discurs que compartim entre els socis.

Manca involucració moviment

La tercera gran crítica és que no ens impliquem amb el moviment o no volem construir. Quan critiquem o qüestionem, no se’ns escolta. Quan anem a vendre el nostre model, se’ns diu que és únic i no escalable. Quan preguntem com millorar, hi ha un silenci total. Com podem participar més?

Hem anat a diversos fòrums internacionals, per exemple aquest any el Celtic Knot, la WM CON, a debatre, a aprendre dels altres i a aportar les nostres idees. Hem ajudat altres viquipèdies a implementar els nostres projectes de més èxit (com biliowikis fora de línia o les infotaules automatitzades en línia). Intentem col·laborar en concursos d’altres comunitats (per això ens diuen Catalan Army) i tenim apartats internacionals d’alguns projectes propis. Hem coorganitzat projectes amb WM-ES i hem participat més activament a meta, com se’ns va demanar. Llegim i comentem la documentació que ens arriba. Mirem d’establir diàleg amb nous agents i el nostre alliberat i cara visible és conegut arreu del moviment. Per què se’ns diu que no volem involucrar-nos?

El que no farem és anar a reunions que no tinguin cap sentit per a nosaltres i que suposin una despesa injustificable, com passa darrerament a fòrums, projectes i conferències buidades de contingut (tema que portem denunciant des de fa temps https://www.wikimedia.cat/2015/09/02/la-fundacio-wikimedia-vol-coneixer-la-comunitat/ i malgrat tot continuem fent-ne seguiment). Abans teníem presència en grups però racionalitzem esforços en veure que no hi ha un intercanvi real. Veiem molta reunió i trobada sense resultats, justament per part d’equips que després demanen el contrari. Constatem una tendència cap a una suposada «professionalització» (poca presència comunitats de base i molt staff) i repeticions sense justificar. Hem seguit els learning patterns i els canvis repetits en els departaments d’avaluació i programes (que depenen massa del rumb de contractacions que van i venen).

Normalment fem una anàlisi detallada dels comentaris a tots els APG (abans FDC anuals), perquè respectem el procés, fins i tot van escriure algun apunt al bloc sobre els aprenentatges que en podíem treure (https://www.wikimedia.cat/2015/04/08/aprenent-dels-altres-chapters-llicons-de-les-recomanacions-de-la-wikimedia-foundation-als-plans-anuals ). Però hem vist que, malauradament, cada cop estan més buits també pels punts esmentats anteriorment.

Part del nostre programa de discurs i comunicació té com a pota la relació amb el moviment (i més enllà, a partir de participació en taules rodones o escrivint articles). Per això no entenem que es digui que no volem fer aportacions. Posteriorment s’afirma que l’actitud de la comunitat, en termes generals, és «concerning» per manca de voluntat de diàleg. No hi ha cap exemple de cas on s’hagi donat una actitud negativa per poder-la corregir o una mica d’argumentació al respecte. El comentari és ofensiu cap a a nosaltres i no creiem que respongui a la realitat per tot el que estem comentant.

Pertinença al procés de recaptació de fons

Una de les crítiques que se’ns fa és que no seguim les recomanacions del FDC. En l’auditoria se’ns va aconsellar passar a un altre model de recollida de fons, que vam rebutjar però que ara sembla que no ha agradat. Volem explicar-nos al respecte. Primerament se’ns deia que el criteri era econòmic, demanem menys de 100.000 euros i llavors no compensa la paperassa. Vam argumentar que la divisió entre plans complexos o simples no pot dependre solament de la quantitat demanada i que justament l’austeritat és un dels nostres principis bàsics. No oblidem que la majoria d’aportacions venen de lectors que volen que fem articles i paguem servidors. Vam explicar que comptar amb un pla anual com l’APG (que finança terminis i projectes i no actes individuals) ens dona més llibertat per adaptar-nos a les peticions de la comunitat, un dels punts bàsics de la nostra estratègia i que ens allibera de temps de justificació comptable que no aporta valor afegit als projectes, especialment tenint en compte que de moment la tasca de tresoreria la porta a terme un voluntari. No volem donar-li una feina desagraïda no imprescindible. Vam reclamar que precisament volíem un model com APG on s’avalua l’estratègia global i els projectes qualitatius i no una sèrie d’actes o viquimaratons. A la reunió presencial tot semblava entès però ara ens trobem que novament qüestionen la nostra «pertinença al procés», així sense gaires aclariments més.

Repetidament se’ns ha qüestionat per demanar pocs diners i en canvi aquest any quan fem un lleuger augment (explicat per la diferència d’ús en reserves pròpies i per reptes nous com actes lligats a la Hackathon), també se’ns critica, insinuant que no fem els projectes tan sostenibles com diem. Posteriorment es cau novament en una contradicció: es reconeix que tenim voluntaris, staff i una junta competent per dur a terme les tasques que li pertoquen però s’afirma que difícilment podrem complir els requisits del FDC, pitjor, «del nivell FDC». Quins requisits? Es penalitza tenir una infraestructura mínima i basada en el voluntariat en el món wiki?

Creiem que el FDC ha de ser una eina per distribuir els recursos i ajudar que les comunitats s’organitzin. Pensem que hem demostrat amb la feina feta que usem els diners de manera adequada, sempre alineada amb els principis de la WMF i del moviment. Hem respost sempre a les preguntes plantejades (cf. Pàgines de discussió) i enviem cada mes un report del que fem. Volem més diàleg com hem indicat més amunt. On no estem col·laborant amb el procés d’APG?

Conclusions

En definitiva, ens han ofès alguns comentaris i pensem que no reflecteixen ni la realitat de la nostra trajectòria ni el que hem explicat en persona cada cop que hem tingut ocasió. No considerem que responguin a la feina avaluadora de programes que se suposa del FDC i demanem per tant un altre tipus de resposta, que tingui en compte els nostres requeriments i necessitats, ja que se suposa que les estructures de la WMF estan per ajudar les comunitats i fer-les créixer, una voluntat que no apreciem en aquest quadre resum que hem rebut.

Amical Statement (English translation)

We have just received the first assessment about our APG proposal and we wanted to answer it because we believe it's not fair. Neither the content (comments not adjusted to facts and about the past interactions with the evaluator) and the tone used (for example using the phrase "Making fun of us all" in the summary, we think it's just not adequate for pure reasons of respect and formality that this process requires)

We thank the time taken in evaluating our proposal and the awareness of our strong points. In the next paragraphs we want to address some of the criticism of the text that we consider not adjusted to reality:

Amical is an unique organisation; we borned being special and questioning the model of state Chapters. We have periodically warned about the amount of money that is being wasted in the Wikimedia movement, and even though we accept that all context do not have the same characteristics, we strongly believe that a huge part of this over-dependence on money and staff is precisely a consequence of not applying a firm strategy focused in building communities and quality medium-term external projects. We might be wrong, but we do have strategy and it is obvious that it is working (the metrics everyone like, such as the amount of ongoing educational courses could be an example, or included in our parameters, also talks with professors that make these projects being sustainable year after year). We consider that diversity of models is not accepted as well as critics, although monographic conferences are addressed in this regard. They want us to make exactly the same as the rest without considering the context. Furthermore, if we raise our voice for things we do not seem clear inside the moviment, we are replied that we do not take the differential contexts into consideration. Are they applying double standards?

Lack of evaluation

We are repeatedly criticized for not evaluating our programs or not facilitating metrics. We have expressed several times that we are only asked for quantitative metrics that don't reflect our program's essence moreover they are not providing useful information for improvement. Nonetheless we fulfill in providing all these metrics each year: ttps://meta.wikimedia.org/wiki/Grants:APG/Proposals/2015-2016_round1/Amical_Wikimedia/Impact_report_form/Global_Metrics_program_details Also it can be deduced in the monthly reports in were we give the numbers and the dates of off-line events (and nobody has ever said anything). We have argumented in several places why these metrics are not fit for us, last time in the personal audit in June. We understand that numbers are important for explaining that something exists but this cannot be the only measure.

We have explained that we do much more qualitative and periodical evaluations and that we are interested in long-time engaged partners who can run their own events and help in changing the mentality towards Wikimedia projects. During the personal audit we included a meaningful meeting with one of these partners for explaining the presence of our discourse within the sectors we work with (and we received congratulations for this meeting).

For us it's more important that community members feel fulfilled thus making them interested in repeating (many of them by their own, running their own wikiprojects or even including the wiki principle to their work) than the output number of articles from an event (moreover being this number usually high). We prefer spending our time in constructing a much more coherent discourse about why we do what we do than counting number of bytes added, we don't include this figure because we believe is not correspondent to reality. How do we count all the different contributions by our members? Can we count the deleted ones? Shall we include more explanations to grow the length of quantitative measures? All theses metrics favors "editcountitis" (https://en.wikipedia.org/wiki/Wikipedia:Editcountitis). Sometimes numeric milestones are good to engage community in participating more (for example, "let's see if we can enlarge these articles", "let's finish this list", etc.), they are good as challenges but cannot be the essence of the evaluation. We prefer a qualitative evaluation centered in the people, our most precious resource, the community of voluntaries that makes possible the creation of all the articles. An encourageous comment made by a new partner of us, a new project that starts without our intervention, an editor coming back after supportive messages left in his talk page, are the most indicative comments about the health in the mid-terms of a project rather than the required documents. But again, we are never asked. He have added some points shared here https://www.wikimedia.cat/2017/03/28/how-do-our-partners-see-us/

In fact, when we asked for evaluation, is not given. If all the follow-up that WMF can do is reading two reports based on a numerical spreadsheet, we understand that we are not understood correctly. We want that FDC could be a resource that takes a look each month to the reports, that establishes a real dialog with the communities (it's supposed that on-site visits should fulfill this but later the information is lost or doesn't correspond to the public reports). We want them to help us with comments about programs, but we are only asked "how many?" and that's it. We feel injustice when we are criticized about a lack of evaluation when this is precisely one of our demands, real evaluation. Quantification and evaluation are misunderstood, moreover, after we fulfill the required spreadsheets we are told that we are blocking the global process. Incomprehensibly.

Lack of strategy

We have explained continuously that all our actions are included in the Strategic Plan 2014 – 2018 and we spend lots of resources to comment them broadly to win in global coherence. We believe that it is our strong point, but it doesn't seem so from outside if we look at the accusations we are receveing (and even less with this scornful tone). We would appreciate clues for explaining us better, useful advices to make the discourse reach all our affiliates.

Lack of movement engagement

The third greater critique states that we do not get involved with the movement, or that we are not constructive. When we criticise something or ask about something nobody is listening. When we try to sell our model, we are told that it is not scalable. When we ask about how can we improve, the answer is total silence. How can we get to be more involved?

We have been in different international forums, i.e. this year we went to the Celtic Knot and WM Con, to debate, learn from the others and share our ideas. We have helped other Wikipedias to deploy our most successful projects (as offline Bibliowikis or the online automated infotables). We do our very best to collaborate in contests from other communities (this is why some call us the «Catalan Army») and we have some international sections in some of our own projects. We have co-organized projects with WM-ES and we have been more active in meta, as we were asked to. We read and comment the documentation we receive. We try to establish a dialog with new agents and our public face is known around the movement. How is it possible to say that we do not want to get involved?

What we will not do is go to meetings without that do not have sense for us, and that imply unjustifiable expenses, as happens lately in forums, projects and conferences without any content (what we have been denouncing from long time ago (https://www.wikimedia.cat/2015/09/02/la-fundacio-wikimedia-vol-coneixer-la-comunitat/ although we keep tracking them). Previously we participated in groups, but we now rationalize our efforts after we observed that there is no real exchange in them. We see there are plenty of meetings without any results, held by teams that afterwards asks for exactly the opposite. We have confirmed that there is a growing tendency towards «professionalization» (few community base presence and plenty of staff) and repetitions without any reason. We have followed the learning patterns and the changes stated repeatedly by the evaluation and programs departments (which rely excessively in the way staff members come and go).

We usually make a detailed analysis of the comments in every APG (previously annual FDC), because we respect the whole process, even so somebody wrote a blog post about what we could learn from it (https://www.wikimedia.cat/2015/04/08/aprenent-dels-altres-chapters-llicons-de-les-recomanacions-de-la-wikimedia-foundation-als-plans-anuals ). But we have seen that, unfortunately, as time has gone by they are more and more empty of content, as we mentioned previously.

Part of our speech and communication program relates to the relation with the movement (and, furthermore being part of round tables or writing articles). This is why we do not understand how anybody can say we do not want to make any contribution. Later on it is stated that the community's attitude, in general, is «concerning» for its lack of dialog. There is no example at all of any situation where there has been a negative attitude for us to be able to correct it, or any reasoning about it. The comment is offensive by itself towards us, and we do not believe that has any relation at all with reality, as we are trying to explain.

Engagement with the fundraising process

One of the points raised is that we don't follow FDC recommendations. In the last audit we were advised to move towards another model of fundraising, which we opposed, it seems this has not been liked. We would like to give some more explanations. Firstly we were told it was just an economical criteria, we ask for less than $100,000 therefore bureaucratic work is not worth it. We argumented that division between complex and simple plans cannot be decided just by the quantity of funds requested moreover austerity is one of our basic principles. It's good to remember that the majority of funds come from readers that want us to write articles and pay for servers. We explained that for us having an annual plan like APG (that finances projects and not individual events in terms) gives us more freedom to adapt towards the petitions of the community, one of our basic points of our strategy and that frees us a considerable amount of time of fund justification that doesn't add any value to the projects themselves, specially having into account that for the moment the treasury tasks are done be a voluntary. We don't want to ask for an unnecessary and ungrateful work. Therefore we precisely asked for a model like APG were our global strategy and the qualitative projects are evaluated rather than an array of events or editathons. In the in-person meeting everything seemed understood but now we face that our "pertinence in the process" is questioned, without much clarification.

We have been repeatedly questioned for asking for little money, so this year when we made a slight increase (explained by the difference in use in own reserves and new challenges as acts linked to the Hackathon), we are also criticized, insinuating we do not make projects anymore as sustainable as we used to say. Subsequently, it falls into a contradiction: it is acknowledged that we have volunteers, staff and a competent board to carry out the tasks that pertain to it but affirms that we can hardly meet the requirements of the FDC, worse, «from the FDC level». What requirements? Is it penalized to have a minimal infrastructure based on volunteering in the wiki world?

We believe that the FDC should be a tool to distribute resources and help communities to self-organize. We believe that we have demonstrated with the work done that we use money properly, always aligned with the principles of the WMF and the movement. We have always answered the questions raised (see Talk Pages) and we send a report every month of what we do. We want more dialogue as we have indicated above. Where is that we are not collaborating with the APG process?

Conclusions

In short, some off the comments have offended us, so we think that they do not reflect the reality of our trajectory or what we have explained in person every time we have had occasion. We do not consider that they are responding to the program evaluation work that is assumed by the FDC and therefore we ask for another type of response, which takes into account our requirements and needs, since it is assumed that WMF structures are to help communities and make them grow, a will that we do not appreciate in this summary picture that we have received.

réponse de WMF

en français Merci à tous chez Amical de vos réponses à notre staff assessment. Elles ont été prises en compte lors des délibérations du FDC. J'ai tout d'abord supprimé la note "they make fun of us all" qui, je le reconnais, ne fait pas avancer le débat et demande à Amical de bien vouloir nous pardonner son utilisation. Nous n'allons pas répondre à toutes les remarques sur cette page, mais j'ai demandé, comme chaque année, à rencontrer mes contacts chez Amical pour discuter de la décision du FDC dans les jours qui viennent. Deux informations qui peuvent informer la discussion :

• Le fait que la mesure des "bytes" n'est plus obligatoire depuis deux ans et que Amical soi-même a l'année dernière défini des "grantee defined metrics", sur lesquelles il n'a été donné aucun résultat par le biais du progress report. Fait qui a été mentionné lors de la réunion destinée à discuter du progress report, pendant laquelle la demande a été faite à Amical de donner ces informations.
• Lien vers le rapport de notre visite de site à Barcelone en juin 2017, qui donne beaucoup d'informations sur les discussions que nous avons déjà eues ensemble.

A bientôt, Delphine (WMF) (talk) 17:12, 23 November 2017 (UTC)

en anglais:

• Thanks to all at Amical for your answers to our staff assessment. They have been read and taken into account during the FDC deliberations. I have taken out the note "making fun of us all", which, I agree,

does not help advancing the debate and ask here for forgiveness in using it. We will not answer all remarks on this page, but I have reached out, like I do every year, to our Amical contact to discuss the FDC decision in the next few days. Here are two pieces of information that may inform the discussion:

• The fact that bytes measurement has not been mandatory for the past two years, and that Amical has itself, last year, defined "grantee defined metrics" that they did not report against in the progress report. This fact has been mentionned during the progress report review, during which Amical was asked to give those informations.
• A link to the site visit report in June 2017 in Barcelona, which gives a lot of informations on the discussions we have already had together.

Best, Delphine (WMF) (talk) 17:12, 23 November 2017 (UTC)