Empowering Italian GLAMs/Criteria for assessment and evaluation

Criteria to assess and evaluate the case studies selected within the project Empowering Italian GLAMs

Analysis strategy

Understanding the Access Policy (qualitative)

Survey Questionnaire/ Interview

This first part , through a qualitative analysis, will aim to investigate the accessibility policy currently adopted, but also opportunities and barriers to open access policy adoption. The case studies selected are different and were selected for their willingness to initiate an open access policy. This qualitative analysis will allow us to investigate the different ways and reasons behind the choice to approach an open access policy.

Policy formulation

•  What is the current policy of the museum regarding the access and use of contents( e.g images, texts, videos, conferences, etc) ? Is the collection posted on the website and easily available? Has this policy changed recently? If so, when?
• Are the licenses clearly indicated? Are the term and conditions coherent with the open access policy?
• Describe the museum’s approach to the use of images for:
  • scholarly research and publication purposes (scholarly publications up to a specified print run)
  • other educational or not-for-profit usage (college and university teaching, and other not-for-profit use)
  • commercial usage (any usage through which the user could potentially earn money or through which the image could be associated with a for-profit concern). If the museum has chosen to treat these three usages (or others I have not identified) with different approaches, please explain why
  • Internal management

Decision-Making Process

• What is the rationale for the museum’s current policy? What was the decision-making process? Was the decision influenced by external factors? Were there any obstacles?
• If there have been recent changes to the policy, what are they? Was there a precipitating event? Who was responsible, and who was involved in changing the museum’s approach?
• Were there initial concerns expressed over any changes that have been made? If so, how were they overcome? Have there been any reservations about having made this change?
• Were there changes in staffing, position descriptions, workflow, budget, or issues related to revenue generation?

Policy change considerations

We can investigate expectations or concerns regarding policy change for:

• museums that have implemented some open access policy
• museums that have not yet implemented OA policy but are open for change (most of case studies are in this second condition)

The case studies selected are different. This qualitative analysis could give us ways to think about open access policy in different ways and for different reasons.

Measuring the effects and outcomes of the Access Policy (quantitative & qualitative)

After qualitatively investigating the choice to adopt an open access policy, it is possible to measure the effects and outcomes through qualitative and quantitative method of analysis. This part needs combined quantitative and qualitative indicators. Quantitative measures included webometrics, log file analysis, and content analysis. These will be complemented some qualitative measures (e.g. user feedback). This phase will be guided primarily by three research questions:

How is the collection accessible online? What effects?

• Number of digitized images/contents available in open access out of the total collection (adoption rate). E.g. if content in open access are 10% of the digital content of the institutions or the 70% (which I would consider the maximum). maybe we can consider several categories: 0 (nothing); < 2% (something); 2%-10% (a collection), 10%-50% (several collections); 50%-70% (large part of the collections); >70%
• Number of platforms\services used and which one (add explicit examples of platforms and provide information about the kind of platforms they are proprietary, open and libre software, terms and conditions of the platforms)
• Type of digital asset (images, 3d prints, virtual tour, games,etc)
• Delivery Mechanism (standard web image, high- resolution images, etc)
• Number of views reached (from Wikipedia, or from site analytics/API)
• Number of users reached ? hyperlinks ? measure content and metadata quality ?
• User feedback
• Geographic reach
• Number of Open Educational Resources (OER)?

Are the images reused? Where? How?

• Use RIL (Revers Image Lookup) methods to investigate where images are used (via google images or TinEye)
• Usage statistics, as Number of Wikipedia pages, download, shares, uses in academic literature (add metrics on how to measure the knowledge produced?)
• Number of derivative projects or art works (Tracking and measuring creativity through social use, e.g. #vangoghinspires)
• Number of collaboration with other institutions
• User behaviour (Improve public outreach by learning more about audiences through social tagging, public comments, and the resulting social dialogue - Gather information about the interests of the audiences and enhance the documentation and interpretation of its collections using the knowledge, perspectives, and experiences of these audiences- Allen, 2009:26)

What economic costs and benefits?

• Transaction costs (eg. Staff cost for collection management saved, enforcement costs= legal management cost, opportunity cost of time required to provide the content, search costs)
• Potential revenues from the sale of images (no. images sold in past years x p)
• Visibility, website traffic, brand identity?
• Potential funding opportunities and partnerships: Number of sponsorships and funding received
• Economic value of images on wiki (how much money is saved by institutions and people by accessing open access content instead of paying or using Wikimedia)

Models

The metrics were inspired by the data analysed and reported from the following case-models:

• Smithsonian Open Access: https://www.si.edu/openaccess ; https://www.si.edu/dashboard
• Cleveland Museum of Art: https://www.clevelandart.org/open-access ; https://medium.com/cma-thinker/cma-open-access-celebrating-our-fourth-anniversary-955a97690c36 ; https://medium.com/cma-thinker/introducing-open-access-at-the-cma-for-the-benefit-of-all-the-people-forever-d3cd81964616
• Statens Museum for Kunst: https://www.smk.dk/en/article/smk-open/ ; https://medium.com/smk-open/smk-open-year-one-the-first-12-months-of-smks-online-collection-423e073d61e1
• The Metropolitan Museum of Art: https://www.metmuseum.org/about-the-met/policies-and-documents/open-access ; https://www.metmuseum.org/blogs/collection-insights/2020/met-api-third-anniversary