Research:Image recognition of painting subject matter

Contact

Logan Lee

University of Virginia

researcher

Sudeepti Surapaneni

University of Virginia

researcher

Sana Syed

University of Virginia

researcher

Raf Alvarado

University of Virginia

faculty advisor

Jennie Choi

Metropolitan Museum of Art

subject matter expert

Lane Rasberry

University of Virginia

Wikimedian

---

Research:Projects

This page documents a completed research project.

Students in the School of Data Science at the University of Virginia used machine learning techniques in image recognition to topic tag paintings in the collection of the Metropolitan Museum of art.

Challenge

Identify the subject matter of paintings, including details such as recognizable elements and features

Sample images are in commons:Category:Paintings in the Metropolitan Museum of Art and here:

• 
  The Musicians Caravaggio
• 
  The Fortune Teller de La Tour
• 
  The Horse Fair, Rosa Bonheur
• 
  A Girl Asleep, Vermeer
• 
  Golden Cock and Hen, unknown
• 
  Pity, William Blake

Objective

create metadata for what paintings depict; add this to Wikidata "depicts"statements (d:Property:P180) and integrate output into "structured data on Commons", in addition to producing a standalone dataset

Timeline

Late August 2019

Students select research projects from an available pool

Late September 2019

Proposal presentation

May 2020

Project ends

Background

Data management in Wikimedia projects

• https://dumps.wikimedia.org/, "A complete copy of all Wikimedia wikis, in the form of wikitext source and metadata embedded in XML."
• d:Wikidata:Data access
• d:Wikidata:How to use data on Wikimedia projects
• Research:Quarry, a tool with a support community which could assist with presenting the list of users who received a block

Metropolitan Museum of Art digital media

• Tallon, Loic (7 February 2017). "Introducing Open Access at The Met". Metropolitan Museum of Art. 
• Knipel, Richard (7 February 2017). "The Metropolitan Museum of Art makes 375,000 images of public domain art freely available under Creative Commons Zero – Wikimedia Blog". Wikimedia Blog. 
• "Open Access at The Met". www.metmuseum.org. Metropolitan Museum of Art. 
• "Image and Data Resources - Open Access Policy". www.metmuseum.org. 
• Chen, Amanda (20 December 2017). "A Deep Dive into The Met’s Collection Information Digital Work System". Medium. 

• In Wikimedia projects
  • commons:Category:Paintings in the Metropolitan Museum of Art

Similar efforts

• wmania:2019:Research/Discovering and Analyzing Wikimedia Images
  • Projects of Miriam (WMF), research scientist at Wikimedia Foundation
  • 2019 tutorial on GitHub
• commons:Commons:Structured data

Outcomes

• Surapaneni, Sudeepti; Syed, Sana; Lee, Logan Yoonhyuk (24 April 2020). "Exploring Themes and Bias in Art using Machine Learning Image Analysis". 2020 Systems and Information Engineering Design Symposium (SIEDS) (IEEE). doi:10.1109/SIEDS49339.2020.9106656. 
• University of Virginia School of Data Science (3 April 2020). "Creating a Global Museum: UVA School of Data Science 2020 Capstone Project". YouTube. University of Virginia School of Data Science. 
• Evett, Meg (13 April 2020). "Creating a Global Museum of Art". School of Data Science. University of Virginia.