Like most industries, cultural sector institutions are beginning to integrate general-purpose AI tools such as ChatGPT, text summarization, and chatbots into their daily operations to increase efficiency. However, to truly harness the revolutionary potential of machine learning, they will need specialized tools that can be customized for specific tasks.

This problem calls for a interdisciplinary approach this would allow experts from cultural institutions to collaborate with AI developers. The resulting models should be simple enough for museum staff to use internally by institutions of all sizes.

This mission was at the forefront of a recent three-year research project Transforming collections: reinventing art, nation and heritage in the UK, the aim was to explore how machine learning could usefully address long-standing problems facing the custodians of our public collections, such as how to counter structural bias or how to surface deleted stories. Supervised by the University of the Arts London (UAL), the project is part of the five-year program Towards a national collection programme, which aims to find technological solutions to break down barriers between cultural institutions in the UK

Stephanie Dinkins speaks at the Museum x Machine x Me conference at Tate Modern, London in October 2024. Photo: Josh Croll, courtesy of Tate.

The results of Transforming Collections were made public during the Museum x Machine x Me conference at the Tate Modern in London in early October. Artnet News spoke to Professor Mick Grierson, a researcher at UAL’s Creative Computing Institute who led the research project, to hear his three main takeaways.

1. The power of interdisciplinary computing

Machine learning solutions for the cultural sector should be developed by an interdisciplinary team of experts including curators, art historians and data protection officers.

“Our collaborators at the Tate museums and elsewhere were giving us the examples we needed to understand what AI was actually going to be useful for,” Grierson explained. “Our goal was to build something that allowed them to do this the way they wanted, and that meant really understanding what their concerns were.”

He gave the example of computer scientists who have enabled museums to automatically label digitized objects in their collections to make the creation of large searchable databases more efficient. “We found out in about 30 seconds that this was never going to work,” Grierson said. In one case, for example, they found that general-purpose computer vision models failed to indicate the race of a subject’s photographs or paintings, which is relevant information in a museum context , especially when decolonizing collections is a high priority.

Museum x Machine x Me conference at Tate Modern, London in October 2024. Photo: Josh Croll, courtesy of Tate.

“There is no commercial incentive (for Big Tech) to deal with this problem, so it is not solved. What you need is a public sector approach,” Grierson added. “What we need to do is change the patterns by training them (ourselves).” This approach is also suitable for museums who, of course, do not want to freely hand over their data to AI companies.

2. Fix long-standing problems in the public sector

The private sector’s reluctance to solve problems that are of high importance to public institutions creates an opportunity to create sector-specific tools that will “break the narrative that big tech companies are selling us.” As Grierson explains, “They’re actually pretty simple systems that you can retrain for your own purposes and make them do what you want, much more than just prompts.”

For example, many museums are currently working to identify omissions or absences in their collections. An AI tool developed by the Transforming Collections project allows users to analyze and search texts or images around a certain theme and identify even subtle structural biases. This “parser” model can be downloaded by museum employees and customized to their own context-specific needs, even using only small amounts of data on a local computer system. If useful, the model can then be easily shared with external colleagues.

Erika Tan, (r) Ancestral evocations (2024). Image courtesy of the UAL Decolonizing Arts Institute.

Tools like these should counter the idea that AI is too complicated to use if you don’t have expertise in the field. “We can empower people to integrate their perspectives into technology,” Grierson said. “It’s just a computer that does things. There is no magic. If there is an interface that allows you to do this, you can do it.

3. The importance of inclusiveness

Ideas shared at AI conferences can seem abstract when we don’t see how they could be implemented in the real world. A week-long program of cultural events ran alongside the conference, highlighting artists working with AI through displays installed by students on UAL’s Creative Computing Masters course.

Together, these projects “really represented the inclusiveness that we were looking for,” Grierson said. He highlighted the contribution of Stephanie Dinkinsa leading artist questioning how AI intersects with race and gender, whose keynote conversation kicked off the conference.

Another star was that of Erika Tan (r) Ancestral evocationsfor which she fed data on items from the Tate and Wellcome’s South East Asian collections into a “diagnostic instrument”, which converted them into a moving sound installation.

“These artists thought about the (museum) archives in the context of data and information,” Grierson said. “You get the feeling of a community that is meaningfully engaging with ideas and technology. We didn’t feel like we were just a group of academics. It was alive.