The de Young is now open to the public. Learn about extra precautions to welcome you back.

Buy Tickets

Visit the de Young

Exhibitions

Collections

Learn

Join + Give

Exhibitions & Events

Forensic Architecture, “Model Zoo: Introspecting the Algorithm”

Claudia Schmucki / Jun 19, 2020

With nearby Silicon Valley driving the development of artificial intelligence, or AI, Uncanny Valley: Being Human in the Age of AI is the first major museum exhibition to reflect on the political and philosophical stakes of AI through the lens of artistic practice. This series highlights select artworks included in the exhibition.

Forensic Architecture is an independent research agency that uses spatial methods to investigate civil and human rights violations. Founded in 2010 by Eyal Weizman at Goldsmiths, University of London, it employs architects, filmmakers, software developers, investigative journalists, lawyers, and scientists. The group works to reconstruct incidents, partnering with the communities and persons affected as well as with nongovernmental organizations, environmental justice and human rights groups, activists, and media organizations.

The group relies on open-source, citizen-gathered evidence: civilian camera footage, audio recordings, and eyewitness testimony. It analyzes this information through an architectural lens, using digital and physical models, 3-D animations, virtual reality environments, and cartographic platforms. Its pioneering methods have yielded decisive evidence in legal cases in national and international courts in Germany, The Hague, Greece, Israel, and Guatemala, in citizen tribunals and human rights processes, and have led to military, parliamentary, and United Nations inquiries. The group’s findings have influenced the types of evidence that courts will accept in legal settings. In addition, through its collaborations with outlets such as the New York Times, the Guardian, Der Spiegel, the BBC, and Al Jazeera, its work has influenced the way media platforms operate in an open-source environment.

Forensic Architecture began using machine learning in 2018, recognizing the technology’s potential to amplify and accelerate its research methods. In an early instance of this use, it produced a body of evidence confirming reports of a Russian military presence in Eastern Ukraine during the 2014 Battle of Ilovaisk. The group used algorithmic classifiers to sift through thousands of video clips of the region for frames that contained exclusively Russian military hardware. It submitted the project to the European Court of Human Rights in 2019.

Using the Unreal engine, Forensic Architecture generated thousands of photorealistic 'synthetic' images, situating the Triple-Chaser in approximations of real-world environments. Coloured 'masks' tell the classifier where in the image the Triple-Chaser grenade exists. Image: Forensic Architecture / Praxis Films, 2019
Using the Unreal engine, Forensic Architecture generated thousands of photorealistic 'synthetic' images, situating the Triple-Chaser in approximations of real-world environments. Coloured 'masks' tell the classifier where in the image the Triple-Chaser grenade exists. Image: Forensic Architecture / Praxis Films, 2019

Its recent project Triple-Chaser similarly used machine-learning methods to investigate US weapons manufacturer Defense Technology (DT). DT sells a tear-gas grenade known as Triple-Chaser to governments around the world that in turn use it on civil society protesters. Training an algorithm (or model, in machine-learning parlance) to detect an object of interest among millions of online images typically requires using thousands of annotated images from different perspectives, preferably in variable resolutions and contexts. Able to find only a few hundred images of Triple-Chaser online, the group trained its model on synthetic data—digital renderings of a variable 3-D architectural model of the canister. It placed this model within virtual environments simulating those in which Triple-Chasers are often used, such as urban streetscapes, and among randomly generated patterns and color fields to train the algorithm to distinguish the foregrounded object of interest from various backgrounds. In this exhibition, the models used in Triple-Chaser are at the core of a growing “model zoo,” which also includes models of munitions, weapons, bomb clouds, and sound effects. Intended to facilitate open- source research into violations of human  rights worldwide, such models also represent a catalogue of some of the most horrific weapons used by states in conflicts today.