Whale songs drive the render engine in a Singapore show
Marshmallow Laser Feast built the work with filmmaker Tom Mustill and bioacousticians in Barcelona, then let the animals’ own recordings deform the picture in real time.
The technical fact at the centre of Seeing Echoes in the Mind of the Whale is better than the wall text. Marshmallow Laser Feast fed raw cetacean vocalizations
into a real-time render engine as a live input, so the audio does not accompany the picture — it deforms it. Sound sculpts the image while you watch. That is a rendering pipeline built to argue a point about perception, and it is currently the centrepiece of Into the Ocean: Journey Beneath at the ArtScience Museum in Singapore, where it runs through November 1.
The piece moves through three species, and the first chapter carries the whole idea. Bottlenose dolphins process echolocation in the visual cortex, which means the returning echo arrives as something closer to an image than a sound. There is no good human word for that, so the studio built the experience instead: a world assembled out of returned signal. Humpbacks get the second chapter and their songs; sperm whales get the third, clicking across distances that make the notion of a rendered draw distance look modest.
Marshmallow Laser Feast — Ersin Han Ersin, Barnaby Steel and Robin McNicholas — developed it with the filmmaker Tom Mustill and the Laboratory of Applied Bioacoustics at Universitat Politècnica de Catalunya, working from recordings supplied in part by the Monterey Bay Aquarium Research Institute. It was commissioned by Fundación Telefónica and Disseny Hub Barcelona in 2024 and curated by José Luis de Vicente, so Singapore is a later home rather than a birthplace. It reaches the MIT Museum on September 28 for its US premiere.
What keeps this on our beat is the tooling. Everything here — volumetric reconstruction, a generative pipeline for segmenting footage, an engine running the whole thing live — came out of games and got pointed at an animal. Jakob Kudsk Steensen made the same move in our conversation about ecology and the game engine. The engine turns out to be very good at building an umwelt that is not yours. The studio’s own opening-day post has the installation shots.
Game engines keep turning up in places nobody files under games. We follow them there every week.
