Research project · OICL · AHO · CRAITR · MishMash

OpenWaves

Acoustic telepresence for remote vessel operations

An open-source research project investigating acoustic telepresence for remote vessel operations. Operators shape incoming data streams — telemetry and captured audio — into soundscapes, crafting an acoustic environment for each vessel they monitor. The same soundscape also serves as input to an AI system that interprets both the data streams and the operator's configuration choices, and the AI communicates back by inflecting the soundscape itself: a continuous loop between human attunement and machine analysis.

Project lead: Enrique Encinas, Associate Professor, Oslo School of Architecture and Design (AHO); director of CRAITR
Home: Ocean Industries Concept Lab (OICL), with AHO · CRAITR
Programme: MishMash Centre for AI and Creativity · WP7, industrial operators and creative problem-solving (led by OICL), with WP1 and WP2
Research frame: Groise — Grammars of Noise
Instrument: Eight sound streams of a remotely operated vessel, mixed on an Intech Grid; fifteen vessels; its body is TAO, a self-hosted AI instrument running open-weights models only
Interface: OpenBridge, the open-source design system for maritime interfaces
Demonstrated: MishMash LIVE, Kraftverk, MishMash Opening Conference, Kristiansand, 14 September 2026
Collaborators: Ricardo Simian · sensiBel · Soundly · Markéta Dolejšová and Mikuláš Rett (AVU Prague)

Research question

Does this relation to the vessel — presence through acoustic attunement, alongside observation through visual dashboards — support different capacities: sustained awareness without fatigue, pattern recognition through embodied familiarity, sensitivity to change through dwelling in a soundscape over time?

Three ideas

Presence through sound: Remote monitoring places operators as observers of screens. Here an operator enters an acoustic environment and shapes it until the vessel's rhythms become legible. This crafting is attunement.
The AI inflects; it does not alert: The AI speaks through the soundscape: a stream brightens, moves forward in the mix, or gains texture. Three moves: brighten, sympathetic pull, return.
Visual noise as resource: Remote interfaces cancel boat motion, engine vibration, sea clutter and horizon drift to spare the operator. That erased data carries information; OpenWaves sonifies it.

Extends OpenBridge

OpenBridge is an open-source design system for industrial and maritime interfaces, developed since 2017 by the OICL consortium of 45 partners. It sets visual standards for safe, efficient operations. OpenWaves adds the auditory dimension: acoustic design guidelines beside the visual standards, multimodal alert hierarchies, and joint evaluation. This site is built from OpenBridge's own web components.

On this site

Kraftverk · demoThe demonstration at MishMash LIVE, 14 September 2026: the evening replayed from the instrument's own record, and the instrument to play.
The body: TAOThe self-hosted AI instrument underneath, from Lanzarote and Nezabudice to the box on the table.
ManualThe concept in full: streams, hands, moves, sound, architecture.
TimelineEvery deployment and room, November 2025 to October 2026.
Photos82 images: two builds, five places, the spectrograms.
Partners & resources

Where the same box ran

Kraftverk · demo

MishMash LIVE at Kraftverk

Kristiansand, Monday 14 September 2026, 19:00

The MishMash Opening Conference (Kristiansand, 14–15 September 2026) opened the MishMash Centre for AI and Creativity. On its first evening, at MishMash LIVE at Kraftverk, OpenWaves was demonstrated as a working instrument, beside Ricardo Simian's OpenWave V0.1, which asks the same question at the scale of a single interface event.

Visitors stepped between fifteen boats and mixed eight sound streams of a remotely operated vessel on an Intech Grid, with TAO on the table. Visitors accumulated: the state left by one was the seed of the next, and by the evening the instrument was not the one it had been in the morning. The same day TAO ran the participatory session NE #01 (11:40) and was presented as a poster (12:45 and 15:30).

On the table

Hands: Intech Studio Grid (EF44 · TEK2 · VSN1 · PBF4), 34 learned addresses: faders are stream weights, encoders each stream's character, two jog wheels the vessel and world masters
Body: TAO: NVIDIA GB10, 128 GB unified memory, 1.48 kg, open-weights models only, no cloud in the room
Streams: Engine · Wind · Sonar · Alarms (the vessel half) · Vessels · Radio · Humans · Marine (the world half)
Boats: Fifteen, each with its own world and recorded loops; a change of boat crossfades over two seconds
Sound: Recorded loops mixed by the operator's gestures; alarms as exact one-shots, never generated. The generative layer (Magenta RealTime 2) was held back from the table
AI: Three moves on the stand's rules: brighten, sympathetic pull, return
Screen: The OpenBridge front end, playable by touch when no Grid is on the table

The evening in numbers

From the instrument's own record: its sound, taken from the sound card's playback (no room microphone), and its state ledger, one line per second. The visitors' voices are not here.

The night, replayed

Play the evening

0:00 / 2:59:36

Boat on the table

Kólga

Instruments: simulated from the boat's world. Faders and moves: the ledger.

Play the instrument

The bridge as it stood at Kraftverk, with the loops the stand played. Choose a boat, press Sound on, and mix: each stream has a weight and a character (near to far); the masters set the two halves. Hold a stream high and the AI answers.

Fifteen boats · Kraftverk, 14 Sep 2026

Own vessel

Kólga · crisis

Wind

Bow thruster

TAO, in the field

Eight streams · Kólga

Sound on

The stand's own loops, mixed in your browser the way the stand mixed them at Kraftverk. Headphones help.

The instruments are simulated from each boat's world; the sound is the stand's. Hold a stream high: the AI's moves answer on the stand's own rules, ten times faster. The world half sways with the roll.

VESSEL master WORLD master

How the halves meet

Direct

Split

Sidechain

Convolve

Spectral

Freq divide

Vocoder

Beyond

The modes steered the generative layer (Magenta RealTime 2), which stayed on the box: here they are shown, not heard.

Advice

The AI does not alert. It inflects.

Motion

Propulsion

The fifteen boats

The body

TAO

Transductive Acoustic Organism

A persistent, open, self-hosted AI instrument for artistic and design research. The instrument at Kraftverk runs on one machine, an NVIDIA GB10 with 128 GB of unified memory, 1.48 kg, running only open-weights models. No cloud in the room. Its ear is a sensiBel AURORA: two studio-quality optical MEMS microphones.

The name has changed once: the 2025 report calls it Noise Catcher for Acoustic Telepresence; the 2026 poster, Transductive Acoustic Organism.

Aim

What AI infrastructure do artists and cultural institutions actually need to work critically, experimentally and sustainably, when the compute that shapes cultural production is owned and governed elsewhere? And where is the line between analysis and confabulation in a machine that runs continuously, accumulates context over months — and reads people?

Methods

Artistic research by building and operating. TAO is a persistent instrument on one self-hosted box running open-weights models only: a 27B language model, two ears, an embedding model, a model that hears and sees, music (Magenta RealTime 2) and sound (Stable Audio 3). Two prototypes exercise it — Open Ambient (acoustic telepresence for maritime operations, with OICL) and Warp (latent audio space as live material) — and two field experiments, NE #00 and NE #01, run the mechanics of a model on a room of people: encoding, embedding, forward propagation, with consent as the first component.

Results

NE #00, Ars Electronica, 9 Sep: 24 people, no internet, one box; 22 encodings mapped into five groups with a stated reason each; 1,501 relations on the wall; the room amended the machine's ledger. NE #01, Kristiansand: 274 registered participants read from the public web in one night and gated for wrong-person risk — most came back thin, and that is the finding; ten statements; a hundred people placed where they stand by what they said, told why afterwards. Measured on the way: a hotspot that seats 14 phones; unified memory where a second model kills the first; a room that needs 1.5-second service no cluster provides.

Value

An instrument as research object and infrastructure for multimodal creative inquiries. OpenWaves is the instrument's maritime limb: the same box that seats a room of a hundred people on Monday morning plays the ship on Monday night — infrastructure small institutions can own, built for a purpose rather than for scale.

Four bodies of one idea

v1 · Jan 2026: On the desk. A Logitech MX controller and a laptop; Ableton Live, mock telemetry, OSC.
v2 · 19 Mar 2026: OICL seminar, 27 industry participants. The Intech Grid and four bridge screens; the eight-stream mixer in Web MIDI and Web Audio.
OpenWaves · 14 Sep 2026: MishMash LIVE, Kraftverk. The same Grid and TAO; 24 loops per vessel, the OpenBridge front end, a generative layer measured on the box.
The stage · 13 Oct 2026: Black Box, Helsinki Music Centre. TAO and eight simulated bodies; TAO hears the room and moves the world's knobs.

In the field, 2025

The first TAO was autonomous listening infrastructure in exposed places — Lanzarote in November 2025, Nezabudice in December — with methods for acoustic capture in wind, power-constrained transmission and operation without maintenance. Those methods carry to vessel-mounted microphones.

98.8%uptime over a 7-day deployment

99.2%upload success

95%less modem power, store-and-burst

2.61Wsystem average, down from 4.45 W

Continuous streaming at 4.45 W could not balance on solar. A 19-minute acquisition / 1-minute transmission duty cycle keeps the modem active 5 % of the time.

Latency as gravity

The limits of computation and networking don't just affect performance; they shape what becomes perceptually salient in remote sensing, and in what order. Audio arrives first and becomes the continuous ground truth; video follows as a secondary, discontinuous layer.

We hear the planet before we see it.

Recordings

From the TAO report, captured with sensiBel AURORA MEMS microphones.

The anthropomorphized form

TAO V2 consciously places itself in a lineage of anthropomorphized listening technologies. Like the Neumann KU 100 "Fritz", its sensiBel AURORA microphones inside furry windscreens make a ritualized head-form — a modern interpretation of binaural recording.

TAO sits at the intersection of scientific instrument, autonomous agent and ritual object, channeling environmental acoustics into networked presence.

Timeline

Every place TAO listened, every room it ran

November 2025 to October 2026, newest first

Manual

How OpenWaves works

Streams, hands, moves, sound, architecture, and the TAO report's findings

Photos

Two builds, five places, the spectrograms

Partners & resources

Who it is made with

Part of Groise — Grammars of Noise, and of MishMash WP7 with WP1 and WP2. Project lead: Enrique Encinas. Ocean Industries Concept Lab and the Oslo School of Architecture and Design · CRAITR.

The second table at Kraftverk was Ricardo Simian's OpenWave V0.1 — five soundsets, seven interface events, four controls: the sound of the interface at the scale of a click. OpenWaves is the same idea at the scale of the watch.