Trencadís — hear what you see

The lore

Born from a dream about a cyborg

Neil Harbisson was born with achromatopsia — he sees the world entirely in greyscale. So he implanted an antenna that translates colour into sound, with green tuned to 440 Hz, the middle of the visible spectrum. He became the first person officially recognised as a cyborg by a state, learned to hear colour, and turned his condition into his art.

Around the same time, I was building something adjacent: a Holophonic User Interface — spatial sound that let blind users hear where the icons were on a screen and navigate towards them. It won a business-ideas contest and became my degree thesis. It never shipped. But it left an idea humming in the back of my head.

“I can hear colour.” — Neil, in the dream

“I can make people hear where things are in a virtual space.” — me, in the dream

I woke up thinking: so what if you could hear where the colours are?

That dream really happened. What followed was a Processing sketch that grew into a beautiful little monster: the camera feed crushed into fat pixels, each one rotated by its hue until the image looked almost cubist, while a Pure Data synth underneath sang whatever the colours told it to — quantized to a musical scale, so it was always harmonic, never noise. The scale-quantizing idea came from another project of mine: a one-string bass you could lock to any scale and improvise on like a virtuoso.

2013

The Processing monster

A sketch grows out of control but works. Camera in, cubist pixels out, libpd singing underneath.

A 2013 Mac desktop running sonoChromAceed, a Processing sketch showing a camera feed crushed into cubist pixels beside synth sliders, with Processing code and an Arduino sketch open behind it — **sonoChromAceed** · 24 oct 2013, 16:00 · the Processing original — libpd sliders on the right, the monster's code behind

2013 · six months later

It sounds like music

One holiday of stubborn coding later, colour is melody, quantized to a scale, surprisingly nice to listen to.

2026

Reborn in Kotlin

Ported to Kotlin and Jetpack Compose with Claude Code as copilot. Blob detection, convex-hull shards, mortar gaps — it finally earns the name Trencadís, after Gaudí's broken-tile mosaics.

Now

A camera that jams

MIDI clock in, notes out. It locks to your DAW and arpeggiates in rhythm and harmony with the rest of the band.

How it works

The synesthesia engine

Every shard and tile in the image is translated, property by property, into sound. Not random but alive — just like the palette of colors of what you are looking at.

Hue→ Pitch quantized to the scale you choose, so every colour lands on a right note. Green sits near 440 Hz — a nod to Neil.

Saturation→ Filter vivid colours open the filter bright; washed-out ones stay dark and soft.

Brightness→ Loudness & size bright shards ring louder and draw bigger on screen.

A good warning

Trencadís does not record, gather, or send any data to anywhere — and never will. The camera feed is processed entirely on your device, in real time, and discarded the instant it becomes sound and mosaic. Nothing is stored, nothing is transmitted, nothing is watched. Your camera is yours. Nobody sees it, ever.

In / out

What it sees · what it makes

An oil portrait on the studio wall, and the same painting through Trencadís: colour blobs hulled into shards, hues quantized to a scale, control panels waiting at the edges of the screen. The green dot in the corner is MIDI clock — alive and counting.

Expressionist oil portrait of a man wearing a beret and a blue scarf, painted in thick strokes of red, yellow, green and blue — **Input** · oil on board, thick strokes, no electricity involved

The same portrait rendered by the Trencadís app as a mosaic of flat colour shards, with round control icons along the screen edges and a green MIDI clock indicator — **Output** · the same painting, broken into tiles and sung

Under the tiles

A real synth wearing a mosaic

Sound engine

Pure Data running on-device via libpd. Five mixable oscillators — sub, sine, saw, square, noise — through a resonant filter with envelope modulation.

FM, chorus, delay and reverb on top. Tap tempo, seven octaves of range.

Mosaic vision

Connected-component blob detection finds regions of colour; convex-hull polygons with mortar-line gaps render them as authentic broken tiles.

Temporal smoothing lets shapes evolve organically instead of flickering, plus ten animated acid patterns when you want to melt.

Ways to play

Four pixel-selection modes: Sequence cycles melodically through the frame, Brightest chases light sources, Center holds a stable drone, and Pointer turns your finger into a theremin.

In the rig

USB MIDI note output and clock reception, Bluetooth LE MIDI peripheral mode, 16 channels, and a visual clock indicator.

Send it a chord or scale change from your DAW and it re-tunes on the beat.

The uncommon part

Point it at anything. It plays in your key.

Sync it to your DAW and Trencadís becomes an arpeggiator whose keyboard is the visible world — locked to your clock, tuned to your chords, improvising from whatever the lens is looking at.

A houseplant, a traffic light, your bandmate's shirt: everything the camera sees becomes a voice in the arrangement. Few instruments work this way — which is exactly the point.

Demo film

See it move, hear it sing

The pitch — the short version

The full demo — in all its glory

Beta

Get it

Try it now — or build it yourself

Testing phase

Trencadís is in closed testing on its way to the Play Store. Google requires a crew of brave testers before the doors open — and that's where you come in.

Send me the Gmail address you use on your Android phone, using the form below.
I add you to the testers list — then open the opt-in link and tap Become a tester. (It only works once you're on the list.)
Install it from Google Play, play, and tell me everything. Break it. Especially break it.

✅ Join the testers list ▶ Trencadís on Google Play

Already on the list? The buttons above are your door in.

📱 Download for Android View on GitHub

Requirements:

Android 7.0+ (API 24)
Camera permission required
Optional: Bluetooth MIDI controllers

# clone and install on a connected device git clone https://github.com/Nasker/trencadis_app.git cd trencadis_app ./gradlew installDebug

Built with · Kotlin · Jetpack Compose · CameraX · pd-for-android · Material 3
License · The Unlicense — public domain. Take the tiles, build your own mosaic.