skip to content
← (projects)

synesthesia.jpg

Synesthesia – The Art of Hearing Pictures

Lets blind and visually impaired people explore paintings through sound instead of spoken description.

Context

Synesthesia — The Art of Hearing Pictures is Heidi's ongoing project to make art and museums accessible to blind and visually impaired people through sound rather than spoken description. It sits exactly at her favourite intersection: art × technology × accessibility. The first prototype was built in 24 hours at the Music & AI Hackathon in the Austrian Alps.

The problem

Museums mostly offer blind visitors audio descriptions: someone tells you what a painting looks like. You never get to explore it yourself, in your own order, at your own pace. The same question applies to music creation, which is rarely built to be accessible and intuitive for people with disabilities.

What I built

A picture-to-music translation tool. What starts as a static image becomes something you can actually play:

  • The artwork is divided into a grid, and each section is read for its colour palette, composition, objects (via AI detection) and the mood of the scene.
  • Each of those signals is mapped to chords, melodies and soundscapes, so the painting becomes a soundscape.
  • With a Chordcat controller, users move across the painting section by section and hear how each part translates into sound, building their own mental picture instead of listening to a narrator.
  • Users can save the melodies they like and build their own composition by remixing the mappings or swapping instruments.
  • A working demo and a pitch deck (Canva) for presenting the project.

Tech

AI object detection and image analysis · colour, composition and mood-to-music mapping · Chordcat controller · sonification / sound design · web app (built with Lovable) · Canva (pitch)

Why this shows I can…

Turn a human insight into a novel interaction concept and a working demo in a single weekend. It builds on her earlier MRI sonification work.

Status

Working demo; pitch in preparation. Born at the Music & AI Hackathon in September 2026.

Links