music-ai-hackathon-alps.jpg

Music & AI Hackathon – Austrian Alps

24 hours at 2,315 m: a team of engineers and artists built Synesthesia, a picture-to-music tool that lets blind and visually impaired people hear paintings.

Context

Mountains instead of an office. At 2,315 m in the Austrian Alps, Heidi joined the Music & AI Hackathon: 24 hours of building, far too little sleep and one of the most unusual working environments of her 30+ hackathons.

Team: David Smith, Nelli Funk, Sara Rutz, Berkay Cengiz, Zardasht Ilyas and Heidi — engineers and artists thrown at the same problem.

The challenge

How do you make music creation more accessible and intuitive, especially for people with disabilities?

What we built: Synesthesia

The idea started with a single image: someone standing in front of a painting in a museum, unable to fully experience its colours, shapes or subject because of a visual impairment. What if the painting could be heard instead of seen?

Synesthesia is a picture-to-music translation tool. It divides an artwork into a grid and reads each section for its colour palette, composition, objects (via AI detection) and even the mood of the scene. Each signal is mapped to chords, melodies and soundscapes. With a Chordcat controller, you move across the painting section by section, listen to how each part translates into sound, save the melodies you like and build your own composition by remixing the mappings or swapping instruments.

The project continues as an ongoing project: see Synesthesia – The Art of Hearing Pictures.

Behind the scenes

Sitting in a hotel lobby at 6 am, deep in signal data, trying to get the last piece working before the deadline. The team was so heads-down that it never managed a proper team photo.

Thanks

To Jan Bendick, who helped untangle some of the trickiest parts, and to the organisers Florian Bauer, Nobuko Okazaki, Michael Otter and Seiichiro Matsumura for bringing so many different minds together at the top of a mountain.

Why this shows I can…

Go from a human insight to a working prototype in 24 hours, work across engineering and art, and design technology for accessibility.

Result

Mountains, minimal sleep and a working prototype by the end.