- Team
- Kobe Phillips
- Course
- Design for Physical Interaction, Fall 2024
Drum & Vase
LLM Music Generator
- Arduino
- Sensors
- OpenAI API
- Stable Audio
- Bluetooth audio
Idea
Give a large language model a sense of place, so it can “read the room” and compose music that reflects it.
Approach
Arduino sensors feed an LLM that describes how the space feels, then an audio model turns that into music played from inside the object.
Outcome
A working object whose looping soundtrack shifts as its environment changes.
What if our environments could respond to us?
How it works
- 01
Sensors read sound, soil moisture, light, proximity and temperature
- 02
Values go over serial to prompt the OpenAI API
- 03
The model describes the place, then writes a music prompt
- 04
Stable Audio generates an mp3
- 05
A Bluetooth speaker inside plays it on a loop
The system
Why it matters
The constant loop and a changing environment transform the sound over time, creating a living ambience.
Design features
Stackable modules, customizable skins, and an Arduino hidden inside as a black box.
Testing it in alternative locations
The same object, three environments: what the sensors read, what the model felt, and the music it asked for.
Buttermilk Field
Sound 8 · 6.2 °C · moisture 427 · 503 cm away · 354 lx
It felt
“A sense of solitude in this cold, spacious environment … desolate yet peaceful.”
It asked for
Slow and melodic, ambient or classical
Sibley Studios
Sound 34 · 22.3 °C · moisture 683 · 18 cm away · 498 lx
It felt
“Lively activities taking place, but the distance reading of 18 cm makes me feel a sense of loneliness.”
It asked for
A lively beat, upbeat pop with contemplative ballads
Disney World (in theory)
Sound 70 · 40 °C · moisture 587 · 3 cm away · 900 lx
It felt
“The chaotic sound around me is overwhelming, making me feel anxious and on edge.”
It asked for
Fast tempo and energetic beats, electronic and world music
Final prototype · click to enlarge
Outdoors
In hand