Using p5js and Tone.js, I created a series of compositions that reconfigure temporarily on input from Kinect, a motion sensor, and generate soothing and energizing synthetic sounds that match the visual elements of the piece. The people in front of the projected visuals and sensor, in effect, create sonic compositions simply by moving, almost like a conductor in front of an orchestra but with more leeway for passive (if one simply walks by) or intentional interaction.
The sound is easy to ignore or pay attention to. It is ambient and subtle.
The next steps are to create more nuanced layers of interactivity that allow for more variation of manipulation of the sound and the visuals. Right now I am envisioning the piece becoming a subtle sound orchestra that one can conduct with various hand movements and locations of the screen.
Composition 1: Ode to Leon's OCD
In the editor.
Composition 2: Allison's Use of Vim Editor
Mouseover version for testing on the computer.
In the editor using Kinect.
Composition 3: Isobel
In the p5js editor.
Composition 4. "Brother"
In the p5js editor.
For this synchopated piece I sampled sounds of tongue cliks and and claps found on freesound.org. No tone.js was used in this piece.
Composition 5. "Free Food (in the Zen Garden)"
For this composition I used brown noise from Tone.js.
In the p5js editor.
After using mouseOver for prototyping, I switched over to Kinect and placed the composition on a big screen to have the user control the pattern with the movement of the right hand instead of the mouse.
Initial Mockups
Sketches of ideas
Inspiration
The simple grid-like designs I am creating are inspired by Agnes Martin's paintings.
Minimalist and sometimes intricate but always geometric and symmetrical, her work has been described as serene and meditative. She believed that abstract designs can illicit abstract emotions from the viewer, like happiness, love, freedom.
Code
Composition 1: "Ode to Leon's OCD" using mouseover code.
Composition 2: "Allison's Use of Vim Editor" with Kinectron code, using Shawn Van Every and Lisa Jamhoury's Kinectron app and adapting code from one of Aaron Montoya Moraga's workshops.
Composition 2 with mouseover for testing on computer.
Composition 3: "Isobel" with mouseover.
Composition 4 "Brother" with mouseover.
Composition 5. "Free Food (in the Zen Garden)"