Sonic Mutations (2023)
Live performance visuals for exploratory electronic music
In 2023 I worked with Kopi Su Studio to produce visuals for Sonic Mutations; a generative music exploration developed by Kopi Su with artists Alexis Weaver and Rowan Savage (salllvage).
The work put bleeding edge AI music/audio generation tech in the hands of artists. Alexis and Rowan each developed a performance incorporated recorded and sampled audio with sounds generated live using the AI tools built by Kopi Su.
I came on to the project to develop accompanying visuals for the performances that would respond live to the performance audio. The secondary goal was to reveal a bit of the AI model’s internal state to the audience.
Sample clip of the visuals developed for Alexis Weaver's piece. Visuals are primarily abstract, and aimed to reflect the mutation of recorded audio in the piece. Text at the bottom reflected the text prompt Alexis used at any given moment and the percentage of denoising. This clip has been sped up.
The visuals for both performances were developed in consultation with the artists. I made a bunch of early experiments that we presented back to Rowan and Alexis and adapted the style based on what they resonated with.
The works primarily used graphics shaders (opengl) for most imagery, though for Rowans work we additionally used 3d models of a crow and a human, rendered with partial white lines joining points in the wireframe at random.
Sample clip of the visuals developed for Rowan Savage's piece. Rowan's performance transitioned his voice slowly to the call of a crow, then back to the voice of a human as understood by the generative AI. The visuals tracked this arc transitioning from the model of the human to that of the crow and back again. This clip has been sped up.
Further performances with the tool (sonic mutations) and accompanying visuals are planned for this year. The full live stream of the original performance is available to watch on the Sydney Opera House website.