During a five-day residency in Milan, choreographer Giovanni Sabelli Fioretti and creative coder & visual artist Martyna Chojnacka worked intensively to develop an interactive system capable of transforming audience participation into a live compositional element. What emerged was not simply a technical setup, but a performative environment in which spectators actively shape the atmosphere of the piece through their own responses.
“The project changed the way we think about audience participation. The spectators were not simply invited to interact with the work; their responses became part of the choreographic material itself. This created a subtle but powerful shift: the performance was no longer something happening in front of them, but something unfolding with them.”
The audience enters the experience through a simple gesture: scanning a QR code and connecting to a local network. From there, they access a custom-designed website featuring an animated survey. Their responses are then transmitted in real time into a TouchDesigner environment, where they influence visuals, sound, and lighting throughout the performance.
Behind this apparently seamless interaction lies a complex technical architecture. During the residency, the team implemented a WebSocket server within TouchDesigner, developed a Python-based survey engine, and created a scene-management system capable of switching dynamically between visual chapters. They also integrated Stream Diffusion to generate AI images live on stage, blending these outputs with camera feeds of the dancers. Audience-generated data was simultaneously routed via MIDI to both the musician and the lighting desk, allowing the entire performance environment to react continuously to the spectators’ input.
“The most interesting part was connecting all the elements together — the audience interaction, AI visuals, live camera, sound, and lighting — into one responsive environment that kept evolving throughout the performance. Because the system reacted live and wasn’t completely fixed in advance, every performance unfolded differently, which kept the process exciting for us as well — even as people creating the piece behind the scenes, we never fully knew what was going to happen.”
The result is a flexible and reusable framework in which participation becomes part of the choreography itself.
From Prototype to Performance
One of the most significant outcomes of the residency was the creation of a fully functioning prototype that was not only tested during development, but successfully integrated into the premiere of M.oving O.thers in Berlin earlier than originally anticipated.
Alongside the live system, the project also produced a custom-built website, documented TouchDesigner patches and code structures, as well as an extensive archive of notes, images, and video excerpts now used for communication and future funding applications.
Yet some of the most important outcomes were intangible. The residency deepened a strong trust-based collaboration between choreography and creative coding, transforming the relationship from a technical collaboration into a genuinely co-creative process. Artistic and technical questions evolved together, with dramaturgy, audience engagement, network systems, AI integration, and interface design continuously influencing one another.
The spectators were not simply invited to interact with the work; their responses became part of the choreographic material itself. This created a subtle but powerful shift: the performance was no longer something happening in front of them, but something unfolding with them.
Reflecting on the process, the collaborators noted: “What moved us most in this process was the discovery that technology does not need to distance us from the body. On the contrary, when it is approached with care, it can create new forms of attention, listening and presence. In M.oving O.thers, AI, audience input, sound, light and movement became part of the same living organism.”
Through this process, both collaborators expanded their skills in interactive systems, AI image generation, and audience-centred dramaturgy. At the same time, the project strengthened their visibility as artists working at the intersection of dance and AI-driven live media.
Reflecting on the residency, the team considers the outcomes to be closely aligned with their initial vision. Not only were the original goals achieved, but the system was integrated into a full-scale performance sooner than expected, creating a strong foundation for future works and iterations.
Where Choreography Meets Code
One of the defining aspects of the project was the way different disciplines became inseparable from one another.
The collaboration moved beyond a model in which technology merely “supports” performance. Instead, choreography, coding, interface design, and audience interaction became part of the same creative language.
The survey website itself evolved into a dramaturgical element, with its animations and interaction design functioning as extensions of the choreographic space. Audience data was treated not as information to collect, but as compositional material capable of shaping visuals, sound, and light in real time.
“One of the strongest impacts of M.oving O.thers was the possibility of treating AI not as a visual effect, but as a live performative partner. The generated images were unstable, responsive and unpredictable, just like another presence on stage. This opened a new dialogue between bodies, machines and spectators.”
At the same time, AI was approached not as a post-production effect, but as an active live presence within the performance. Through Stream Diffusion, generated imagery continuously rewrote the visual environment on stage, creating an unstable and evolving relationship between bodies, machines, and spectators.
This process fundamentally reshaped the collaboration itself. The residency highlighted the importance of involving the creative coder from the earliest stages as a co-author rather than a technical service provider. Dramaturgical decisions were directly influenced by technical structures such as WebSocket logic, latency, and system architecture, while artistic intentions simultaneously pushed the technical setup to evolve.
Learning Through Interaction
As the system developed, one of the clearest lessons emerged from observing audience behavior.
The success of the interaction depended not on technological complexity, but on simplicity and clarity. The audience needed to feel comfortable participating without disrupting the poetic atmosphere of the performance. This required careful attention to interface design, instructions, and accessibility across different devices and operating systems.
The team realized that even highly experimental systems must remain intuitive at the level of user experience. A stable local network, lightweight website design, and clear visual and verbal guidance became essential parts of the artistic process itself.
Another important discovery was the dramaturgical power of audience data. Allowing spectators’ responses to influence sound, visuals, and lighting transformed their role within the piece. They were no longer passive observers, but active co-shapers of the performance environment.
One of the strongest impacts of M.oving O.thers was the possibility of treating AI not as a visual effect, but as a live performative partner.
At the same time, the residency reinforced practical lessons around technical production. Extensive testing, modular system design, and detailed documentation proved essential for ensuring the setup could remain adaptable, transferable, and reliable across different venues and future projects.
Perhaps most importantly, the project confirmed that experiments at the intersection of art and technology require time for dialogue, prototyping, failure, and revision. Yet it is precisely through this process that new performative formats can emerge.
Challenges and Adaptation
The main challenge involved making the interactive system accessible and reliable across different devices and browsers. Early tests revealed connection issues and uncertainty among spectators about how to participate.
To address this, the team simplified the interaction process by using a single QR code, clearer instructions, and a lightweight website optimized for different devices. Rehearsals and repeated network testing also helped stabilise the connection between visuals, sound, lighting, and AI systems during live performance conditions.Although unpredictability remains part of any networked live system, these adjustments allowed the team to create an experience that felt both stable and engaging for audiences.
Watch their full performance
hereImages: Courtesy of M.oving O.thers
This case study was created under Creative FLIP, an EU co-funded project aimed at further increasing the long-term resilience of the CCSI in key areas such as Finance, Learning, Working Conditions, Innovation & Intellectual Property Rights.
Key Takeaways
- Audience participation can become an active compositional element in live performance
- Creative coders are most impactful when involved as co-authors from the beginning
- Simplicity and intuitive interaction are essential for audience engagement
- AI can function as a live performative partner rather than a post-production tool
- Modular and well-documented systems support future adaptability and reuse
- Cross-sector experimentation requires time, testing, and openness to failure
