ravenet
~

senSInt work on networks, control, agency, and liminal space in performance

2023 - present

This ongoing work, led by Dr. Paul Strohmeier and the Sensorimotor Interaction Group (senSInt) at the Max Planck Institute for Informatics (MPI-INF), is designing sensor-actuator networks between musicians in performance. The project has developed a series of instruments that use biosignal interaction to manipulate instruments - both own and others’ - to investigate themes of control, presence, and the blending of spaces in augmented performance.

Collaborators: Paul Strohmeier (senSInt, MPI-INF), Valentin Martinez-Missir (senSInt, MPI-INF), Rachel Freire (CiTIUS, University of Santiago de Compostela & MPI-INF)

abstract

RaveNET connects musicians through their biosignals and allows performers to manipulate performances through sensor-actuator networks. Networks of communication and control, albeit subtle and often directed internally between the performers, already exist in performance. Using biofeedback, these networks can become more overt and pronounced, allowing performers and audiences to understand the interconnectedness and liminal spaces — where the boundaries of interaction between musicians, instrumnets, and listeners become blurred — in collaborative music interaction.

RaveNET currently consists of three wearable controllers that function as nodes in the performance. Bones, an anti-corset for myself on vocals, uses capacitive sensing to detect stretch as the singer breathes. Tendons, a half-glove designed for Paul on bass, measures his galvanic skin response, pulse, and movement. Finally, Veins, a cap with electrodes for surface electromyography, captures Valentin’s facial expressions as he performs on drum machine. Interacting with signals in performance, we collaboratively reflect on our improvised performance: creative processes, spaces between our different bodies, our presence and control within the network, and how this makes us adapt our movements in order to be noticed and heard.

The RaveNET in action between myself, Paul, and Valentin. We explore the intersection of networks between us for performance communication and creation (yellow) and digital, biosignal, and electrical control.

contributions

RaveNET nodes — Bones, Tendons, and Veins designed and crafted by Rachel for instrument technique, individual bodies, and performance needs.
Exploration of technology’s presence and impact in performance and reflection on adaptation to novel feedback.
Design accounts from Bones and our Body Lutherie approach exploring tensions in vocal control and designing wearable for a changing body.

Bones (top), Tendons (middle), and Veins (bottom) through their prototyping. Each row shows the initial design by Rachel for our bodies and performance specs, initial fabric toile for fitting, revised design, and first prototype.

theory & methods

eTextile sensing and biofeedback
stretch shearing fabric sensors
electromyography (EMG)
galvanic skin response
audio processing
acceleration and gyroscopy
agential realism
vocal organology
feminist design

More images from the prototyping process for Bones and the Body Lutherie approach used to work with my changing physical body as a human-yet-separate collaborator in vocal performance.

2024

Body Lutherie: Co-Designing a Wearable for Vocal Performance with a Changing Body

Rachel Freire, and Courtney N. Reed

In Proceedings of the International Conference on New Interfaces for Musical Expression, Sep 2024

Abs Bib PDF

Research at NIME has incorporated embodied perspectives from design and HCI communities to explore how instruments and performers shape each other in interaction. Material perspectives also reveal other more-than-human factors’ influence on musical interaction. We propose an additional, currently unaddressed perspective in instrument design: the influence of the body not only the locus of experience, but as a physical, entangled aspect in the more-thanhuman musicking. Proposing a practice of “Body Lutherie”, we explore how digital instrument designers can honour and work with living, dynamic bodies. Our design of a breathbased vocal wearable instrument incorporated uncontrollable aspects of a vocalist’s body and its physical change over different timescales. We distinguish the body in the design process and acknowledge its agency in vocal instrument design. Reflection on our co-design process between vocal pedagogy and eTextile fashion perspectives demonstrates how Body Lutherie can generate empathy and understanding of the body as a collaborator in future instrument design and artistic practice.
@inproceedings{Freire_NIME24_BodyLutherie, title = {{Body Lutherie: Co-Designing a Wearable for Vocal Performance with a Changing Body}}, author = {Freire, Rachel and Reed, Courtney N.}, year = {2024}, month = sep, booktitle = {{Proceedings of the International Conference on New Interfaces for Musical Expression}}, address = {Utrecht, The Netherlands}, numpages = {10}, pages = {117--126}, issn = {2220-4806}, url = {http://nime.org/proceedings/2024/nime2024_18.pdf}, doi = {10.5281/zenodo.13904800}, editor = {Bin, S M Astrid and Reed, Courtney N.}, }
Liminal Space: A Performance with RaveNET

Rachel Freire, Valentin Martinez-Missir, Courtney N. Reed, and 1 more author

In Proceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction, Feb 2024

Abs Bib PDF

We present our musical performance exploration of liminal spaces, which focuses on the interconnected physicality of bodies in music, using biosignals and gestural, movement-based interaction to shape live performances in novel ways. Physical movement is important in structuring performance, providing cues across musical ensembles, and non-verbally informing other musicians of intention. This is especially true for improvised work. Our performance involves the use of our musicking bodies to modulate audio signals. Three bespoke wearable nodes modulate the performance through control voltages (CV) and interface with specific technical aspects of our instruments and techniques: 1) an “anti-corset” that measures the expansion and resistance of Reed’s abdomen while singing, 2) an augmented glove that assists Strohmeier’s bass/guitar signal routing across his pedal board and modular setup, and 3) a cap-like device that captures Martinez-Missir’s subtle facial expressions as he manipulates his modular synthesizer and drum machine setup. Through these performances we explore the notion of control in musical improvised performance, the interconnectedness and communications between our ensemble as we learn to collaborate and interpret each others’ bodies in this novel interaction.
@inproceedings{Freire_TEI24_RaveNETPerf, title = {{Liminal Space: A Performance with RaveNET}}, author = {Freire, Rachel and Martinez-Missir, Valentin and Reed, Courtney N. and Strohmeier, Paul}, year = {2024}, month = feb, booktitle = {{Proceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction}}, location = {Cork, Ireland}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, series = {TEI '24}, articleno = {105}, numpages = {4}, doi = {10.1145/3623509.3635337}, isbn = {9798400704024}, url = {https://doi.org/10.1145/3623509.3635337}, }
RaveNET: Connecting People and Exploring Liminal Space through Wearable Networks in Music Performance

Rachel Freire, Valentin Martinez-Missir, Courtney N. Reed, and 1 more author

In Proceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction, Feb 2024

Abs Bib PDF

RaveNET connects people to music, enabling musicians to modulate sound using signals produced by their own bodies or the bodies of others. We present three wearable prototype nodes in an inaugural RaveNET performance: Bones, an anti-corset, uses capacitive sensing to detect stretch as the singer breathes. Tendons, a half-glove, measures galvanic skin response, pulse, and movement of the bass player’s hands. Veins, a cap with electrodes for surface electromyography, captures the facial expressions of the drum machine operator. These signals are filtered, normalized, and amplified to control voltage levels to modulate sound. Together, musicians and nodes form RaveNET and engage with shared liminal experiences. In designing these wearables and evaluating them in performance, we reflect on our creative processes, spaces between our different bodies, our presence and control within the network, and how this made us adapt our movements in order to be noticed and heard.
@inproceedings{Freire_TEI24_RaveNETWiP, title = {{RaveNET: Connecting People and Exploring Liminal Space through Wearable Networks in Music Performance}}, author = {Freire, Rachel and Martinez-Missir, Valentin and Reed, Courtney N. and Strohmeier, Paul}, year = {2024}, month = feb, booktitle = {{Proceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction}}, location = {Cork, Ireland}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, series = {TEI '24}, articleno = {89}, numpages = {8}, doi = {10.1145/3623509.3635270}, isbn = {9798400704024}, url = {https://doi.org/10.1145/3623509.3635270}, }

abstract

contributions

theory & methods

related publications:

2024