The voice is body, instrument, and identity: To explore and unpack the relationship that vocalists have with their voices and vocal practice, this work involved autoethnographic research in my own singing and work with other vocalists and practices.
This work also explores the tensions and opportunities between third-person view of the voice by listeners, sensors, and digital agents, and the critical, embodied, first-person relationship of the vocalist. The vocalist’s understanding of their multi-sensory experiences is through tacit knowledge of the body. This knowledge is difficult to articulate, yet awareness and control of the body are innate. In the ever-increasing emergence of technology which quantifies or interprets physiological processes, we must remain conscious also of embodiment and human perception of these processes. Focusing on the vocalist-voice relationship, this project expands knowledge of human interaction and how technology influences our perception of our bodies.
Designing a vocal electromyography instrument, the VoxEMG. The original prototype worn here on my suprahyoid (left) with wired electrodes, conductive paste, and kinetic tape. The current PCB (right) is designed for textile incorporation and fabric electrode connections with castellated inputs.
contributions
VoxEMG electromyography platform: a novel vocal interaction method which uses measurement of laryngeal muscular activations through surface electromyography (sEMG).
Long-term first-person/autoethnographic and in-depth work with other vocalists on reflection on the incorporation of biosignal feedback understanding body movements and vocal practice and how such feedback can function as a metaphor.
Comprehensive examination of how technology and the feedback we receive in human-computer interaction (HCI) can shape our perception and understanding of our bodies and our actions.
Strategy for adopting technologies from other practices into traditional arts and other contexts through the use of soft wearables, e.g., the Singing Knit EMG wearable.
Analysis of how metaphors used in fundamental vocal pedagogy and how metaphorical communication between humans works; this proposed novel ways in which we can structure interaction with technology to aid in sensory communication.
Biofeedback-based reflections on ways in which vocalists are in control and controlled by their voices, work with and against their bodies.
Nuanced account of human interaction and perception of the body through vocal practice, as an example of how technological intervention enables exploration and influence over embodied understanding.
Implementing the VoxEMG into the Singing Knit with designs by Sophie Skach (top). The VoxEMG board can be affixed to textiles and use conductive thread inputs (left); the conductive thread traces to fabric electrodes are woven into the knit (centre), which stretches with the garment (right).
theory & methods
mixed-methods
electromyography (EMG) biofeedback
entanglement theory
agential realism
somaesthetics and soma design
cognitive science and music psychology
first-person methods & autoethnography
micro-phenomenology
thematic analysis
contemporary metaphor theory
vocal organology
The Singing Knit inside with conductive fabric electrode pads (top left) and outside with affixed VoxEMG boards for 8 channels of EMG data (top right). As modelled by me, the garment is able to stretch for flexible performance wear (bottom row).
related publications:
2024
Auditory imagery ability influences accuracy when singing with altered auditory feedback
Courtney N. Reed, Marcus Pearce, and Andrew McPherson
In this preliminary study, we explored the relationship between auditory imagery ability and the maintenance of tonal and temporal accuracy when singing and audiating with altered auditory feedback (AAF). Actively performing participants sang and audiated (sang mentally but not aloud) a self-selected piece in AAF conditions, including upward pitch-shifts and delayed auditory feedback (DAF), and with speech distraction. Participants with higher self-reported scores on the Bucknell Auditory Imagery Scale (BAIS) produced a tonal reference that was less disrupted by pitch shifts and speech distraction than musicians with lower scores. However, there was no observed effect of BAIS score on temporal deviation when singing with DAF. Auditory imagery ability was not related to the experience of having studied music theory formally, but was significantly related to the experience of performing. The significant effect of auditory imagery ability on tonal reference deviation remained even after partialling out the effect of experience of performing. The results indicate that auditory imagery ability plays a key role in maintaining an internal tonal center during singing but has at most a weak effect on temporal consistency. In this article, we outline future directions in understanding the multifaceted role of auditory imagery ability in singers’ accuracy and expression.
@article{Reed_MSX_AAFVocalAccuracy,title={{Auditory imagery ability influences accuracy when singing with altered auditory feedback}},author={Reed, Courtney N. and Pearce, Marcus and McPherson, Andrew},year={2024},month=feb,journal={Musicae Scientiae},publisher={SAGE Publications},volume={28},number={3},pages={478–501},doi={10.1177/10298649231223077},issn={2045-4147},url={http://dx.doi.org/10.1177/10298649231223077},}
2023
Negotiating Experience and Communicating Information Through Abstract Metaphor
Courtney N. Reed, Paul Strohmeier, and Andrew P. McPherson
In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Apr 2023
An implicit assumption in metaphor use is that it requires grounding in a familiar concept, prominently seen in the popular Desktop Metaphor. In human-to-human communication, however, abstract metaphors, without such grounding, are often used with great success. To understand when and why metaphors work, we present a case study of metaphor use in voice teaching. Voice educators must teach about subjective, sensory experiences and rely on abstract metaphor to express information about unseen and intangible processes inside the body. We present a thematic analysis of metaphor use by 12 voice teachers. We found that metaphor works not because of strong grounding in the familiar, but because of its ambiguity and flexibility, allowing shared understanding between individual lived experiences. We summarise our findings in a model of metaphor-based communication. This model can be used as an analysis tool within the existing taxonomies of metaphor in user interaction for better understanding why metaphor works in HCI. It can also be used as a design resource for thinking about metaphor use and abstracting metaphor strategies from both novel and existing designs.
@inproceedings{Reed_CHI23_VocalMetaphor,title={{Negotiating Experience and Communicating Information Through Abstract Metaphor}},author={Reed, Courtney N. and Strohmeier, Paul and McPherson, Andrew P.},year={2023},month=apr,booktitle={{Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems}},location={Hamburg, Germany},publisher={Association for Computing Machinery},address={New York, NY, USA},series={CHI '23},articleno={185},numpages={16},doi={10.1145/3544548.3580700},isbn={9781450394215},url={https://doi.org/10.1145/3544548.3580700},}
As the Luthiers Do: Designing with a Living, Growing, Changing Body-Material
Through soma-centric research, we see the different interaction roles of our bodies: they are the locus of our experience, a conduit for our expression and engagement, a sensor of feedback in the world, and a collaborator in our interaction with it. More" traditional" examinations of the body might look at control over it; for instance, in my research around vocal embodiment, I see many teachers and practitioners alike talking about how we can maintain control over the body. However, bodies are living, inconsistent, and typically weird. In reality, we do not have as much control over them as we would like or think we do. In this position paper, I will touch on my research around vocal physiology and sonified and vibrotactile feedback as I frame our role in a new light—designers as Body Luthiers, who must address the body as a material with inconsistencies, flaws, and variability, and work with it as a partner, embracing its uniqueness and changeability.
@inproceedings{Reed_CHI23_BodyLutherie,title={{As the Luthiers Do: Designing with a Living, Growing, Changing Body-Material}},author={Reed, Courtney N.},year={2023},month=apr,booktitle={{ACM CHI Workshop on Body X Materials}},location={Hamburg, Germany},}
The Body as Sound: Unpacking Vocal Embodiment through Auditory Biofeedback
Courtney N. Reed, and Andrew P. McPherson
In Proceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction, Feb 2023
Multi-sensory experiences underpin embodiment, whether with the body itself or technological extensions of it. Vocalists experience intensely personal embodiment, as vocalisation has few outwardly visible effects and kinaesthetic sensations occur largely within the body, rather than through external touch. We explored this embodiment using a probe which sonified laryngeal muscular movements and provided novel auditory feedback to two vocalists over a month-long period. Somatic and micro-phenomenological approaches revealed that the vocalists understand their physiology through its sound, rather than awareness of the muscular actions themselves. The feedback shaped the vocalists’ perceptions of their practice and revealed a desire for reassurance about exploration of one’s body when the body-as-sound understanding was disrupted. Vocalists experienced uncertainty and doubt without affirmation of perceived correctness. This research also suggests that technology is viewed as infallible and highlights expectations that exist about its ability to dictate success, even when we desire or intend to explore.
@inproceedings{Reed_TEI23_BodyAsSound,title={{The Body as Sound: Unpacking Vocal Embodiment through Auditory Biofeedback}},author={Reed, Courtney N. and McPherson, Andrew P.},year={2023},month=feb,booktitle={{Proceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction}},location={Warsaw, Poland},publisher={Association for Computing Machinery},address={New York, NY, USA},series={TEI '23},articleno={7},numpages={15},doi={10.1145/3569009.3572738},isbn={9781450399777},url={https://doi.org/10.1145/3569009.3572738},}
Imagining & Sensing: Understanding and Extending the Vocalist-Voice Relationship Through Biosignal Feedback
Courtney N. Reed
PhD Computer Science, Queen Mary University of London, Feb 2023
The voice is body and instrument. Third-person interpretation of the voice by listeners, vocal teachers, and digital agents is centred largely around audio feedback. For a vocalist, physical feedback from within the body provides an additional interaction. The vocalist’s understanding of their multi-sensory experiences is through tacit knowledge of the body. This knowledge is difficult to articulate, yet awareness and control of the body are innate. In the ever-increasing emergence of technology which quantifies or interprets physiological processes, we must remain conscious also of embodiment and human perception of these processes. Focusing on the vocalist-voice relationship, this thesis expands knowledge of human interaction and how technology influences our perception of our bodies. To unite these different perspectives in the vocal context, I draw on mixed methods from cognitive science, psychology, music information retrieval, and interactive system design. Objective methods such as vocal audio analysis provide a third-person observation. Subjective practices such as micro-phenomenology capture the experiential, first-person perspectives of the vocalists themselves. Quantitative-qualitative blend provides details not only on novel interaction, but also an understanding of how technology influences existing understanding of the body.
@phdthesis{Reed_PhD_ImaginingSensing,title={{Imagining & Sensing: Understanding and Extending the Vocalist-Voice Relationship Through Biosignal Feedback}},author={Reed, Courtney N.},year={2023},month=feb,school={PhD Computer Science, Queen Mary University of London},}
2022
Exploring Experiences with New Musical Instruments through Micro-phenomenology
Courtney N. Reed, Charlotte Nordmoen, Andrea Martelloni, and 6 more authors
In Proceedings of the International Conference on New Interfaces for Musical Expression, Jun 2022
This paper introduces micro-phenomenology, a research discipline for exploring and uncovering the structures of lived experience, as a beneficial methodology for studying and evaluating interactions with digital musical instruments. Compared to other subjective methods, micro-phenomenology evokes and returns one to the moment of experience, allowing access to dimensions and observations which may not be recalled in reflection alone. We present a case study of five microphenomenological interviews conducted with musicians about their experiences with existing digital musical instruments. The interviews reveal deep, clear descriptions of different modalities of synchronic moments in interaction, especially in tactile connections and bodily sensations. We highlight the elements of interaction captured in these interviews which would not have been revealed otherwise and the importance of these elements in researching perception, understanding, interaction, and performance with digital musical instruments.
@inproceedings{Reed_NIME22_Microphenomenology,title={{Exploring Experiences with New Musical Instruments through Micro-phenomenology}},author={Reed, Courtney N. and Nordmoen, Charlotte and Martelloni, Andrea and Lepri, Giacomo and Robson, Nicole and Zayas-Garin, Eevee and Cotton, Kelsey and Mice, Lia and McPherson, Andrew},year={2022},month=jun,booktitle={{Proceedings of the International Conference on New Interfaces for Musical Expression}},address={The University of Auckland, New Zealand},articleno={49},doi={10.21428/92fbeb44.b304e4b1},issn={2220-4806},url={https://doi.org/10.21428%2F92fbeb44.b304e4b1},presentation-video={https://youtu.be/-Ket6l90S8I},}
Communicating Across Bodies in the Voice Lesson
Courtney N. Reed
In ACM CHI Workshop on Tangible Interaction for Well-Being, Apr 2022
In this position paper, I would like to introduce my research on vocalists and their relationships with their bodies, and how the use of haptic feedback can improve these connections and the way we communicate sensory experience. I use the voice lesson and vocal performance as an environment to understand more broadly how people perceive very refined movements which they feel internally. My research seeks to understand how we communicate these sensory experiences in human-to-human interaction and how we can augment or communicate sensory experience through technology. I examine perception of these experiences through different feedback modalities, namely auditory and haptic feedback. Providing new ways to communicate our sensory experiences can lead to improvements in understanding between two individuals (for instance teacher and student). In virtual singing lessons, where the majority of voice study is being done in early 2022, this is especially important, as many of the common ways of interacting with the voice have disappeared with the transition to online interaction.
@inproceedings{Reed_CHI22_CommunicatingBodies,title={{Communicating Across Bodies in the Voice Lesson}},author={Reed, Courtney N.},year={2022},month=apr,booktitle={{ACM CHI Workshop on Tangible Interaction for Well-Being}},location={New Orleans, LA, USA},}
Sensory Sketching for Singers
Courtney N. Reed
In ACM CHI Workshop on Sketching Across the Senses, Apr 2022
This position paper outlines my study of vocalists and the relationships with the voice as both instrument and part of the body. I study this embodiment through a phenomenological perspective, employing somaesthetics and micro-phenomenology to explore the tacit relationships that singers have with their body. While verbal metaphor is traditionally used to articulate experience in teaching voice, I also use body mapping and material speculation to help articulate tactile and auditory experiences while singing.
@inproceedings{Reed_CHI22_SensorySketching,title={{Sensory Sketching for Singers}},author={Reed, Courtney N.},year={2022},month=apr,booktitle={{ACM CHI Workshop on Sketching Across the Senses}},location={New Orleans, LA, USA},}
Singing Knit: Soft Knit Biosensing for Augmenting Vocal Performances
Courtney N. Reed, Sophie Skach, Paul Strohmeier, and 1 more author
In Proceedings of the Augmented Humans International Conference 2022, Mar 2022
This paper discusses the design of the Singing Knit, a wearable knit collar for measuring a singer’s vocal interactions through surface electromyography. We improve the ease and comfort of multi-electrode bio-sensing systems by adapting knit e-textile methods. The goal of the design was to preserve the capabilities of rigid electrode sensing while addressing its shortcomings, focusing on comfort and reliability during extended wear, practicality and convenience for performance settings, and aesthetic value. We use conductive, silver-plated nylon jersey fabric electrodes in a full rib knit accessory for sensing laryngeal muscular activation. We discuss the iterative design and the material decision-making process as a method for building integrated soft-sensing wearable systems for similar settings. Additionally, we discuss how the design choices through the construction process reflect its use in a musical performance context.
@inproceedings{Reed_AHs22_SingingKnit,title={{Singing Knit: Soft Knit Biosensing for Augmenting Vocal Performances}},author={Reed, Courtney N. and Skach, Sophie and Strohmeier, Paul and McPherson, Andrew P.},year={2022},month=mar,booktitle={{Proceedings of the Augmented Humans International Conference 2022}},location={Kashiwa, Chiba, Japan},publisher={Association for Computing Machinery},address={New York, NY, USA},series={AHs '22},pages={170–183},numpages={14},doi={10.1145/3519391.3519412},isbn={9781450396325},url={https://doi.org/10.1145/3519391.3519412},}
Examining Embodied Sensation and Perception in Singing
Courtney N. Reed
In Proceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction, Feb 2022
This paper introduces my PhD research on the relationship which vocalists have with their voice. The voice, both instrument and body, provides a unique perspective to examine embodied practice. The interaction with the voice is largely without a physical interface and it is difficult to describe the sensation of singing; however, voice pedagogy has been successful at using metaphor to communicate sensory experience between student and teacher. I examine the voice through several different perspectives, including experiential, physiological, and communicative interactions, and explore how we convey sensations in voice pedagogy and how perception of the body is shaped through experience living in it. Further, through externalising internal movement using sonified surface electromyography, I aim to give presence to aspects of vocal movement which have become subconscious or automatic. The findings of this PhD will provide understanding of how we perceive the experience of living within the body and perform through using the body as an instrument.
@inproceedings{Reed_TEI22_EmbodiedSingingDC,title={{Examining Embodied Sensation and Perception in Singing}},author={Reed, Courtney N.},year={2022},month=feb,booktitle={{Proceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction}},location={Daejeon, Republic of Korea},publisher={Association for Computing Machinery},address={New York, NY, USA},series={TEI '22},articleno={47},numpages={7},doi={10.1145/3490149.3503581},isbn={9781450391474},url={https://doi.org/10.1145/3490149.3503581},}
2021
Surface Electromyography for Sensing Performance Intention and Musical Imagery in Vocalists
Courtney N. Reed, and Andrew P. McPherson
In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, Feb 2021
Through experience, the techniques used by professional vocalists become highly ingrained and much of the fine muscular control needed for healthy singing is executed using well-refined mental imagery. In this paper, we provide a method for observing intention and embodied practice using surface electromyography (sEMG) to detect muscular activation, in particular with the laryngeal muscles. Through sensing the electrical neural impulses causing muscular contraction, sEMG provides a unique measurement of user intention, where other sensors reflect the results of movement. In this way, we are able to measure movement in preparation, vocalised singing, and in the use of imagery during mental rehearsal where no sound is produced. We present a circuit developed for use with the low voltage activations of the laryngeal muscles; in sonification of these activations, we further provide feedback for vocalists to investigate and experiment with their own intuitive movements and intentions for creative vocal practice.
@inproceedings{Reed_TEI21_sEMGPerformance,title={{Surface Electromyography for Sensing Performance Intention and Musical Imagery in Vocalists}},author={Reed, Courtney N. and McPherson, Andrew P.},year={2021},month=feb,booktitle={{Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction}},location={Salzburg, Austria},publisher={Association for Computing Machinery},address={New York, NY, USA},series={TEI '21},articleno={22},numpages={11},doi={10.1145/3430524.3440641},isbn={9781450382137},url={https://doi.org/10.1145/3430524.3440641},}
2020
Surface Electromyography for Direct Vocal Control
Courtney N. Reed, and Andrew McPherson
In Proceedings of the International Conference on New Interfaces for Musical Expression, Jul 2020
This paper introduces a new method for direct control using the voice via measurement of vocal muscular activation with surface electromyography (sEMG). Digital musical interfaces based on the voice have typically used indirect control, in which features extracted from audio signals control the parameters of sound generation, for example in audio to MIDI controllers. By contrast, focusing on the musculature of the singing voice allows direct muscular control, or alternatively, combined direct and indirect control in an augmented vocal instrument. In this way we aim to both preserve the intimate relationship a vocalist has with their instrument and key timbral and stylistic characteristics of the voice while expanding its sonic capabilities. This paper discusses other digital instruments which effectively utilise a combination of indirect and direct control as well as a history of controllers involving the voice. Subsequently, a new method of direct control from physiological aspects of singing through sEMG and its capabilities are discussed. Future developments of the system are further outlined along with usage in performance studies, interactive live vocal performance, and educational and practice tools.
@inproceedings{Reed_NIME20_VocalsEMG,title={{Surface Electromyography for Direct Vocal Control}},author={Reed, Courtney N. and McPherson, Andrew},year={2020},month=jul,booktitle={{Proceedings of the International Conference on New Interfaces for Musical Expression}},publisher={Birmingham City University},address={Birmingham, UK},pages={458--463},doi={10.5281/zenodo.4813475},issn={2220-4806},url={https://www.nime.org/proceedings/2020/nime2020_paper88.pdf},editor={Michon, Romain and Schroeder, Franziska},presentation-video={https://youtu.be/1nWLgQGNh0g},}
