Empowering the Next Generation of Women in Audio

Join Us

Making Sonic Magic from Auditory Illusions

So, if a tree falls in the woods, perhaps it makes infinite sounds

A few years ago, I attended a talk on wave field synthesis, and to say I was captivated feels like a sorry understatement. Wave field synthesis, if you are unfamiliar with it as I was, is a spatial auditory illusion and rendering technique that produces a holophone, or auditory hologram, using many individually driven loudspeakers. The effect is that sounds appear to be coming from a virtual source and a listener’s perception of the source remains the same regardless of their position in the room. Its application in theatrical contexts is very new, but as the techniques and technology slowly become more widely available, the potential for theatrical applications is astounding.

This introduction to wave field synthesis, in addition to being quite exciting, pointed me towards a categorical lack of knowledge about auditory illusions. Since then, I’ve been filling in the gaps and adding these illusions to my sonic toolbox. Now, quite a bit of theatrical sound design could be considered spatial illusions like, for example, when we recreate actual physical phenomena like the doppler effect. Auditory illusions, however, encapsulate many effects extending far beyond this.

Optical illusions have long been the inspiration for and integrated into visual arts. M.C. Esher’s work, for instance, presents the viewer with impossible objects and perceptual confusions. In psychology and neurology, the study of optical illusions has played a large role in understanding the visual perception apparatus. Due to the historical ease of reproducing and distributing visual material, as opposed to auditory material, visual illusions have long been widely encountered, studied, and applied in artistic works. The history of auditory illusions and their use in psychology, music, sound design, and elsewhere is much shorter.

Auditory illusions, much like visual illusions, reveal the deficiencies and oddities of our perceptual processes, but the auditory and visual systems have their own unique attributes. The field of psychoacoustics examines how the brain processes sound, music, and speech. Hearing is not strictly mechanical but involves significant neural processing and is influenced by our anatomy, physiology, and cognition. Researches have even found that how we unconsciously interpret sounds is influenced by our individual environments, backgrounds, and dialects. Auditory illusions provide key information in unpacking our auditory processes for psychologists and neurologists. In artistic applications, auditory illusions provide similar insight into our perceptional processes and illustrate that there is no one true sonic reality.

Dr. Diana Deutsch, a psychologist at the University of California, San Diego is at the forefront of psychoacoustic research and her work has utilized countless auditory illusions and sonic paradoxes. If you want to hear examples and read her work, visit her website here: http://dianadeutsch.com/. Due largely to her research, there has been increasing understanding of the cognitive factors in the auditory system and how it has evolved over time to help us interpret our sonic environments effectively. Psychoacoustic research has been applied in myriad contexts including modeling compression codecs like mp3s, software development, audio system design, drone flying, car manufacturing, and even, terrifyingly, in acoustic weapon development. In the arts, psychoacoustics and auditory illusions have been applied in musical contexts, sound art, film, and theater, though these applications are fairly nascent.

There are a number of types of illusions that can be roughly categorized as spatial illusions, perpetual motion, and non-linear perceptual effects. More auditory illusions continue to be uncovered and understood, so these categories aren’t rigid. Spatial illusions are already a mainstay of theatrical sound design. We frequently manipulate spatialization to make it seem as though sounds are coming from a particular source or direction other than the loudspeaker producing the sound. Holophones can be created in a number of ways including wave field synthesis as I’ve mentioned. Binaural recording is another example of spatial manipulation, reproducing interaural features and anatomical influences of the head and ear. All of these spatial illusions exemplify a distinction between the physical properties of the sound field and the perception of what listeners actually hear.

Unreal sounds created in the inner ear or brain are a part of our daily lives that we typically don’t notice, and there are several auditory illusions that mirror common visual illusions. A Zwicker Tone, for example, is the sonic equivalent of an after image. Illusions of Auditory Continuity show us that when an acoustic sound signal is momentarily cut off and replaced by another sound, listeners perceive the original signal to continue through the interruption. Through the familiar Precedence or Haas Effect, we perceive a singular sonic event when one sound is followed by another with a short delay time, and that we ascribe directionality based on the first arriving sound. While subtle, these are all valuable design techniques.

Less subtle are perpetual motion illusions. Pitch and tempo circularity is roughly analogous to the barber pole illusion in which a sound seems to be endlessly ascending or descending or a rhythm seems to be endlessly increasing or decreasing in tempo. Both pitch and tempo circularity encapsulate a number of techniques and effects. The Risset Rhythm and Shepard Tone are complex versions. The Shepard Tone most notably influenced the film score for Dunkirk and created a palpable sense of anxiety. Much like Esher’s impossible stairs, circularity illusions are both unsettling and entrancing, a powerful design technique.

There are a number of speech-related auditory illusions. Most famously, the Laurel/Yanny internet phenomenon of 2018 brought speech interpretation illusions into the spotlight. It also demonstrated the incredible subjectivity of our hearing. Similarly, The McGurk Effect presents a puzzling phenomenon in the interaction of vision and speech. When a visual component of a person mouthing a sound is paired with a different sound, listeners perceive neither of the two sounds, but instead a third sound.

Dr. Deutsch has amassed an immense number of Stereophonic Illusions including Phantom Words, Binaural Beats, the Glissando Illusion, the Octave Illusion, the Scale Illusion, the Tritone Paradox, and more. Her work shows us how differently people perceive the same sounds. When we listen to speech the words we perceive are influenced by our expectations, knowledge, dialect, and culture, in addition to the physical sounds we hear. Much of her work has also demonstrated how left and right-handedness influences how complex sounds are synthesized and localized in our heads. In the Tritone Paradox, utilizing sequentially played Shepard tones a tritone apart, some listeners hear the tone ascending while others hear it descending. The potential for designing sounds in which some of the audience experiences the inverse of what others experience is, to me at least, a riveting notion.

While this is brief overview is the tip of the ever-expanding metaphorical iceberg of auditory illusions, I have found that looking into psychoacoustics and auditory neurology provides incredible design techniques and ideas that are not always at our disposal. The potential here that I’m so excited about, is to create audience experiences that rouse questions about the subjectivity of their perceptions of the world around them. Audiences can leave the theater not believing their ears. It also illuminates a greater need for interdisciplinary collaboration and cooperation between fields that often feel disparate: psychology, neurology, audiology, engineering, music, sound design, etc. In my own work, I have yet to utilize almost any of this material (with the exception of spatialization techniques, of course), but it is leading me to think about designing for the whole head, the ear, the brain, and the mind. I so look forward to the continued integration of auditory illusions in theatrical designs, creating sound magic.

 

 

On Aesthetic Distance & Virtual Live Performance

 

In a theater, sound design is a mode of transportation into the world of the story. As we adapt to creating virtual live performances, however, we are up against greater challenges in effectively inviting the audience into the worlds we are building. In the virtual space, live performances are flattened – both literally and figuratively. So, the question becomes, how do we translate live theatrical performances to the virtual space in a way that offers a satisfying experience comparable to that of in-person theater? How do we address the lack of sensory immersion and narrow the gap in the aesthetic distance?

The X factor at home is huge and complicated – far beyond the question of what sort of headphones or speakers are used. How is the audience watching the show? Have they set everything up on their TV, lights dim, no distractions or are they balancing a tablet on the kitchen counter while they cook dinner, their kids watch cartoons, dog barks, and upstairs neighbors do jazzercise? Background noise and viewing setup aside, it’s more difficult to maintain focus at home. Distractions are everywhere, the level of commitment is much lower, and the energetic benefits of being in the room with other people are just not available. All of these issues prevent sensory immersion, widen the aesthetic distance, and impede audiences’ ability and willingness to suspend their disbelief.

In order to really transport our audiences without the control we typically have in a theater, we have to shift our approach. Narrative Transportation Theory (in the context of storytelling) suggests that reaching the state of complete mental absorption in a narrative hinges on two key ingredients: empathy and imagination. As sound designers, we certainly have tools to facilitate empathy along with the work of the text, the actors, fellow designers, etc. But imagination might be the most critical ingredient to pay attention to right now. How are we inviting our audiences to imagine?

In trying to convert our 3D medium to a 2D medium online, it makes sense initially to emulate television and film. And our audiences are already accustomed to enjoying television and film from the comfort of their own homes. However, virtual live theater is not television and it needn’t be. With incredibly tight budgets, the constraints of streaming platforms and accessibility to those platforms, achieving a production level comparable to that of a tv show is a maddening and often disappointing challenge. Beyond the production logistics, there is the issue of “Zoom fatigue.”

Over the past few months, I’ve been meditating on what variation of theater I would like to experience and how I would like to experience it. I desperately want to be transported and lose myself to a story while I’m trapped in my tiny apartment in real life. At the same time, I stare at my computer all day long and the idea of staring some more just for fun seems, well, not fun. I’ve also found it difficult to become immersed in virtual programming as I’m hyper-aware of virtual backgrounds, delays, all of the same things we grapple with behind the scenes. The risk of these elements is that they can stifle imagination rather than ignite it.

At first, my focus was on all of the incredible technology in our hands what innovative things we can do with it from our own homes. Lately, I’ve turned in the opposite direction. More specifically, towards radio plays. These are bare-bones times in need of bare-bones productions. Taking cues from radio plays along with gripping low-budget productions of any sort, memorable black box performances, anything that makes way for a “less is more” sort of approach is worthwhile. Right now, less is more. We certainly have less to work with, but we also need to allow more space for audiences to use their imagination and hopefully be transported into the story. Some of the designs may be best left to the mind’s eye or mind’s ear. We can perhaps benefit from paying special attention to what not to include. And, of course, I am biased towards sound (and would love to give my eyes a break), but I think this idea can apply more widely. In order to close the gap in the aesthetic distance, we have to craft a path towards the imagination.

Now, I am not advocating a strict return to radio plays. Nor am I advocating that we necessarily turn to sound-only media. In thinking about the process, however, live radio plays are a suitable jumping-off point. I certainly don’t have the answer to creating effective virtual productions and I can’t offer prescriptive advice or best practices as every show is completely individual. I do think, though, that there are some elements of a live radio play and its process that can enhance or at least refresh our approach in the virtual space.

Trying to capture attention by increasing stimuli can be more exhausting right now. Stripping a production down, counterintuitively, may offer a better opportunity for immersion. Knowing the challenges that we are up against, considering first what elements need to be presented and which can be forgone is a good first step to streamlining. Do we really need to hear that door close? Is it essential to see that room? As sound designers, we are already cognizant of silence as a strategic part of our design. What we do not hear is equally as important as what we do. We can expand this line of thinking to create more space in a production. What can be strategically left to the imagination? Then, keeping the two-dimensionality of the final product in mind, we can critically assess how each element is best portrayed, whether aurally, visually, as dialogue. The question being, what is the best way to provoke the mind’s ear or mind’s eye to complete the picture?

Lowering the complexity of virtual performances, in addition to making room for enhanced imagination, also allows for more focus and energy within the production. With fewer distractions and fewer opportunities for latency and glitches, actors and audiences alike have more space to relax and concentrate. The energy of live performance is one of the things that makes theater so compelling. When actors are in a flow state, completely absorbed in their characters and the story, all of their energy carries directly to the audience. With a stripped-down design, we can protect that precious energy.

Every show, every production can be approached, produced, designed in infinite ways, none of them right or wrong. As we all wrestle with creating live virtual theater, its limitations and possibilities, it is important to take a step back and re-evaluate our own personal design processes. For me, this has been a helpful way of reframing my work and my goals for virtual performances. It’s my hope that some of these ideas can at least come in handy when a virtual design hits a virtual wall. And when in need of some inspiration, go ahead and close your eyes and listen to some radio plays.


Abigail Nover is a sound designer and composer based out of Miami, Florida. She works as a freelance designer for theatrical productions in English and Spanish throughout the country. She holds a BFA in Sound Design from Carnegie Mellon University School of Drama and an MA in Folklore from the University of North Carolina at Chapel Hill. Her work is often rooted in cultural memory and immersion. In addition to theatrical work, Abigail conducts oral histories and writes about cultural and sound studies. She is a member of the OISTAT Sound Design Group.

 

Looking Back on Changes by Charles Bradley 

 

When Charles Bradley’s final album, Changes, was released in April of 2016, I had it on repeat for months and months. It felt prophetic. As the political and social landscape became increasingly turbulent, the album provided me with some sort of a grounding force. If we think of the months leading up to the 2016 election as the beginning of the current moment, then now we are at the next inflection point, and over the past few violent and difficult months, I have been returning to Bradley’s Changes. It is just as relevant and necessary as ever.

Immediately remarkable about the album is its footing as pure, unadulterated soul. Not neo-soul, not retro-soul, but capital S Soul. Changes didn’t merely draw upon the ‘60s sound, it seamlessly brought the genre – and all of the traditions that go along with it – into 2016. Daptone’s Thomas Brenneck produced the album in a minimalist fashion, with very few special effects, but he spared nothing when it came to backing musicians. Most of the songs feature musicians from The Menahan Street Band, a few are backed by the Budos Band, and The Gospel Queens are featured on two songs. Changes is laced with organ trills, horn jabs, an almost psychedelic bass, and on several tracks, you can hear people partying along with Bradley in the background. Each track pays homage to ‘60s soul in its instrumentation without simply recreating the ‘60s Stax/Atlantic sound or relying on nostalgia. The album is Soul at its core, but the sound is distinctly Bradley.

Central to every song is Bradley’s voice. When he starts to get going, it’s like sitting in a motorboat while someone revs the engine, rough and hearty, and then gliding away on the water. His voice is what earned him the nickname “The Screaming Eagle of Soul.” This was his third record with Daptone, and while his characteristically roaring voice has always held fast, Changes stands apart. By far his most cohesive album, this one is personal to Bradley. At 67, the former James Brown impersonator, known for his high energy performances and astounding vocals, solidified his individual voice and sang from a profoundly personal perspective. In doing so, he deviated from a sound sternly reverent to the ‘60s soul aesthetic. Between Bradley’s voice and his emotionally provocative lyrics, he reignites a genre exactly when we need it most.

Charles Bradley sings right to the soul: the soul of the listener and the soul of the country. Soul grew in tandem with the Civil Rights Movement and the two are closely entwined. Bradley continues this legacy, beginning the album with a rendition of “God Bless America,” which he sings as a love song, directly addressing America, the country that’s caused him much pain and grief. He opens the track by introducing himself:

“Hello, this is Charles Bradley
A brother that came from the hard licks of life
That knows that America is my home
America, you’ve been real, honest, hard, and sweet to me
But I wouldn’t change it for the world
Just know that all the pains that I’ve been through,
It made me strong,
To stand strong, that know
America represents love for all humanity and the world

I say from my heart
God Bless America
My home sweet home”

In the heat of police violence, Black Lives Matter, and an ever-deepening racial divide in 2016, Bradley tackles what it means to be a Black person in the United States head-on as “God Bless America” rolls right into “Good to be Back at Home.” He sings about America in the same terms as he has sung about tumultuous romantic relationships.

In “Change for the World,” Bradley invokes a rallying call to action. “Hate is poison in the blood/ Heaven is cryin’, the world is shakin’/ God is unhappy, the moon is breakin’/ Blood is spillin’, God is comin’.” He urges listeners to “Put away the guns and take this love.” With the reverb and delay on his voice in this track, it sounds like Bradley is imploring us to change from on high. His tone is powerful yet optimistic and carries through the whole album.

The title song, “Changes” is, as you may have guessed, a cover of the Black Sabbath song. This surprising rendition of the Ozzy Ozbourne classic is nothing but awe-inspiring. Kevin Young in The Gray Album notes the long-standing tradition of Black soul artists covering white popular music, like Isaac Hayes’ cover of Burt Bacharach’s “Walk on By” or Curtis Mayfield’s cover of The Carpenter’s “We’ve Only Just Begun. Young writes that soul is “a black means of transforming material, … to return to the blues tone of the music.” Bradley does just this as he reimagines the song which was originally about an ex-lover to be about his late mother, to whom the album is dedicated. It is her memory and wisdom that he seeks to imbue throughout this album. With or without that knowledge, his rough voice on this track can summon tears.

After Changes was released in 2016, the radical optimism that saturates the album felt necessary to get by and to summon the energy needed to take action. In the current moment, optimism feels inappropriate, insensitive, and nearly impossible. Bradley’s optimism, however, is not one rooted in ignorance (willful or otherwise), but rather in an acute understanding of the issues facing this country that need to be confronted. The album creates the space to acknowledge silver linings and small victories while still bearing witness to violence and cruelty. Listening and re-listening to Bradley’s 2016 brand of radical optimism in Changes can help in regaining energy, envisioning and working towards the just future that Bradley summons in the album. Bradley died in 2017, leaving Changes as his greatest legacy which, I believe, will serve us well into the future as it’s served us these past four years. As one of the most fraught years in recent memory turns into another and with so much uncertainty about what lies ahead, it is worth re-listening and considering Bradley’s voice and vision as a means of solace, rest, and reinvigoration. Works Consulted 

Charles Bradley. Changes. Daptone Records, 2016, CD.

“Charles Bradley.” Daptone Records. 2013. Accessed October 02, 2016. https://daptonerecords.com/charles-bradley/.

Young, Kevin. “Chorus Four: Moanin’: Soul Music and the Power of Pleasure.” Essay. In The Grey Album: on the Blackness of Blackness, 249–73. Minneapolis, MN: Graywolf Press, 2012.


 

Abigail Nover is a sound designer and composer based out of Miami, Florida. She works as a freelance designer for theatrical productions in English and Spanish throughout the country. She holds a BFA in Sound Design from Carnegie Mellon University School of Drama and an MA in Folklore from the University of North Carolina at Chapel Hill. Her work is often rooted in cultural memory and immersion. In addition to theatrical work, Abigail conducts oral histories and writes about cultural and sound studies. She is a member of the OISTAT Sound Design Group.

Read Abigail’s Blog

X