Before audio post-production was even a possibility, composers would incorporate the emotion and the action of what they saw on the screen into their musical scores. They played the role of sound effects editor and composer, with a technique referred to as “Mickey-Mousing” where the composer would exaggerate a character’s movements with specific orchestration and musical motifs. Now that sound FX editors have taken this role in post-production, Mickey-Mousing is less common, so it’s key for sound effects editors to make cuts that work with the music in the overall mix for a film or tv show. Here are some considerations and tips that our team shared on their approach to sound editing with music in mind.
Consider the musicality of chosen sound elements for a build
“When designing sound effects of a musical nature, it’s very important to steer clear of anything with a defined pitch. It’s especially important to avoid any chords, whether arpeggiated (like an upward harp gliss) or played together (like a synth chord used as a steady for a magic beam). There is very little chance that you will happen to choose the same key and chord as the score, so most likely, these elements will need to be muted as soon as the music is added to the mix.
To avoid this, I always choose to use inharmonic instruments, such as chimes, cymbal, water-phone, etc., when I want to add a musical element. Their non-integer-multiple harmonics keep them from sounding like any particular pitch, which in turn keeps them from interfering with the tonality of the music. If you absolutely need to use a musical element, always be sure that you have a non-tonal backup element in place. That way, if your star element is muted in the mix, there will be something left to cover the action.”
– Kate Finan, MPSE
Here is a depiction of how harmonics work in periodic waveforms heard in music and tonal sound FX.
Think about the role that the score will play in the final mix
“Oftentimes, if the guide track provided to us for our sound effects edit includes music, it’s an indication that it’s an important musical moment in the show. This could be a montage or a song sequence. Knowing that music will take center stage helps us think about the sequence in terms of how sound effects will support the music. This may mean cutting less and focusing on what will cut through in the final mix. Or for a song, we may want to cheat some of our effects so they land on the beat and work seamlessly with the music.”
– Jeff Shiffman, MPSE
Be confident in bold builds when a scene shares heavy sound EFFECTS with the music
“It’s important to cut sound effects that are substantial enough that they’ll cut through in the mix once music is added. A lot of the time, music drives the emotion of the scene and therefore is fairly present in the mix, so if you cut a sound effect that is extremely subtle, it will very likely get lost. Make your choices clear and significant enough to not get buried.”
– Tess Fournier, MPSE
Focus on texture to support tonality
“Try to avoid using sound effects that have a musicality to them. You never know if something with a melody or tune will clash with what is going on in the score. This comes up a lot with things like magic effects, so I always try to go textural and nondescript, rather than musical or tonal.”
– Brad Meyer, MPSE
Notice how you’re using the full frequency spectrum
“During songs or music-driving scenes, make sound effect builds that are a bit more punchy and larger than normal scenes so it can cut through the score better if needed. That way your work won’t be lost in the mix!”
– Ian Howard, Re-Recording Mixer
This diagram illustrates the frequency spectrum as it relates to common musical instruments and the qualities that audio editors and mixers use to describe certain frequency ranges.
Always benefit the progression of the story
“When mixing, remember that it is your job to help create harmony between the sound effects and music coverage. In some cases, both parties will cover a moment or action in a similar way, and it’s essential to figure out whether music, sound effects, or a combination of the two will best serve the story.
Along the same lines, during songs or key musical moments, you can really enhance a scene by adjusting the sync of certain sound effects to hit on the beats of the music. This is especially powerful during title sequences and music video moments, where the music is driving the story.”
– Jacob Cook, Re-Recording Mixer
The bottom line is that there are multiple ways for sound effects to work with music in any given scene with considerations like atonality, sync, texture, and frequency spectrum. Keeping this in mind allows for a story to shine through action and emotion.