Applying film post-production techniques to game sound design

Author: Nick Peck
It has been my experience that game companies often rely on commerical CD sound
effects libraries for the majority of their raw sound material. While these libraries are very
useful, there are other methods of collecting sound material that can achieve excellent
creative results. The film industry often uses custom field and foley recording to give each
project a personal and unique flavor, augmenting their needs with CD libraries where
appropriate. In this session, techniques of field and foley recording will be discussed, using
examples and parallels between the film Being John Malkovich and the game Escape from
Monkey Island. Audio portions of the game will be broken down track by track, showing how
the dialog, music, hard sfx, foley, and ambient layers combine to create a unified sound
experience.
Let’s whet our appetite by hearing some great film sound. Here are snippets from three very
different films with excellent sound: Apocalypse Now, The Exorcist, and Castaway.
Our challenge is to meet this level. Games don’t sound like that. Even taking into account
the differences in the medium, games often don’t sound as good as they could. Why not?
Four reasons immediately come to mind: time, money, communication and delay across the
team, and accepted work techniques.
How can we improve the situation?
Not having enough time or money are always a huge problem. The solution is to budget
more of both for sound! Sound Designers are often limited by having poor, outdated
equipment, not enough off-the-shelf sound libraries, but most importantly, not enough time
to go out and get new, original sounds for the game project. Remember: SOUND IS ART.
To make a game sound artful, let the sound designers have the time and money to practice
their art!
The next problem is poor communication with the rest of the team, and delays in production
propogating to further diminish the amount of time to develop sound for the game. We can
say that effluent follows the laws of gravity in changing state from higher potential energy to
lower potential energy. The translation will be left to the reader, but the point is that sound is
a post-production process. We are at the end of the line, where everyone is out of money,
out of time, and out of patience. This is true in film as well. By the time sound is done, the
programmers are burned out, the deadlines are absurdly close, and it is very hard to get the
sound wired up with the level of detail you’d like. You can address this somewhat by clearly
communicating your needs early on. If all else fails, become a programmer.
The final problem limiting sound production in games that I’d like to touch on is accepted
work techniques. It seems that pulling most or all raw sound materials from commercial
SFX libraries is often the primary approach. It is a model that is well-understood, and easy
to implement. While libraries are hugely useful, though, they do limit your creativity, and
give you the same raw sound as everyone else. It is true that the crafty sound designer will
take these materials as a starting point and manipulate them, often to the point of
unrecognizability, but there are still only so many wind recordings in the Sound Ideas 6000
series library, and most every game company owns that library and uses it.
To some degree, these problems will always be there. Bringing awareness of sound needs
to the people that pay the bills is not always easy. But there are ways that we can improve
game audio incrementally: by bringing more film post-production techniques into game
audio.
Why apply film techniques to games? Simply put, the movie industry has been around for a
long time. Film sound designers have honed their craft and figured out what works. They
know how to make films sound unique and interesting. As game sound designers, we can
steal their ideas to make games sound unique and interesting too.
Film sound is broken into a series of layers: dialog, music, hard SFX, foley, and ambience.
Let’s examine these each rather briefly, looking at their relation to the greater whole.
Dialog comes first. Always remember that dialog is king. It must be intelligible above all
else, or your story is lost. In game audio, this usually means that the dialog is compressed
and limited severely, to make sure it reads above the music and SFX within the limited
dynamic range we have to work with. Film dialog is not compressed as much, because the
sound is carefully massaged at the mixing stage to make sure of intelligibility.
The music sets the emotional context of the project. It tells the player what to feel, whether
a moment is placid or tense, majestic or scary. Music and sound effects share the same
space, and work together in it (or not). Both film and games have the same problem of
these elements competing with each other. The best compromise is to try to make both
audible. This can be a tight-rope act, particularly in interactive settings. I have found that
having greater dynamic range, particularly in the music, allows the elements to rise and fall
in audibility, poking through each other when appropriate.
Hard SFX are the meat and potatoes of game sound. These are spells, weapon hits,
engine loops, door slams, and all other foreground sound material. The sonic character of
the game is most strongly defined by the choices the sound designer makes in creating the
hard SFX.
Foley is sound made by humans: footsteps, clothing rustles, the manipulation of props and
tools. In film, foley is recorded to picture to cover the movements the characters made on
the screen that could not be picked up by production microphones, due to the noise on the
set. Games are primarily animated, so of course there is no production recording to be
used, and all movements are recorded after the fact. Or more often, not recorded at all.
Foley is the sound layer that brings subtle realism to film. We can bring it to games as well.
Practically by definition, foley work involves recording custom material every time. There
are limited footsteps and clothing rustles available in some SFX libraries, that are often
used by game sound designers to fill in some movement. In my experience, foley recording
sounds better than using canned material, and is cheaper than editing it as well. It actually
takes less time to walk footsteps against picture than to edit library footsteps against it. Of
course, foley recording does require a foley pit with multiple surfaces for different types of
footsteps, as well as a good-sized prop collection and an extremely quiet recording
environment. But in my opinion, any game with a reasonably sized budget should do at
least some custom foley recording.
The last layer of sound is ambience. Ambience is the background recording of a particular
place that identifies it aurally. Swamp ambiences are filled with birds and frogs, beach
ambiences have the endless rumble of waves crashing on the shore, cave ambiences
might have a slow, reverberant dripping of water, restaurant ambiences might have muffled
conversation and the clatter of silverware on plates, and factory ambiences would have low
rumbling and the clatter of huge machines in the background. If music sets the mood,
ambience brings the location to life.
Ambiences have two components: The ambient loop, which is a long, streaming, stereo
recording that can be mixed with the music track, and specifics, which are separate, short
elements (bird chirps, foghorns, etc) that trigger randomly to break up repetition.
The way to bring convincing ambiences into your game is through field recording. Portable
DAT machines and stereo microphones expand your horizons to the end of the Earth. Field
recording is great fun, and rewards you with original material that has never been used in a
game before. As an added bonus, it is a great way to get out of the office for a while. I went
on vacation to Canada right before beginning production on Escape from Monkey Island. I
took the opportunity to record every type of water setting I could, from every distance and at
different times of day: waterfalls, beaches, gentle harbors, and streams. All of this material
made it into Monkey, and the result is a rich aural environment.
I’d like to show an example of how these elements fit together in film by using a scene I
sound designed for the film Being John Malkovich. In this scene, Malkovich enters his own
mind and ends up in a restaurant, where everyone he sees is a version of himself. I’ll show
how foley, hard sfx, ambience, music, and dialog fit together to create a complete picture.
Games have two different types of segments: Linear segments, often called animations or
cutscenes, and interactive segments. Each of these can benefit from a filmic approach.
Linear segments are short, animated movies with no interactive elements. The approach is
clear: Make sure to create all the layers of sound described above to create a rich
experience.
I’d like to show the GMRR (Giant Monkey Robot) cutscene from Escape from Monkey
Island as a case study. Just as in the restaurant scene from Being John Malkovich, I will
play back the scene several times, soloing the ambient, foley, hard sfx, and music tracks
separately. I will then play back a mix, to show how all the elements fit together.
Interactive segments are, of course, the part of the game where the player is actually
making things happen. Events are not completely predictable, and take place as a result of
the player’s decisions. Clearly, the interactive portions of games are very different animals
than linear film.
But there are still concepts to steal that can improve these segments.
Filmic improvements to interactive game sound would include: Filling the environment with
ambient loops and specifics, minimizing repetition by having as many alternate SFX as they
will let you (especially for footsteps and weapon hits), and letting the soundtrack have some
dynamics. Don’t compress/limit the life out of everything! Finally, to repeat my main theme,
do as much custom recording as possible! Make new footsteps, grunts, hits, weapons fires,
UI clicks, and anything else you can. It will sound different than other games, and will be a
lot more fun as well.
As a case study in interactive segments, I’d like to look at the sushi boat puzzle from
Escape from Monkey Island. This complex puzzle involves a lot of careful timing and
thinking outside the box by the player. There are various mechanisms at work, each with
their own sounds. By carefully working with these different sounds, turning them on and off
and changing their volume relative to decisions the player makes, the audio soundtrack
enhances the puzzle logic, and gives the player clues to help solve the puzzle.
To put my ideas in a nutshell: We can improve the sound of games by borrowing
techniques that have been widely used in film production for a long time, and adapting them
to our needs. I suggest making liberal use of foley, field recording of custom ambiences,
and recording as many hard SFX from scratch in the studio as possible, rather than relying
on sound effect libraries. Be careful not to over-compress the audio, and strive for the best
mix of dialog, music, and effects possible. The results will be game soundtracks that are
more unique, interesting, and beautiful to listen to.
CONTACT INFORMATION:
Nick Peck
Perceptive Sound Design
37 Matilda Ave, Mill Valley, CA 94941
Tel/Fax: 415-388-2628
Email: nick@tyedye.com
Web: http://www.perceptivesound.com
BIOGRAPHY
Nick Peck owns and operates Perceptive Sound Design, a firm specializing in audio postproduction
for the game and film industries. His sound design projects have included such
games as Escape from Monkey Island, Vampire the Masquerade: Redemption, Grim
Fandango, Star Wars Super Bombad Racing, and New Legends, as well as the films Being
John Malkovich and the remake of Vampire Hunter D. Peck is also a composer and
keyboardist, holding an MFA in Electronic Music from Mills College. He has released six
albums, ranging from avant garde electronic music to progressive rock, and performs
frequently with his quintetTen Ton Chicken. In March, 2000, Peck completed construction of
a new post production recording facility in Mill Valley, California. Featuring a foley pit,
voiceover booth, grand piano, 5.1 surround sound, 2 Pro Tools systems, high quality
microphones, synthesizers, and recording gear, excellent acoustics, and a 200 gigabyte
online sound effects library.