Category: Video games

Jed Miclot, Senior Sound Designer at Double Helix Games (now part of Amazon Game Studios), delivered an insightful online guest lecture on the sound design of Killer Instinct for Xbox One. In this engaging session, he provided a detailed breakdown of his creative and technical approach to crafting the game’s dynamic and immersive audio experience.

From Film to Games: Miclot’s Journey into Sound Design

Miclot began by sharing his professional background, highlighting his transition from film post-production to video game sound design. Having worked on Harry Potter and other film projects, he eventually shifted his focus to interactive media, drawn by the challenge of designing sound for dynamic gameplay scenarios.

Building a Unique Sonic Identity for Killer Instinct

One of the core themes of the lecture was the importance of creating a distinct audio identity for each character in Killer Instinct. Miclot explained how he designed unique sound palettes that reflected each fighter’s personality, abilities, and fighting style.

Jago, the Tibetan monk fighter, features martial arts-inspired sonic elements that reflect his disciplined yet powerful combat style. His movements are accompanied by crisp martial arts strikes, recorded using real wooden staffs, hand-to-hand impacts, and air displacement effects to simulate the speed of his attacks. To heighten realism, Miclot layered subtle breathing effects and controlled exhalations, making each attack feel deliberate and refined.

Glacius, an alien composed of ice, required frozen textures and resonant impacts to capture his otherworldly nature. To achieve this, Miclot recorded frozen fabric being twisted and broken, ice cubes cracking in water, and glass-like resonance using contact microphones on frozen metal objects. His attacks, which involve ice shards and liquid nitrogen-inspired transformations, were enhanced by recording icicles being shattered and the sound of dry ice sublimating.

For Sabrewulf, the werewolf, a blend of organic growls and Foley elements such as breaking wood and cloth ripping emphasized his primal nature. Miclot layered real wolf growls, lion roars, and bear vocalizations, processed to create a hybrid beast-like voice. His claw attacks were enhanced using recordings of splintering wood and ripping fabric, simulating the forceful tearing of his enemies.

Spinal, the skeletal pirate, was brought to life through creaking bones and wooden textures to enhance his eerie presence. Miclot recorded old wooden floorboards creaking, bones knocking together, and rattling chains to create an undead, cursed aesthetic. Spinal’s vocalizations were constructed using manipulated human screams, whispery ghostly echoes, and reversed percussion elements.

Foley Recording and Creative Sound Sourcing

Miclot’s approach to Foley embraced experimentation with physical objects and environmental interactions to craft a rich and immersive soundscape. To enhance the weight and impact of heavy-footed characters like Sabrewulf, he recorded the sound of pumpkins being smashed, allowing the mix of soft pulp and hard shell impacts to produce a visceral quality that made movements feel raw and animalistic. For Glacius, Miclot soaked an old pair of jeans in water and froze them, manipulating the fabric once solid to capture the crisp crackling of frozen textures. This method proved so effective in simulating ice fractures that it even led to confusion among coworkers when they discovered frozen jeans in the office freezer.

To enhance the eerie atmosphere of Spinal’s stage, Miclot recorded his girlfriend’s snoring while she was unwell, capturing deep, guttural breaths that he later pitched down to resemble an eerie, spectral presence. He also manipulated the sound of air shifting in a toilet bowl, producing unsettling moaning effects that contributed to the ghostly ambiance of Spinal’s environment.

For Orchid’s electrical attacks, Miclot recorded a real Tesla coil generating powerful electrical discharges, using its raw, high-voltage arcs to provide an authentic crackling intensity. He controlled the coil’s amplitude and rate of sparks in real time, capturing variations that could be used dynamically during combat sequences. Similarly, for Sadira’s web-based attacks, he needed a sound that conveyed both elasticity and tension. Stretching duct tape across a long surface and peeling it at different speeds allowed him to mimic the sticky, sinewy strands wrapping around enemies, creating a uniquely organic yet unnerving sound.

Innovative Sound Techniques: Layering and Positional Audio

A key aspect of Killer Instinct’s audio design was its innovative approach to impact sounds. Rather than relying on a single, static sound effect, Miclot designed each impact to be dynamic and multi-layered, enhancing spatial awareness and immersion. When a character is slammed to the ground, the sound is composed of multiple elements, including positional slapback echoes that create a sense of depth and space.

Miclot demonstrated how this system worked using Orchid’s backflip slam, a move where the character is thrown to the ground with a heavy impact. Instead of a single sound event, the slam triggered seven different sound layers, including a shockwave layer, multiple slapback echoes, and a low-frequency boom that played through the subwoofer to reinforce the force of the impact.

For Glacius’s ice-based attacks, different layers of sound simulated the fracturing and shifting of frozen structures. When Glacius smashes an enemy with an ice attack, multiple sound components activate: an initial impact recorded using frozen jeans snapping, a delayed crackling sound simulating stress fractures in the ice, and a distant slapback echo mimicking sound reflections off frozen surfaces.

This dynamic approach was also applied to environmental destruction. When objects in the stage break, multiple sound layers are triggered based on how close the player is to the destruction. If debris falls in the background, the slapback echoes adjust dynamically, making it feel as though the sound is traveling across the space. Miclot’s use of adaptive layering and positional audio ensured that every attack felt spatially alive, adjusting dynamically whether a character was fighting in a confined, echo-heavy environment or an open battlefield.

Adaptive Music: Enhancing Gameplay Feedback

Miclot also discussed the role of Killer Instinct’s dynamic music system, which was designed in collaboration with composer Mick Gordon. Unlike traditional game scores that loop continuously, Killer Instinct’s soundtrack adapts to player actions. The music shifts intensity when a player achieves a high combo streak, providing real-time feedback on gameplay performance. A granular processing effect momentarily distorts the music when a combo breaker is performed, reinforcing the action’s impact. If players stop fighting for six seconds, the music transitions to classic themes from the original Killer Instinct soundtrack. During an ultra combo, each successful hit triggers a sequence of musical notes tied to the character’s theme, turning the final blows into a rhythmic spectacle.

Final Reflections

Miclot’s guest lecture provided an in-depth look at the intricacies of fighting game sound design. His work on Killer Instinct showcased how experimental Foley, creative recording techniques, and adaptive audio implementation can enhance a game’s engagement. By sharing practical insights and demonstrating the thought process behind each sound, his lecture offered valuable knowledge for those looking to push the boundaries of game audio design.

Benoit Tigeot delivered an engaging online lecture on his experiences working on the Just Dance series and the intricacies of sound design in dance video games. His talk provided an in-depth look at the challenges and creative processes involved in crafting immersive audio for an interactive, music-driven game.

From Live Sound to Game Development

Benoit’s journey into sound design began with work on live shows, concerts, and exhibitions, which provided him with a strong foundation in audio engineering. After completing his studies in France, he gained experience in television production, animation dubbing, and studio recording before transitioning into video game audio. His background in live and recorded sound gave him a unique perspective when he joined Ubisoft to work on Just Dance.

Adapting to Game Audio

Despite having no prior experience in game audio, Benoit quickly adapted to the demands of interactive sound design. He worked on multiple Just Dance titles, learning how to integrate music and sound effects into gameplay while ensuring high-quality production standards. The fast-paced development cycle required him to balance creativity with efficiency, as each version of Just Dance was produced in a matter of months.

The Sound Design Workflow

Benoit outlined the workflow for sound design in Just Dance, highlighting key stages such as:

• Track Preparation: Receiving licensed music, ensuring audio quality, and making necessary edits, including removing inappropriate language. For example, in Black Eyed Peas’ songs, multiple words were edited out using backward reverb and other subtle audio modifications to keep the track family-friendly while maintaining its musicality.
• Marker Placement: Adding timing markers to synchronise choreography, animations, and gameplay elements. Benoit emphasised the importance of precision, as even a millisecond difference could impact the timing of dance moves and scoring.
• Sound Effects (SFX) Design: Creating introductory and concluding sound effects for each song, as well as UI and gameplay sounds. In Just Dance Japan, additional sound effects were incorporated at the beginning and end of tracks to enhance the user experience. The sound team also created unique effects for different dance modes, such as battle mode, where transitional audio had to blend seamlessly between competing tracks. Over 150 different SFX variations were tested to find the right balance between energy and smooth musical transitions.
• Integration and Testing: Implementing audio into Ubisoft’s proprietary engine, collaborating with developers and artists, and ensuring synchronisation across multiple platforms. Benoit described how the team used text-based scripting in Sublime Text to adjust pitch, loop points, and volume, allowing for quick iteration and adjustments across the game. He also discussed how the team recorded crowd reactions and player feedback sounds in a dedicated studio space to ensure an immersive experience.

Challenges in Dance Game Audio

Working on Just Dance presented unique challenges, including:

• Multi-platform Development: Adapting audio for different consoles and ensuring consistency across devices.
• Cross-Studio Collaboration: Coordinating with teams worldwide, including those in France, India, and the UK.
• Real-time Testing: Evaluating sound integration in a dynamic, open-plan workspace filled with music and dance rehearsals. Benoit noted that sound designers had to contend with a noisy environment, making it difficult to hear and refine subtle audio details.
• Genre Adaptability: Designing sound for a wide range of musical styles while maintaining a cohesive experience. He explained how the team had to ensure that different styles—ranging from electronic dance music to country—had consistent and engaging audio treatments without overwhelming players with excessive effects.

Reflections on Sound Design in Just Dance

Benoit’s lecture provided a valuable look at the evolution of Just Dance’s audio technology. He discussed the transition to a new game engine, which improved workflow efficiency and allowed for greater creative flexibility. His work on developing in-game sound effects, enhancing music transitions, and refining player feedback mechanisms contributed significantly to the game’s audio experience. For instance, in Just Dance’s battle mode, the team spent weeks fine-tuning SFX to ensure that energy levels were maintained across song transitions without jarring interruptions. Additionally, subtle effects such as footstep sounds, applause, and even costume rustling were layered in to enhance immersion.

For aspiring sound designers, Benoit’s talk underscored the importance of adaptability, collaboration, and technical proficiency. His ability to bridge creative and technical aspects of sound design made him a key contributor to one of Ubisoft’s most successful franchises. He also highlighted how working in a rhythm-based game required constant iteration, as any mistake in beat markers or mixing could significantly impact the player’s experience. The balance between technical precision and creative storytelling through sound remains an essential aspect of game audio development.

Benoit’s lecture offered a fascinating glimpse into the behind-the-scenes work that brings rhythm-based games to life. His experiences serve as an inspiration for those interested in audio design for interactive media, highlighting the rewarding challenges of working in the field of game sound.

The world of game audio presents unique challenges and opportunities, and few individuals have navigated this space with as much depth and insight as David Thiel. In an online guest lecture, Thiel shared his extensive experience in interactive audio, covering its evolution, principles, and creative approaches. With a career spanning over four decades, his work has influenced interactive entertainment, from early arcade machines to modern gaming environments.

Interactive Audio vs Linear Audio

Thiel began by distinguishing between interactive audio—used in games—and linear audio, which is typical of film and television. Unlike linear audio, where sounds are meticulously timed to match fixed visuals, game audio must be dynamic. It adapts in real-time based on player interactions, requiring a more flexible and responsive approach to composition and sound design.

One key challenge he highlighted is unpredictability. In a film, every sound effect and piece of music is placed with precise timing. In contrast, game audio must account for numerous possibilities—players might trigger events in different sequences or at varying speeds. This means that game sound designers must think beyond static cues, ensuring that every sound conveys meaning while enhancing immersion. Thiel illustrated this with an analogy: Imagine posting a movie where you know which events will occur but have no idea when or in what order they will happen. He then expanded on this by explaining how game audio is akin to composing for an orchestra where each instrument plays independently based on player input, making real-time adaptation essential.

Making Game Audio Meaningful

One of Thiel’s core principles is that game audio should always provide useful information to the player. Sounds should not just be aesthetically pleasing but should also enhance gameplay. For example, when a player fires a weapon in a game, the sound can communicate crucial details such as the type of gun, whether it is running out of ammunition, or if the shot has hit a target. He elaborated on this concept by breaking down the sound design process for firearms: A shotgun blast should feel weighty and reverberate differently in an open space versus a confined corridor, while an energy weapon should have an otherworldly charge-up sound. Additionally, missed shots and ricochets can provide players with subtle cues about their accuracy, reinforcing the importance of audio feedback.

Another fundamental aspect is variation. If a game reuses the same audio sample repeatedly, players may quickly lose their sense of engagement. Thiel demonstrated how game audio can introduce subtle variations based on contextual factors, such as the shooter’s position, the remaining ammunition in a weapon, or environmental influences. He provided an example from Borderlands 2, where he spent over 1,000 hours playing and noted how the game’s procedural gun system extended to audio, ensuring that weapons sounded unique based on their make and function. Each gun has a different reload sound sequence, creating deeper immersion and ensuring that players can distinguish between weapons purely through audio cues.

Additionally, Thiel discussed the importance of environmental sounds in enhancing game immersion. He explained how in Winter Games (1985), all sound effects were synthesised in real-time, yet they managed to convey the distinct feel of ice skating. By manipulating pitch and timbre, the sound team created convincing audio cues that responded dynamically to skater movements.

The Role of Music in Interactive Audio

Music in games also requires a different approach compared to linear media. Thiel recounted his early experiences in the 1980s, where hardware constraints required music to be generated in real-time using algorithms. Though modern hardware allows for pre-recorded music with high production values, he highlighted the benefits of runtime-generated music, such as the ability to synchronise musical cues dynamically with gameplay.

A particularly engaging example was his work on pinball machines. In Monday Night Football pinball, musical motifs and drum beats were triggered by player actions, enhancing gameplay feedback. When the player scored a goal, a celebratory fanfare played, rising in pitch with each successive goal, reinforcing the excitement. Similarly, synchronised drum fills were used when the ball passed through a spinner, making player actions musically rewarding. Another notable example was from Torpedo Alley, where a cowbell sound layer was introduced when players entered a time-limited game mode, ensuring they knew they had a short window to act. The cowbell was musically integrated, but also acted as a warning that the mode would soon expire, influencing player behaviour.

Thiel also explored how interactive music could adapt to player performance. He referenced a pinball machine where successfully hitting targets would cause the background music to shift in pitch, making victories more rewarding. Each successive successful shot raised the key of the soundtrack, creating a musical escalation that heightened player excitement.

Challenges in Speech and Sound Effects

Thiel also touched on the complexities of speech and sound effects in games. While modern storage capacities allow for extensive voice recordings, game dialogue must be carefully managed to maintain clarity and engagement. He shared insights into ‘speech wrangling’—the process of organising, editing, and integrating thousands of voice clips in a way that is useful to game developers.

Sound effects, meanwhile, are not simply lifted from libraries but are often layered and modified to enhance realism. Thiel illustrated this with an explosion sound effect: Rather than using a single sample, he combined elements such as a low-end impact, a sharp transient, and a synthesised decay to create a more impactful and informative effect. He also explained how the iconic sound of the Ark of the Covenant in the Indiana Jones pinball machine was created using a manipulated orchestral harp sound called ‘Psycho Drone’—an example of how concept sometimes takes precedence over traditional realism in sound design.

Additionally, Thiel described how synchronised sound cues could be used to communicate time-sensitive objectives. In a pinball machine, for example, the sound of a looping crowd chant helped signal an urgent task. Players needed to hit a specific target before the chant faded, using sound as a direct gameplay indicator.

Mixing and Mastering for Different Environments

A crucial part of game audio design is ensuring that sounds are balanced correctly in different playback environments. Thiel noted the differences between public spaces (such as arcades or casinos) and private listening setups (home gaming, mobile devices). In noisy public settings, audio cues must be clear and loud, often using aggressive mixing techniques such as ducking (reducing background volume when key sounds play). In contrast, home environments allow for more subtle layering, offering richer detail and depth.

The Passion Behind Game Audio

Thiel concluded his talk with reflections on the industry’s approach to audio. Despite the immense progress in gaming technology, he observed that audio still receives a smaller share of development resources compared to graphics. However, for those passionate about sound, game audio remains a deeply rewarding field that requires creativity, problem-solving, and technical expertise.

This lecture provided a fascinating exploration of interactive audio, offering both historical perspectives and practical insights. Whether you are an aspiring game audio designer or simply interested in the intricacies of interactive sound, Thiel’s knowledge and experience shed light on the challenges and artistry of game audio creation.

 

The world of video games is filled with sounds that transport players to new dimensions, from the roar of mythical creatures to the hum of futuristic technology. Behind these sounds are dedicated professionals like Brad Meyer, Audio Director at Sucker Punch Productions, who delivered an engaging guest lecture on fantasy sound design.

With extensive experience in the industry, Meyer has worked on a variety of games, including Jurassic Park, Spider-Man: Web of Shadows, and Infamous: Second Son. Throughout his talk, he offered invaluable insights into the creative process of designing sounds for fantastical worlds, emphasising grounding fantasy in reality to create more relatable and engaging audio experiences.

Bringing the Fantastic to Life Through Sound

Fantasy in games is often associated with dragons, magic, and mythical landscapes. However, as Meyer pointed out, fantasy sound design extends beyond these stereotypes, encompassing anything that is imagined and does not exist in the real world. Even games with realistic settings use elements of fantasy, whether in the form of futuristic interfaces, enhanced environments, or supernatural abilities.

Meyer stressed the importance of using real-world sounds as the foundation for fantastical audio effects. By incorporating recognisable sonic elements, such as animal calls, mechanical noises, or environmental recordings, designers create sounds that feel authentic while still supporting the game’s imaginative elements.

For instance, in Jurassic Park 3: The DNA Factor, Meyer and his team used a mix of bird calls, walrus vocalisations, and cat growls to construct dinosaur sounds that felt realistic despite the fact that no one knows what dinosaurs actually sounded like. Similarly, in Spider-Man: Web of Shadows, he layered compressed air bursts and rope creaks to create the sensation of Spidey’s web-slinging.

The Process of Crafting Unique Sounds

Throughout the lecture, Meyer shared several examples of how he and his team develop sounds through experimentation, layering, and creative thinking.

• Using Unconventional Materials – While designing the sound of Frogger’s tongue, Meyer found inspiration in party whistles, using their retracting motion to simulate the flick of a frog’s tongue. By varying the speed and intensity of the whistle, different versions of the sound were created to match in-game actions.
• Inventing New Recording Techniques – To create the sound of moving concrete debris in Infamous: Second Son, Meyer built a tumbling machine by filling a padded rubbish bin with rocks and rolling it around. The method allowed for recording a continuous shifting effect, which was later manipulated to fit different gameplay sequences.
• Manipulating Sound Through Processing – In Infamous, neon powers were designed using fluorescent tube recordings, which were then modified with spectral filters to create a futuristic energy effect. By applying different reverb and pitch-shifting techniques, the sound was given a more otherworldly quality.
• Blending Organic and Synthetic Sounds – The sound of a dragon’s wings in a fantasy game may be created by layering recordings of bird wing flaps, slowed-down flag waving, and subtle jet engine noise to provide depth and power. Time-stretching and low-pass filters can be applied to emphasise the size of the creature.
• Recording and Modifying Everyday Objects – To mimic the sound of a robotic exosuit, designers may record hydraulic lifts, servo motors, and mechanical gears, then process them using granular synthesis and distortion effects to enhance their sci-fi aesthetic.
• Creating Footsteps for Different Surfaces – Footsteps on gravel may be created by recording boots on loose stones, while icy terrain can be simulated by crushing cornstarch or frozen lettuce to create crisp crunching sounds. Layering multiple recordings at different distances can provide more depth and realism.
• Using Water for Unusual Effects – To produce eerie, otherworldly sounds, designers can use hydrophones to record underwater gurgling, later modifying the pitch and layering additional elements. This technique has been used for deep-sea creatures, alien environments, and magical spells.
• Layering Human and Animal Sounds – To create monstrous growls or supernatural voices, sound designers often blend human vocalisations with recordings of animals such as tigers, wolves, and pigs. Adjusting the pitch and applying formant shifting can transform these elements into something unrecognisable yet believable.
• Using Foley Techniques for Impact Sounds – Sword clashes may be recorded using real metal objects, but layering them with additional materials such as celery snaps or baseball bat impacts adds weight and crunch. Similarly, explosions often incorporate recordings of firecrackers, fireworks, and even slowed-down balloon pops to create the necessary sonic texture.

The Importance of Collaboration and Experimentation

Meyer highlighted that sound design is a process of constant learning and experimentation. He encouraged students and professionals alike to embrace failure, as each unsuccessful attempt teaches valuable lessons that contribute to better sound design in the future.

Additionally, he emphasised the importance of community in the sound design industry. Engaging with other professionals, sharing techniques, and attending meetups can lead to fresh ideas and innovation. In Seattle, where Meyer is based, game audio professionals regularly gather to discuss their work, reinforcing the value of collaboration.

Guidance for Aspiring Sound Designers

1. Ground fantasy sound design in reality – Using real-world recordings makes fantastical sounds more relatable and engaging.
2. Experiment with unconventional sources – Anything from household objects to wildlife recordings can become the basis for unique sound effects.
3. Keep an organised sound library – Effective cataloguing of sounds ensures efficiency in future projects.
4. Do not be afraid to fail – Trial and error is part of the creative process.
5. Engage with the sound design community – Collaboration and networking can lead to new opportunities and insights.

Meyer’s talk provided a fascinating look into the art and science of crafting compelling audio for games. Whether designing the roar of a dragon or the hum of a futuristic machine, the secret lies in finding inspiration in the real world and shaping it into something new.

For those interested in pursuing sound design, his advice is clear: stay curious, experiment fearlessly, and never stop learning.

 

Game audio is an intricate blend of creativity and technical proficiency, shaping immersive player experiences. In his lecture, Aaron Marks, a seasoned expert in game audio, shared valuable insights into the evolving landscape of sound design, audio programming, and the industry’s expectations from professionals. His talk covered various aspects of game audio, from creating soundscapes to collaborating with developers, and even the business side of the industry.

Aaron Marks is an accomplished game audio professional with decades of experience in sound design, music composition, and field recording. He is the author of The Complete Guide to Game Audio, a widely respected book used by aspiring and professional game audio designers. Additionally, he has authored Game Audio Development, providing in-depth insights into the technical and creative aspects of interactive sound design. Marks has contributed to numerous games, including NASCAR Heat 4, Ring of Elysium, Ghost in the Shell: First Assault Online, Red Orchestra 2: Heroes of Stalingrad, and Tom Clancy’s EndWar, showcasing his expertise in sound design, field recording, and composition across various genres.

The Role of Audio in Video Games

Aaron Marks emphasised the vital role that audio plays in gaming, from the immersive quality of sound effects to the emotional impact of music. Unlike film, where audio is linear and carefully timed, game audio must be adaptive and dynamic, responding to player actions in real-time.

Marks, who teaches at the Art Institute in San Diego, structures his course to equip students with practical skills in sound editing, implementation, and understanding the development pipeline. He noted that students must leave the course with tangible skills that make them attractive to game developers, including familiarity with middleware like Wwise and FMOD.

The Challenge of Keeping Up with Technology

One of the most common concerns among aspiring game audio professionals is staying up to date with the ever-evolving technology. Marks reassured his audience that instead of trying to master every new tool, they should focus on understanding fundamental audio principles and adapt when needed.

Rather than memorising every function of a software update, he suggested familiarising oneself with tutorials and getting hands-on experience only when required. This approach helps sound designers stay efficient and not be overwhelmed by constant technological changes.

The Growing Demand for Audio Programmers

One key takeaway was the increasing demand for audio programmers. Marks recounted a conversation with an audio director at a leading game developer who was actively seeking an audio programmer even while having numerous sound designers available.

This highlights the importance of programming knowledge in game audio. While not mandatory, having skills in scripting languages such as C# or Python can significantly enhance one’s employability, especially for small development teams where technical implementation is crucial.

Design Examples

For footsteps in different environments, record footsteps on various surfaces such as wood, gravel, concrete, and wet ground using a high-quality field recorder. Enhance realism by layering different recordings, such as separate heel and toe impacts, and adjusting the pitch and volume dynamically to avoid repetition. Use parametric EQ to fine-tune the frequency response and add slight randomisation in playback through Wwise or FMOD to make each step feel unique.

For gunfire effects, combine multiple elements, such as mechanical clicks (captured using metallic objects), muzzle blasts (recorded from actual firearms if possible), and reverb tails (captured from different distances). Use layering techniques to create depth, adjusting low-end frequencies for power and adding a subtle distortion effect to enhance realism. Implement gunfire effects using multiple variations and pitch shifting to prevent repetitive audio patterns.

For ambient soundscapes, capture field recordings in locations that match the intended game environment, such as forests, cities, or caves. Use stereo imaging and reverb to simulate realistic depth and distance, adjusting based on proximity cues in the game engine. Add movement by using modulated panning and volume automation to create a sense of a living, breathing world.

For dynamic music transitions, compose music in layers that can be triggered dynamically in response to in-game events. Use tools like Wwise or FMOD to create seamless crossfades between different musical moods, such as shifting from calm exploration music to an intense combat theme. Implement adaptive musical stingers that introduce new elements based on enemy encounters, player health, or location changes.

For procedural sound effects, instead of using static audio files, generate sounds through synthesis and procedural techniques. For example, generate wind and rain using noise-based synthesis with modulated filters to create natural variation. Use physics-based procedural sound engines to dynamically generate impact sounds based on object weight, speed, and material type.

Final Thoughts

Aaron Marks’ lecture provided a comprehensive look into the world of game audio, covering both technical and business aspects. Whether it’s creating dynamic soundscapes, recording weapons in the field, or optimising casino game audio, the industry offers a wide range of opportunities for those willing to explore. For those looking to break into game audio, the key takeaway is to stay adaptable, build strong relationships, and continuously refine your craft. The world of interactive audio is ever-changing, but with passion and persistence, a rewarding career awaits.

 

Dr. Milena Droumeva, an expert in game sound, acoustic ecology, and digital media, is an Associate Professor at Simon Fraser University. She researches sound studies, interaction design, and immersive audio environments. In an online guest lecture, Dr Droumeva explored how sound shapes experiences across various media, particularly in video games.

The Role of Sound in Games

Game sound serves multiple functions, including:

• Informational: Providing feedback through alerts, warnings, and reward sounds.
• Affective: Setting the emotional tone of the game through music and sound effects.
• Communicative: Enhancing storytelling and narrative engagement.
• Spatial: Creating a sense of atmosphere and immersion.

All game sounds interact dynamically, making each playthrough unique. Unlike traditional media, where sound is fixed, game sound reacts in real time to player input, enhancing immersion and believability in virtual environments.

Fidelity vs. Verisimilitude: Two Paths to Realism

Fidelity in game sound refers to how accurately in-game audio replicates real-world sounds. Technological advancements have drastically improved fidelity, moving from simple 8-bit chiptunes to highly detailed soundscapes with 3D spatial audio. For example, modern first-person shooter (FPS) games utilise high-fidelity sound to replicate gunfire, environmental acoustics, and movement sounds with great precision.

While fidelity focuses on realism, verisimilitude concerns itself with believability within the game world. Not all games aim for strict realism—fantasy RPGs like Final Fantasy or Zelda prioritise creating an immersive, internally consistent soundscape rather than mimicking real-world sounds. Iconic game sound effects, such as Mario’s jump sound or Zelda’s treasure chest chime, are less about real-world accuracy and more about maintaining an established, recognisable aesthetic.

The Evolution of Game Sound

The history of game sound can be divided into key phases:

1. Early Video Games: Minimalist, synthesised melodies with limited sound effects.
2. 16-bit Era: Polyphonic MIDI compositions and richer audio textures.
3. Modern Gaming: High-fidelity digital audio, dynamic soundscapes, and adaptive audio engines.
4. 3D & VR Sound Design: Spatial audio and immersive environmental effects that enhance realism.

Games have evolved from simple beeps and loops to intricate, cinematic experiences where soundscapes enhance gameplay and narrative depth. Today’s games feature dynamic audio that responds to player actions, creating immersive environments that rival film and television in complexity and emotional impact.

Acoustic Ecology and Game Soundscapes

Acoustic ecology, a concept introduced by Professor Barry Truax, views sound as part of an interconnected system where the environment and listener influence one another. In games, this means understanding how various sound elements—background music, ambient noise, dialogue, and sound effects—interact to create a cohesive soundscape.

For instance:

• FPS games use environmental reverb and echo to simulate realistic spaces.
• RPGs incorporate thematic soundtracks to create a sense of place.
• Arcade games employ catchy, repetitive melodies designed to grab attention in noisy environments.

The balance of sound in a game environment is crucial. Overloading a soundscape with too many auditory elements can create clutter, while strategic use of silence can heighten suspense and impact.

The Future of Game Sound

Despite technological advancements, game sound design still faces challenges. Audio design often receives less investment compared to visual graphics, and many game developers rely on conventional sound design approaches rather than exploring new, experimental techniques. However, the rise of AI-generated sound, real-time adaptive audio, and VR-driven spatial audio suggest that the future of game sound will continue to push the boundaries of immersion and interactivity.

Conclusion

Game sound is a rich field that bridges technology, culture, and player experience. Understanding it through the lenses of fidelity, verisimilitude, and acoustic ecology offers a more nuanced perspective on how sound functions within interactive media. Next time you play a game, take a moment to listen—what role does sound play in your immersion? How does it shape the way you experience the game world? For those interested in exploring game sound further, consider experimenting with muting visuals or audio during gameplay to analyse how different sound elements contribute to the overall experience. The world of game audio is vast, and there’s always more to discover!

 

Welcome to an insider’s perspective on audio design at one of the world’s leading video game studios, EA Sports. Jesse James Allen, an audio director with a passion for sound design, video games, and automobiles, shared his expertise in a captivating lecture. With over 30 games and two decades of experience, Allen’s insights are a goldmine for anyone intrigued by the intricacies of video game audio design.

From Passion to Profession

Allen’s journey into EA Sports began with a love for sound and automobiles. His early work included recording cars for documentaries, a skill that eventually led him to a position at EA’s Tiburon studio in Orlando. Starting with the NASCAR series, he collaborated with the Need for Speed team to craft immersive engine sounds—a thrilling career path that remains as dynamic as the games he’s helped create.

The Art of Music Selection

One of the most common questions Allen receives is: “How do I get my music into a video game?” The answer lies in the tailored approach EA takes for each title. For global hits like FIFA, EA curates an internationally appealing playlist featuring emerging artists. On the other hand, cinematic games like Mass Effect rely on composed scores to complement their epic narratives. The meticulous selection process involves predicting future hits to ensure the music resonates with players upon the game’s release.

Interactive Music Systems

Allen highlighted cutting-edge interactive music systems that adapt to gameplay in real time. For instance, dynamic layering allows music to shift seamlessly based on player actions. A great example is SSX’s “Rider Remix” system, where gameplay mechanics like grinding or big air tricks trigger real-time music manipulations. These systems immerse players by aligning audio intensity with their in-game experiences.

Arena vs Open World Soundscapes

Allen distinguished between two core sound design philosophies:

• Open World Design: Games like Mass Effect use a “virtual microphone” attached to the player to dynamically adjust sound volumes as they explore diverse environments.
• Arena-Based Design: Sports games like Madden feature centralised soundscapes, where crowd noises and player interactions are carefully layered to enhance the atmosphere.

The Science of Authenticity

EA’s commitment to authenticity is exemplified by their meticulous recording techniques. From attaching microphones to NASCAR vehicles to simulating crowd reactions in football stadiums, the attention to detail ensures players feel truly immersed. Notable innovations include advanced recording setups for capturing car exhausts and snowboarding sounds, bringing unparalleled realism to games like SSX.

Breaking into the Industry

For aspiring sound designers, Allen provided a wealth of advice on making a successful entry into the video game industry. Here are the key takeaways:

1. Master the Tools: Familiarise yourself with industry-standard tools such as Native Instruments’ Reaktor and Cycling ‘74’s Max. These platforms allow you to experiment with real-time sound manipulation and build a portfolio of interactive sound designs.
2. Learn the Fundamentals: A strong foundation in audio engineering and sound design is essential. This includes understanding concepts like dynamic layering, mixing for interactive environments, and creating adaptive soundscapes.
3. Gain Hands-On Experience: Seek opportunities to work on real-world projects. Whether it’s modding existing games, creating soundscapes for indie projects, or collaborating with other creatives, practical experience is invaluable.
4. Explore Internship Opportunities: Look for paid internship programmes in the video game industry that offer direct mentorship and hands-on involvement in game development. These opportunities provide invaluable real-world experience and a chance to network with industry professionals.
5. Stay Inspired and Persistent: Breaking into the industry requires passion and perseverance. Attend industry events, network with professionals, and never stop learning. Tools like Audio Kinetics’ Wwise can also help you gain a better understanding of runtime audio systems.
6. Build a Unique Portfolio: Stand out by showcasing your creativity and technical skills. Include examples of dynamic audio systems, interactive music compositions, and authentic sound recordings.

Final Thoughts

Jesse James Allen’s lecture offers a rare glimpse into the world of video game audio design. His passion, innovation, and dedication are a testament to the artistry that underpins EA Sports’ iconic games. For anyone dreaming of crafting soundscapes that captivate millions, this masterclass was an invaluable inspiration.

 

Audio in video games isn’t just about what you hear—it’s about how you feel. Rob Pearsall, an experienced sound designer and audio tester, provided a fascinating deep dive into the intricacies of audio testing for games. Drawing from his work on iconic franchises like Halo, he shared valuable lessons on what makes game audio tick and how to test it effectively.

What Makes Game Audio Unique?

Unlike films, where soundtracks sync directly with linear visuals, game audio is dynamic and interactive. It adapts to player actions and constantly shifts depending on the game environment. Rob emphasised this distinction, explaining how sound systems in games are programmed to respond to varying in-game conditions, creating a fully immersive auditory experience.

For example:

• Interactive Music: Games often use “stems” (individual tracks for different instruments) to allow for endless variations of a song. This ensures players rarely hear the same combination twice.
• 3D Soundscapes: Audio in games must convey spatial awareness. A helicopter passing overhead or a distant explosion must sound authentic, adjusting in volume, pitch, and filtering based on the player’s position.

Essential Skills for Audio Testing

Testing audio in games requires more than just a good ear. Rob highlighted two key areas of expertise:

1. Testing Proficiency: The fundamentals of identifying, documenting, and tracking issues are crucial.
2. Audio Knowledge: A professional understanding of sound design is indispensable. From mastering tools like Wwise and FMOD to comprehending audio behaviours like occlusion and attenuation, testers must be equipped to evaluate the nuances of game audio.

Challenges in Audio Testing

Rob shared several challenges he encountered while working on Halo:

• Volume and Memory Balancing: Managing the memory footprint of audio is critical. Overloading the system with sounds can lead to important effects—like gunfire—being cut off during gameplay.
• Consistency Across Game Modes: Audio transitions and behaviours can differ between campaign and multiplayer modes, requiring thorough testing in all scenarios.
• Environmental Sound Design: Ambient sounds and reverb must seamlessly match the environment. For instance, moving from a cave to an open field should feel natural to the player.

One particularly intriguing example involved reverb mismatches on vehicles. In one scenario, players riding in a Halo Warthog experienced reversed indoor and outdoor reverb effects depending on their position in the vehicle—a fascinating bug that required careful debugging to resolve.

Lessons from the Field

Rob’s anecdotes revealed just how intricate audio testing can be:

• Dynamic Sound Systems: Testing dialogue systems, which must adapt to various player interactions, requires a keen ear to ensure appropriate responses are triggered in the right contexts.
• Material Systems: Every surface in a game world—be it grass, metal, or water—affects the sounds of footsteps, gunfire, and interactions. Testing involves ensuring these sounds match their environments accurately.
• Loop Points: Ambient audio loops must blend seamlessly. A poorly looped sound can disrupt the immersion, drawing players out of the experience.

Tips for Aspiring Audio Testers

1. Build Relationships: Collaboration with designers and developers is crucial. Gaining their trust can improve feedback and streamline the debugging process.
2. Leverage Tools: Use profiling tools in middleware systems like Wwise or FMOD to isolate and test specific sounds or scenarios.
3. Study the Game: Understanding the lore and design principles of a game ensures consistency across sequels and expansions.

Final Thoughts

Rob concluded with a message of optimism: as game audio continues to evolve, so too will the tools and techniques used to create and test it. By fostering a partnership between creativity and technology, developers can craft unforgettable soundscapes that immerse players like never before.

For Rob, the ultimate goal is simple: make games fun and immersive. And with experts like him pushing the boundaries of audio, the future of gaming sounds brighter than ever.

Game Credits

Rob Pearsall has an extensive portfolio in game audio, contributing to over 50 titles including The Lamplighters League, Falcon Age, Necropolis, Destiny: The Taken King, Destiny, Halo 4, Halo: Combat Evolved Anniversary, and The Lord of the Rings. His company, Rip Street Audio, has played a significant role in delivering audio direction, music, and sound design for these projects.

 

In the world of video games, audio still is often an unsung hero, subtly enriching our gaming experiences. Yet, it is a complex and intricate field that demands expertise, creativity, and innovation. Saki Kaskamanidis, the late Audio Director at United Front Games, provided compelling insights into the art of audio direction for interactive entertainment, drawing from his experience on the critically acclaimed game Sleeping Dogs.

The Evolution of a Career in Game Audio

Saki’s journey into game audio was as inspiring as it was unique. Starting as a composer for Electronic Arts’ Need for Speed series, his lack of technical knowledge didn’t hinder him—musical skills opened the door. Over 18 years, he grew to master sound design, implementation, post-production, dialogue direction, and even scriptwriting. This diverse background laid the foundation for his role as Audio Director at United Front Games.

The Birth of Sleeping Dogs

Originally codenamed Blue Lotus, Sleeping Dogs was a bold attempt to challenge the dominance of open-world giants like Grand Theft Auto. Set in contemporary Hong Kong, the game’s narrative focused on an undercover cop, delivering a darker, more serious tone compared to its competitors. Despite setbacks, including being dropped by its original publisher, Activision, the project found a second life under Square Enix and became a critical success.

Audio: The Heartbeat of Interactive Storytelling

Saki emphasised that sound was an integral part of storytelling. “Sound and story are one,” he explained, underscoring the importance of integrating audio into every phase of production. Unlike film, video game audio had to adapt dynamically to player interactions, making it a unique challenge.

The Three Pillars of Game Audio

Saki categorised game audio into three pillars: sound design, music, and dialogue. Each played a crucial role in crafting immersive experiences:

1. Sound Design: From the visceral crunch of hand-to-hand combat to the roar of vehicle engines, sound design in Sleeping Dogs was meticulously planned. For instance, vehicle engine sounds were created using recordings of real cars, classified into 15 categories to streamline production.
2. Music: The score for Sleeping Dogs blended traditional Chinese instruments with Western elements, reflecting the game’s cultural setting. The dynamic score adapted to gameplay scenarios, while diegetic music added realism, playing from radios, boomboxes, and more.
3. Dialogue: Dialogue brought authenticity to Sleeping Dogs. Over 75,000 lines were recorded, with 45,000 making the final cut. To ensure cultural accuracy, ambient Cantonese dialogue was recorded on-site in Hong Kong, capturing the city’s vibrant soundscape.

Innovative Tools and Techniques

A key aspect of game audio was the use of middleware tools like Wwise and FMOD, which enabled dynamic soundscapes. For instance, the same explosion sound could vary in intensity and distance based on the player’s position. Such tools empowered designers to create immersive, context-sensitive audio experiences.

Challenges and Triumphs

Saki shared the challenges of creating convincing audio for vehicles, fighting, and gunplay. Early attempts at realistic fight sounds fell flat, leading to the creation of “hyped-up reality” effects inspired by films like The Bourne Ultimatum. Gun sounds required layered effects and innovative use of sound libraries to replicate the visceral punch of firearms.

The Future of Game Audio

Saki envisioned advancements in 3D audio, sound reflections, and occlusion. Technologies like binaural sound and improved computational power promised to elevate audio realism, enhancing player immersion. He believed these developments would bring unparalleled depth to gaming experiences, making them more lifelike than ever before.

Conclusion

Saki Kaskamanidis’ lecture illuminated the artistry and technical mastery behind game audio. From crafting dynamic soundscapes to recording authentic dialogue, his work on Sleeping Dogs exemplified how audio elevated storytelling and gameplay. For aspiring audio professionals, his journey was a testament to the power of passion, adaptability, and continuous learning.