Category: Video games

Dr. Milena Droumeva, an expert in game sound, acoustic ecology, and digital media, is an Associate Professor at Simon Fraser University. She researches sound studies, interaction design, and immersive audio environments. In an online guest lecture, Dr Droumeva explored how sound shapes experiences across various media, particularly in video games.

The Role of Sound in Games

Game sound serves multiple functions, including:

• Informational: Providing feedback through alerts, warnings, and reward sounds.
• Affective: Setting the emotional tone of the game through music and sound effects.
• Communicative: Enhancing storytelling and narrative engagement.
• Spatial: Creating a sense of atmosphere and immersion.

All game sounds interact dynamically, making each playthrough unique. Unlike traditional media, where sound is fixed, game sound reacts in real time to player input, enhancing immersion and believability in virtual environments.

Fidelity vs. Verisimilitude: Two Paths to Realism

Fidelity in game sound refers to how accurately in-game audio replicates real-world sounds. Technological advancements have drastically improved fidelity, moving from simple 8-bit chiptunes to highly detailed soundscapes with 3D spatial audio. For example, modern first-person shooter (FPS) games utilise high-fidelity sound to replicate gunfire, environmental acoustics, and movement sounds with great precision.

While fidelity focuses on realism, verisimilitude concerns itself with believability within the game world. Not all games aim for strict realism—fantasy RPGs like Final Fantasy or Zelda prioritise creating an immersive, internally consistent soundscape rather than mimicking real-world sounds. Iconic game sound effects, such as Mario’s jump sound or Zelda’s treasure chest chime, are less about real-world accuracy and more about maintaining an established, recognisable aesthetic.

The Evolution of Game Sound

The history of game sound can be divided into key phases:

1. Early Video Games: Minimalist, synthesised melodies with limited sound effects.
2. 16-bit Era: Polyphonic MIDI compositions and richer audio textures.
3. Modern Gaming: High-fidelity digital audio, dynamic soundscapes, and adaptive audio engines.
4. 3D & VR Sound Design: Spatial audio and immersive environmental effects that enhance realism.

Games have evolved from simple beeps and loops to intricate, cinematic experiences where soundscapes enhance gameplay and narrative depth. Today’s games feature dynamic audio that responds to player actions, creating immersive environments that rival film and television in complexity and emotional impact.

Acoustic Ecology and Game Soundscapes

Acoustic ecology, a concept introduced by Professor Barry Truax, views sound as part of an interconnected system where the environment and listener influence one another. In games, this means understanding how various sound elements—background music, ambient noise, dialogue, and sound effects—interact to create a cohesive soundscape.

For instance:

• FPS games use environmental reverb and echo to simulate realistic spaces.
• RPGs incorporate thematic soundtracks to create a sense of place.
• Arcade games employ catchy, repetitive melodies designed to grab attention in noisy environments.

The balance of sound in a game environment is crucial. Overloading a soundscape with too many auditory elements can create clutter, while strategic use of silence can heighten suspense and impact.

The Future of Game Sound

Despite technological advancements, game sound design still faces challenges. Audio design often receives less investment compared to visual graphics, and many game developers rely on conventional sound design approaches rather than exploring new, experimental techniques. However, the rise of AI-generated sound, real-time adaptive audio, and VR-driven spatial audio suggest that the future of game sound will continue to push the boundaries of immersion and interactivity.

Conclusion

Game sound is a rich field that bridges technology, culture, and player experience. Understanding it through the lenses of fidelity, verisimilitude, and acoustic ecology offers a more nuanced perspective on how sound functions within interactive media. Next time you play a game, take a moment to listen—what role does sound play in your immersion? How does it shape the way you experience the game world? For those interested in exploring game sound further, consider experimenting with muting visuals or audio during gameplay to analyse how different sound elements contribute to the overall experience. The world of game audio is vast, and there’s always more to discover!

 

Welcome to an insider’s perspective on audio design at one of the world’s leading video game studios, EA Sports. Jesse James Allen, an audio director with a passion for sound design, video games, and automobiles, shared his expertise in a captivating lecture. With over 30 games and two decades of experience, Allen’s insights are a goldmine for anyone intrigued by the intricacies of video game audio design.

From Passion to Profession

Allen’s journey into EA Sports began with a love for sound and automobiles. His early work included recording cars for documentaries, a skill that eventually led him to a position at EA’s Tiburon studio in Orlando. Starting with the NASCAR series, he collaborated with the Need for Speed team to craft immersive engine sounds—a thrilling career path that remains as dynamic as the games he’s helped create.

The Art of Music Selection

One of the most common questions Allen receives is: “How do I get my music into a video game?” The answer lies in the tailored approach EA takes for each title. For global hits like FIFA, EA curates an internationally appealing playlist featuring emerging artists. On the other hand, cinematic games like Mass Effect rely on composed scores to complement their epic narratives. The meticulous selection process involves predicting future hits to ensure the music resonates with players upon the game’s release.

Interactive Music Systems

Allen highlighted cutting-edge interactive music systems that adapt to gameplay in real time. For instance, dynamic layering allows music to shift seamlessly based on player actions. A great example is SSX’s “Rider Remix” system, where gameplay mechanics like grinding or big air tricks trigger real-time music manipulations. These systems immerse players by aligning audio intensity with their in-game experiences.

Arena vs Open World Soundscapes

Allen distinguished between two core sound design philosophies:

• Open World Design: Games like Mass Effect use a “virtual microphone” attached to the player to dynamically adjust sound volumes as they explore diverse environments.
• Arena-Based Design: Sports games like Madden feature centralised soundscapes, where crowd noises and player interactions are carefully layered to enhance the atmosphere.

The Science of Authenticity

EA’s commitment to authenticity is exemplified by their meticulous recording techniques. From attaching microphones to NASCAR vehicles to simulating crowd reactions in football stadiums, the attention to detail ensures players feel truly immersed. Notable innovations include advanced recording setups for capturing car exhausts and snowboarding sounds, bringing unparalleled realism to games like SSX.

Breaking into the Industry

For aspiring sound designers, Allen provided a wealth of advice on making a successful entry into the video game industry. Here are the key takeaways:

1. Master the Tools: Familiarise yourself with industry-standard tools such as Native Instruments’ Reaktor and Cycling ‘74’s Max. These platforms allow you to experiment with real-time sound manipulation and build a portfolio of interactive sound designs.
2. Learn the Fundamentals: A strong foundation in audio engineering and sound design is essential. This includes understanding concepts like dynamic layering, mixing for interactive environments, and creating adaptive soundscapes.
3. Gain Hands-On Experience: Seek opportunities to work on real-world projects. Whether it’s modding existing games, creating soundscapes for indie projects, or collaborating with other creatives, practical experience is invaluable.
4. Explore Internship Opportunities: Look for paid internship programmes in the video game industry that offer direct mentorship and hands-on involvement in game development. These opportunities provide invaluable real-world experience and a chance to network with industry professionals.
5. Stay Inspired and Persistent: Breaking into the industry requires passion and perseverance. Attend industry events, network with professionals, and never stop learning. Tools like Audio Kinetics’ Wwise can also help you gain a better understanding of runtime audio systems.
6. Build a Unique Portfolio: Stand out by showcasing your creativity and technical skills. Include examples of dynamic audio systems, interactive music compositions, and authentic sound recordings.

Final Thoughts

Jesse James Allen’s lecture offers a rare glimpse into the world of video game audio design. His passion, innovation, and dedication are a testament to the artistry that underpins EA Sports’ iconic games. For anyone dreaming of crafting soundscapes that captivate millions, this masterclass was an invaluable inspiration.

 

Audio in video games isn’t just about what you hear—it’s about how you feel. Rob Pearsall, an experienced sound designer and audio tester, provided a fascinating deep dive into the intricacies of audio testing for games. Drawing from his work on iconic franchises like Halo, he shared valuable lessons on what makes game audio tick and how to test it effectively.

What Makes Game Audio Unique?

Unlike films, where soundtracks sync directly with linear visuals, game audio is dynamic and interactive. It adapts to player actions and constantly shifts depending on the game environment. Rob emphasised this distinction, explaining how sound systems in games are programmed to respond to varying in-game conditions, creating a fully immersive auditory experience.

For example:

• Interactive Music: Games often use “stems” (individual tracks for different instruments) to allow for endless variations of a song. This ensures players rarely hear the same combination twice.
• 3D Soundscapes: Audio in games must convey spatial awareness. A helicopter passing overhead or a distant explosion must sound authentic, adjusting in volume, pitch, and filtering based on the player’s position.

Essential Skills for Audio Testing

Testing audio in games requires more than just a good ear. Rob highlighted two key areas of expertise:

1. Testing Proficiency: The fundamentals of identifying, documenting, and tracking issues are crucial.
2. Audio Knowledge: A professional understanding of sound design is indispensable. From mastering tools like Wwise and FMOD to comprehending audio behaviours like occlusion and attenuation, testers must be equipped to evaluate the nuances of game audio.

Challenges in Audio Testing

Rob shared several challenges he encountered while working on Halo:

• Volume and Memory Balancing: Managing the memory footprint of audio is critical. Overloading the system with sounds can lead to important effects—like gunfire—being cut off during gameplay.
• Consistency Across Game Modes: Audio transitions and behaviours can differ between campaign and multiplayer modes, requiring thorough testing in all scenarios.
• Environmental Sound Design: Ambient sounds and reverb must seamlessly match the environment. For instance, moving from a cave to an open field should feel natural to the player.

One particularly intriguing example involved reverb mismatches on vehicles. In one scenario, players riding in a Halo Warthog experienced reversed indoor and outdoor reverb effects depending on their position in the vehicle—a fascinating bug that required careful debugging to resolve.

Lessons from the Field

Rob’s anecdotes revealed just how intricate audio testing can be:

• Dynamic Sound Systems: Testing dialogue systems, which must adapt to various player interactions, requires a keen ear to ensure appropriate responses are triggered in the right contexts.
• Material Systems: Every surface in a game world—be it grass, metal, or water—affects the sounds of footsteps, gunfire, and interactions. Testing involves ensuring these sounds match their environments accurately.
• Loop Points: Ambient audio loops must blend seamlessly. A poorly looped sound can disrupt the immersion, drawing players out of the experience.

Tips for Aspiring Audio Testers

1. Build Relationships: Collaboration with designers and developers is crucial. Gaining their trust can improve feedback and streamline the debugging process.
2. Leverage Tools: Use profiling tools in middleware systems like Wwise or FMOD to isolate and test specific sounds or scenarios.
3. Study the Game: Understanding the lore and design principles of a game ensures consistency across sequels and expansions.

Final Thoughts

Rob concluded with a message of optimism: as game audio continues to evolve, so too will the tools and techniques used to create and test it. By fostering a partnership between creativity and technology, developers can craft unforgettable soundscapes that immerse players like never before.

For Rob, the ultimate goal is simple: make games fun and immersive. And with experts like him pushing the boundaries of audio, the future of gaming sounds brighter than ever.

Game Credits

Rob Pearsall has an extensive portfolio in game audio, contributing to over 50 titles including The Lamplighters League, Falcon Age, Necropolis, Destiny: The Taken King, Destiny, Halo 4, Halo: Combat Evolved Anniversary, and The Lord of the Rings. His company, Rip Street Audio, has played a significant role in delivering audio direction, music, and sound design for these projects.

 

In the world of video games, audio still is often an unsung hero, subtly enriching our gaming experiences. Yet, it is a complex and intricate field that demands expertise, creativity, and innovation. Saki Kaskamanidis, the late Audio Director at United Front Games, provided compelling insights into the art of audio direction for interactive entertainment, drawing from his experience on the critically acclaimed game Sleeping Dogs.

The Evolution of a Career in Game Audio

Saki’s journey into game audio was as inspiring as it was unique. Starting as a composer for Electronic Arts’ Need for Speed series, his lack of technical knowledge didn’t hinder him—musical skills opened the door. Over 18 years, he grew to master sound design, implementation, post-production, dialogue direction, and even scriptwriting. This diverse background laid the foundation for his role as Audio Director at United Front Games.

The Birth of Sleeping Dogs

Originally codenamed Blue Lotus, Sleeping Dogs was a bold attempt to challenge the dominance of open-world giants like Grand Theft Auto. Set in contemporary Hong Kong, the game’s narrative focused on an undercover cop, delivering a darker, more serious tone compared to its competitors. Despite setbacks, including being dropped by its original publisher, Activision, the project found a second life under Square Enix and became a critical success.

Audio: The Heartbeat of Interactive Storytelling

Saki emphasised that sound was an integral part of storytelling. “Sound and story are one,” he explained, underscoring the importance of integrating audio into every phase of production. Unlike film, video game audio had to adapt dynamically to player interactions, making it a unique challenge.

The Three Pillars of Game Audio

Saki categorised game audio into three pillars: sound design, music, and dialogue. Each played a crucial role in crafting immersive experiences:

1. Sound Design: From the visceral crunch of hand-to-hand combat to the roar of vehicle engines, sound design in Sleeping Dogs was meticulously planned. For instance, vehicle engine sounds were created using recordings of real cars, classified into 15 categories to streamline production.
2. Music: The score for Sleeping Dogs blended traditional Chinese instruments with Western elements, reflecting the game’s cultural setting. The dynamic score adapted to gameplay scenarios, while diegetic music added realism, playing from radios, boomboxes, and more.
3. Dialogue: Dialogue brought authenticity to Sleeping Dogs. Over 75,000 lines were recorded, with 45,000 making the final cut. To ensure cultural accuracy, ambient Cantonese dialogue was recorded on-site in Hong Kong, capturing the city’s vibrant soundscape.

Innovative Tools and Techniques

A key aspect of game audio was the use of middleware tools like Wwise and FMOD, which enabled dynamic soundscapes. For instance, the same explosion sound could vary in intensity and distance based on the player’s position. Such tools empowered designers to create immersive, context-sensitive audio experiences.

Challenges and Triumphs

Saki shared the challenges of creating convincing audio for vehicles, fighting, and gunplay. Early attempts at realistic fight sounds fell flat, leading to the creation of “hyped-up reality” effects inspired by films like The Bourne Ultimatum. Gun sounds required layered effects and innovative use of sound libraries to replicate the visceral punch of firearms.

The Future of Game Audio

Saki envisioned advancements in 3D audio, sound reflections, and occlusion. Technologies like binaural sound and improved computational power promised to elevate audio realism, enhancing player immersion. He believed these developments would bring unparalleled depth to gaming experiences, making them more lifelike than ever before.

Conclusion

Saki Kaskamanidis’ lecture illuminated the artistry and technical mastery behind game audio. From crafting dynamic soundscapes to recording authentic dialogue, his work on Sleeping Dogs exemplified how audio elevated storytelling and gameplay. For aspiring audio professionals, his journey was a testament to the power of passion, adaptability, and continuous learning.