Category: Audio Perception

  • Creating Sounds for Things We Cannot See: Kenny Young on VR, Music, and Guiding Attention

    Kenny Young

    Many forms of media depend upon controlling attention. Films decide where audiences look through editing, framing, and camera movement. Theatre guides attention through staging and movement. Conventional games frequently do something similar through interface design, visual effects, or camera behaviour. Important information rarely appears entirely by accident. Designers often decide where attention should go long before audiences realise those decisions are being made. Most of this guidance becomes invisible precisely when it works well. Players rarely stop to think about how often games quietly redirect their attention from one place towards another. Experiences simply feel natural. Objectives appear at appropriate moments, important events seem difficult to miss, and information arrives when required.

    Kenny Young’s guest lecture explored what happens once some of these assumptions begin disappearing. Virtual reality introduces a relatively simple change that gradually creates much larger consequences. Players control the camera continuously. Looking left means physically turning left. Looking upwards requires physically raising the head. Looking away from something important may simply mean missing it altogether. Initially this sounds like a relatively minor alteration, though the consequences begin spreading surprisingly far once control over attention starts shifting away from designers themselves.

    Imagine hearing something important happening behind you in a conventional game. Designers possess numerous methods for ensuring that players notice it. Cameras may shift automatically, indicators can appear around the screen, and control may even be briefly interrupted. Decades of game design have produced increasingly sophisticated methods for solving these problems. Virtual reality complicates many of these solutions. Fixed interface elements become intrusive, large overlays can weaken immersion, and information existing outside the player’s field of view can remain entirely unnoticed. Questions therefore begin emerging around how players discover important information once designers can no longer simply place it directly in front of them.

    Young suggested that sound changes role at precisely this point. Human vision behaves selectively. We actively choose where to direct our eyes and ignore much surrounding information. Hearing functions rather differently. Sounds continue arriving whether or not we intentionally seek them out. A player may choose not to look towards something important, though hearing something nearby can still trigger an immediate response. Sound therefore begins moving away from a supporting role attached to visible events and towards something more active.

    Players do not simply hear sounds in games. They gradually learn them. Initially a sound may exist only as another event occurring within a larger environment. Yet repeated exposure slowly changes its role. Through familiarity, sounds begin accumulating meaning. This process often happens without players consciously noticing it. A sound that initially appears neutral gradually becomes linked with expectations, actions, and outcomes. Eventually hearing the sound no longer involves interpreting something unfamiliar. Players instead recognise patterns they have already learned.

    Young discussed the familiar alert sound from Metal Gear Solid as an example. During early encounters players hear a brief cue alongside visual information, though repeated exposure gradually changes the relationship. Eventually players stop hearing the sound as a sound effect at all. Instead, it begins behaving almost like language.

    Language may not be entirely the right word, though the comparison becomes useful. Words themselves do not naturally contain meaning. People gradually learn relationships between sounds and ideas through repeated experience until recognition becomes almost immediate. Something similar begins happening within games. A short musical cue or brief sound effect acquires meaning through use rather than explanation. Players are not consciously translating sounds each time they hear them. Recognition simply becomes increasingly automatic.

    Nobody pauses a game to explain that a particular sound means danger. Players learn these relationships through repeated experience. Over time certain sounds become linked with expectations, actions, and outcomes until responses begin occurring almost automatically. Listening changes in subtle ways once these associations form. Players stop consciously analysing what they hear, as attention begins shifting before deliberate thought catches up. Sound therefore becomes something more than feedback occurring after an event. It starts creating expectations about what might happen next.

    Music introduces another layer to these learned relationships. Discussions around game music frequently focus on emotion, atmosphere, and immersion. Players may notice tension increasing during combat, emotional themes returning around familiar characters, or changing musical textures supporting movement through a world. Young explored another possibility entirely. Under certain conditions, music may also begin operating as information.

    Much of his work on Tethered explored whether these kinds of relationships could be developed within virtual reality environments. Strategy games already involve unusually large amounts of simultaneous information. Resources require management, environments continue changing, threats emerge unexpectedly, and events occur across multiple locations at once. Conventional interfaces frequently solve these problems visually. Players monitor maps, indicators, menus, and notifications distributed around the screen. Translating these expectations into VR introduced a more difficult question. How can players remain aware of a world once they can comfortably see only part of it at any given moment?

    Rather than functioning purely as atmosphere or emotional support, musical phrases could gradually become learned signals recognised through repeated interaction. Certain sounds became associated with changing conditions, important events, or emerging situations. Initially these sounds carried little meaning beyond existing as recognisable musical gestures. Over time something rather different happened. Players were not simply listening to music accompanying a world. They were gradually learning the world itself.

    Listening consequently begins developing an unusual relationship with navigation. Physical landmarks help people orient themselves within real environments, though players may also begin constructing sonic landmarks. Certain sounds become associated with places, behaviours, or changing conditions. Listening therefore starts becoming part of understanding how a world behaves. Particular musical phrases began functioning almost like landmarks within an environment. Certain combinations of sounds became associated with emerging threats or opportunities requiring attention. Over time players could respond before consciously thinking about what had changed. Listening therefore became intertwined with understanding the behaviour of the world itself.

    Examples such as these begin shifting the discussion slightly. Rather than asking whether music sounds appropriate or emotionally effective, another question begins appearing. How do people learn sonic environments? Under what circumstances do sounds stop behaving like sounds and begin behaving more like information? Underlying processes of this kind may already exist across many forms of game audio, even if virtual reality makes them easier to recognise.

    Extending these ideas into working systems introduced additional challenges. Sounds needed to remain distinctive while fitting comfortably alongside underlying music. Delays had to remain short enough that players still connected events with their causes, while multiple simultaneous events could create confusion or dissonance. Initial solutions often resolved one issue only to reveal another elsewhere. Technical constraints, musical decisions, and player behaviour continually interacted throughout development. Creative work therefore emerged less as a process of executing perfect ideas and more as a continual process of adjustment.

    Running throughout the lecture was a broader observation concerning the role of sound itself. Discussions surrounding game audio frequently emphasise realism, emotion, and atmosphere. These remain important concerns, though Young’s work suggested something slightly different. Once familiar methods for directing attention become less reliable, sound begins taking on responsibilities traditionally associated with cameras and interfaces.

    Virtual reality may therefore reveal something that has existed quietly within games for much longer. Sound has rarely functioned only as decoration or atmosphere. It has also shaped where players look, what they notice, and how they organise experiences around them.

    Perhaps the more interesting question is not whether sound helps players understand virtual worlds. It may instead involve asking how much of our experience has always depended upon sound guiding us in ways we barely notice. Once designers lose many familiar methods for directing attention, sound begins moving from the background towards the centre of interaction itself.

  • Beyond the Frequency Response: Dr Nick Zacharov on Why Sound Quality Refuses to Stay Simple

    Sound quality appears to be something that should be relatively easy to define. Modern audio engineering has become remarkably precise, allowing engineers to measure frequency response, distortion, sound pressure level, impulse response, and countless other properties with extraordinary accuracy. Pages of graphs and measurements can describe how an audio system behaves in minute detail. Looking at these increasingly sophisticated tools, it becomes tempting to assume that the problem has largely been solved. Better systems should produce better measurements, while increasingly detailed measurements should gradually lead towards better listening experiences.

    Yet most people who spend time with sound eventually encounter an uncomfortable contradiction. A new pair of headphones may arrive with impressive specifications and glowing reviews, promising exceptional clarity and technical accuracy. Everything appears correct on paper, yet after listening for a while something feels slightly wrong. Another pair with less impressive measurements somehow sounds more engaging, or perhaps two products that appear remarkably similar perform very differently in practice. Even more confusingly, people listening to exactly the same material can disagree entirely about what they hear. Experiences like these raise an interesting question. If sound can already be measured with extraordinary precision, why do we still need people to listen?

    This question formed the starting point of an online guest lecture delivered by Dr Nick Zacharov, whose work has spent more than three decades exploring sound quality and sensory evaluation across industries including telecommunications, professional audio, and product development. As co-founder of AudioSense Lab, his work focuses on understanding how people experience sound rather than simply measuring the physical properties of signals. Across the lecture, one idea gradually became increasingly clear: sound quality is not hidden solely within the signal itself. It also emerges through the relationship between sound and the people hearing it.

    Part of the challenge begins with assumptions about hearing itself. Measurement systems generally behave in predictable ways. Microphones can be calibrated, repeated measurements can produce highly consistent results, and instruments respond reliably under controlled conditions. Human hearing behaves rather differently. Rather than functioning as a neutral recording device, the auditory system continuously interprets incoming information before we consciously become aware of it.

    Zacharov described hearing as an extraordinarily sophisticated process operating across enormous ranges of frequencies and intensities. Sounds arriving from different directions interact with the shape of the head and ears before even reaching the inner ear, while loudness, timing, and spatial position all influence the information ultimately reaching the brain. Listening therefore involves more than passively receiving information from the outside world. The auditory system actively reconstructs what we hear, continually shaping experience rather than simply recording it. Measuring sound pressure level may therefore be relatively straightforward, though measuring how people actually experience sound quickly becomes much more complicated.

    This issue becomes clearer when considering the language people commonly use to describe audio experiences. Terms such as brightness, warmth, spaciousness, clarity, depth, and fullness often feel straightforward and intuitive, and most listeners immediately recognise what these ideas mean. Yet many of these qualities do not correspond directly to simple physical measurements. Loudness provides a useful example. Loudness is not merely a sound pressure value but a perceptual experience allowing listeners to organise sounds along a continuum extending from quiet to loud. Anyone who has increased the volume of a quiet dialogue scene in a film only to be startled moments later by a sudden explosion or a swelling piece of music has experienced this distinction directly.

    Similar relationships exist for many of the characteristics listeners use when evaluating sound systems. Spaciousness involves more than physical distance between sound sources, warmth cannot simply be reduced to a particular frequency range, and clarity often emerges through interactions between multiple factors rather than a single measurable value. Describing sound therefore becomes surprisingly complicated. People often use similar words while imagining different things. One listener’s idea of warmth may not correspond exactly to another person’s understanding of the same term. Researchers and designers therefore face the challenge of developing shared vocabularies that allow experiences to be discussed more consistently.

    This need for a common language has led researchers towards the development of perceptual descriptors and sensory lexicons. Rather than relying on vague impressions such as “good” or “bad”, listeners are encouraged to think in terms of more specific qualities that can be identified repeatedly. The aim is not simply to produce more words for describing sound. Instead, the goal is to create reliable ways of connecting subjective experiences with measurable characteristics. The question therefore shifts again. Rather than asking whether sound itself can be measured, attention moves towards understanding whether measurements adequately describe the experiences listeners actually care about.

    One of the most interesting ideas discussed during the lecture emerged through a distinction between preference and perception. Initially these concepts seem almost interchangeable. If somebody prefers one sound over another, it appears reasonable to assume that the answer provides everything necessary. Preference feels direct and uncomplicated. Yet preference quickly becomes more unstable than it initially appears. People notice different details, bring different expectations into listening environments, and respond differently depending on context. Prior experiences shape listening behaviour, while cultural backgrounds and individual habits influence interpretation. Preferences can also change over time, meaning two listeners hearing exactly the same material can arrive at entirely different conclusions.

    To explain this distinction, Zacharov introduced an example involving cheese. Imagine placing several cheeses in front of a group of people and simply asking which one they prefer. Most people would answer almost immediately. The process feels natural and instinctive. Yet the situation changes once different questions begin appearing. Which cheese feels creamier? Which one seems more acidic? Which has a stronger texture? Attention gradually moves away from simple preference and towards analysis. People begin thinking differently about the experience itself.

    Listening behaves in much the same way. Zacharov noted that if people are simply asked what they prefer, they often respond immediately and instinctively. Once listeners are instead asked to evaluate characteristics such as distortion, bass, or other specific attributes, something changes in the listening process itself.

    “If I ask people what they prefer, they will instantly tell you. If I ask them to evaluate distortion and bass and all of these different characteristics, they start thinking consciously about things.”

    What initially appears to be a relatively small distinction gradually becomes much more interesting. Once people become consciously aware of what they are listening for, their relationship with sound itself begins changing. They are no longer responding naturally in the same way they might during everyday listening. Instead, they begin examining specific characteristics and separating experiences into individual components. Listening effectively becomes analytical rather than instinctive.

    This distinction sits at the centre of sensory evaluation. Rather than asking people simply whether they like something, sensory evaluation attempts to understand how people perceive particular characteristics and why those characteristics influence experience. The goal is not merely to identify winners and losers but to understand the qualities shaping listening itself.

    Zacharov described how this often involves training listeners to recognise and describe perceptual attributes systematically. This process does not necessarily involve teaching people what they should hear. Instead, it focuses on developing consistency. Listeners learn to identify particular forms of distortion, tonal differences, changes in spatial presentation, or other relevant characteristics. Over time, they develop a shared vocabulary allowing listening experiences to be discussed with greater precision.

    Training becomes particularly important since untrained listeners often respond differently from experienced listeners. Someone listening casually may immediately focus on broad impressions such as whether a sound feels enjoyable or unpleasant, while trained listeners may identify subtle changes in bass response, timbral colouration, spatial width, or artefacts introduced by processing systems. Neither response is inherently better than the other, though they provide different forms of information. One reflects instinctive experience while the other provides analytical detail.

    These methods become particularly valuable within product development. Preference testing can identify whether people generally favour one system over another, though such tests often reveal relatively little about why decisions occur. A product may consistently outperform competitors while leaving important questions unanswered. What exactly are listeners responding to? Greater spaciousness? Reduced distortion? Increased clarity? Better balance?

    Sensory evaluation attempts to bridge this gap by identifying the characteristics influencing perception, allowing researchers and designers to understand not simply whether products succeed but why they succeed. These approaches have applications across a wide range of industries. Telecommunications systems aim to optimise speech quality and intelligibility. Headphone manufacturers seek desirable listening experiences across different musical styles. Automotive companies increasingly design not only engines and interiors but also the sonic experience of travelling within vehicles. Consumer technologies ranging from smart speakers to voice assistants similarly depend on understanding how people perceive sound rather than simply reproducing signals accurately.

    Towards the end of the lecture, another issue gradually emerged concerning the relationship between controlled testing environments and everyday listening experiences. Listening tests often take place under carefully designed conditions intended to isolate variables and remove distractions. Such environments are extremely useful for identifying subtle differences and maintaining consistency, yet real listening situations rarely behave in the same way.

    People do not spend their lives sitting silently in isolated rooms rapidly switching between competing systems. Sound exists alongside movement, conversations, expectations, distractions, and activities unfolding simultaneously. Headphones are used on trains and buses, music accompanies exercise and travel, and films are experienced within social environments rather than laboratories. A technically ideal system under laboratory conditions may therefore not necessarily produce the same experience within everyday contexts.

    This raises questions surrounding ecological validity, a concept concerned with how closely experimental conditions resemble real-world experiences. Zacharov reflected on this as an increasingly important direction for future work, suggesting that listening research has gradually begun moving towards broader and more realistic forms of evaluation.

    “I think there is a trend in going more holistic nowadays and going more ecological.”

    Running throughout the lecture was a wider point. Sound quality is not simply a technical problem waiting to be solved through increasingly detailed measurements. Measurements remain essential and enormously valuable, though they only describe part of the listening experience. Signals, technologies, environments, expectations, and listeners continuously interact with one another. Understanding sound therefore may involve more than measuring systems accurately. It may also require understanding the people hearing them.