Category: Interaction Design

  • How Do Robots Communicate Through Sound? Connor Moore on Audio UX, Robotics, and Designing Meaningful Interactions

    Connor Moore

    How do robots communicate through sound?

    People increasingly interact with technology through sound. Smartphones acknowledge completed payments, electric vehicles alert drivers to potential hazards, wearable devices provide subtle notifications and intelligent products communicate through a growing vocabulary of tones, chimes and alerts. Yet these sounds rarely receive the same attention as visual design. During his online guest lecture for Edinburgh Napier University, Connor Moore explored the growing discipline of Audio User Experience (Audio UX), demonstrating how carefully designed sounds help products communicate naturally, build trust and express personality. Drawing upon projects for companies including Google, Tesla and Postmates, he argued that successful product sound design extends far beyond creating attractive audio. It begins with understanding the people for whom they are designed. Throughout the session, one principle emerged repeatedly. Every sound should communicate with purpose.

    Moore introduced his work by describing the remarkable breadth of modern Audio UX. Working from California’s Bay Area, he collaborates with companies developing products across robotics, automotive technology, connected devices, consumer electronics and digital services. Although these industries appear very different, they all share a common challenge. Products increasingly communicate with people through sound, requiring designers to think carefully about what those sounds communicate and how they contribute to the wider identity of a brand. Rather than approaching each project as an isolated collection of sound effects, Moore described building coherent sonic systems that extend across products, marketing, physical environments and user interactions. Individual sounds matter, though they become most effective when they form part of a larger and recognisable design language.

    This broader perspective also explains why strategy sits at the beginning of every project rather than at the end. Before designing a single sound, his team seeks to understand the objectives of the product, the identity of the organisation and the experience that users should ultimately have. Brand workshops, creative discussions and detailed reviews of existing sounds all contribute towards this early stage of development. Competitor analysis also plays an important role. Understanding how other companies sound allows designers to identify opportunities for meaningful differentiation rather than unintentionally reproducing familiar ideas. The objective is not simply to sound different. It is to create a sonic identity that genuinely reflects the values and personality of the organisation. Sound therefore becomes a strategic design material rather than a decorative addition introduced once the product has already been completed.

    One of the most thought-provoking ideas introduced during the presentation concerned what Moore described as connected audio ecosystems. Many organisations continue to commission isolated sounds for individual products or services, yet users increasingly encounter the same company across multiple devices and environments. A person may hear a notification on a smartphone, interact with a smart speaker at home, use an in-car navigation system and later encounter advertising or public installations produced by the same organisation. Rather than allowing each experience to develop independently, Moore argued that they should all share recognisable sonic characteristics. Consistent instrumentation, similar timbral qualities and carefully related musical ideas allow users to recognise a brand without needing to see a logo or screen. Sound therefore becomes another component of brand identity, working alongside visual design to create familiarity and trust.

    Google’s product ecosystem provided one of the clearest illustrations of this philosophy. Moore described how his work began during the development of Google Glass, a product that sought to make an unfamiliar technology feel approachable. Rather than emphasising futuristic electronic sounds, the design drew upon simple acoustic instruments such as piano, chimes and mallet percussion. These familiar timbres helped ground an otherwise unfamiliar experience, making the product feel more human and intuitive. As Google’s product portfolio expanded through devices such as Pixel phones, Google Pay and automotive systems, this underlying sonic character evolved while remaining recognisably connected. Different products naturally demanded different technical solutions and frequency ranges, though the overall identity remained remarkably consistent. Moore argued that brands evolve in much the same way as people do. Their sonic identities should therefore develop over time while retaining a recognisable sense of continuity.

    Perhaps the most unexpected principle discussed during the session concerned silence. Designers often assume that every interaction requires another notification, another confirmation or another layer of feedback. Moore challenged this instinct directly. As products increasingly incorporate sound into everyday life, designers also acquire a responsibility not to make the world unnecessarily louder. Like negative space in graphic design, silence performs an important communicative function. It creates contrast, draws attention to genuinely important events and prevents users from becoming overwhelmed by constant auditory stimulation. Successful Audio UX therefore depends not only upon knowing which sounds should exist, but equally upon recognising which moments deserve silence instead.

    He illustrated this philosophy through the development of Sense, a sleep monitoring device designed to help users understand the quality of their sleep. Conventional alarm clocks often rely upon abrupt, attention-grabbing sounds that force people awake almost instantly. Moore saw an opportunity to rethink that experience entirely. Instead of beginning loudly, the alarms gradually evolved over time, introducing increasing musical complexity, richer timbres and subtle changes in tempo. Lighter sleepers could wake during the earliest stages, while heavier sleepers would gradually encounter a more energetic composition. Even error tones and voice interactions were designed using soft, restrained timbres that preserved the calm atmosphere of the bedroom rather than disrupting it. The project demonstrated that product sounds need not simply communicate efficiently. They can also influence the emotional quality of everyday experiences.

    Moore then introduced one of his central design philosophies: communicative and expressive design. Throughout the presentation, he repeatedly distinguished between creating sounds that merely reinforce a brand and creating sounds that genuinely help people understand what is happening. Branding undoubtedly matters, though communication always takes priority. Every sound should first convey meaning. Only then should it contribute towards a wider sonic identity. This perspective encourages designers to think carefully about urgency, expectation and human perception rather than treating every notification as another opportunity for creative expression. Product sounds exist to guide behaviour as much as they exist to establish identity.

    Tesla provided a particularly revealing case study. Moore described developing different categories of sounds according to the urgency of the information they needed to convey. Low-priority events, such as incoming calls, were designed to emerge gradually using softer timbres and lower levels of perceptual urgency. Medium-priority notifications, including seatbelt reminders, employed greater repetition and brighter timbres, encouraging users to respond without becoming unnecessarily stressful. High-priority warnings, including forward collision alerts, demanded a very different approach. Higher frequency content, more percussive attacks and rapid repetition ensured that these sounds immediately captured attention during situations where rapid action could prevent an accident. Rather than relying upon arbitrary aesthetic decisions, Moore demonstrated how pitch, repetition, harmonic content and timbre can all be manipulated systematically to communicate different levels of urgency. Sound becomes a carefully designed language through which products communicate urgency, intention and behaviour.

    By this stage, a clear philosophy had emerged. Audio UX is not simply concerned with creating pleasant sounds or memorable sonic logos. It asks how products should communicate with the people who use them every day. Strategy, branding, silence, musical structure and perceptual psychology all contribute towards that objective, though none of them represents the ultimate goal. Every design decision serves the relationship between people and technology. Once sound is understood as a form of communication rather than decoration, the challenge shifts from asking what a product should sound like to asking what it should say. That question became even more significant when Moore turned to the rapidly developing world of robotics.

    The second half of Moore’s presentation shifted from broad design principles towards a detailed case study that demonstrated how those ideas are applied in practice. The project centred on Serve, the autonomous delivery robot developed by Postmates. Rather than treating the robot simply as another product requiring notification sounds, Moore used it to explore a far broader question. How should an intelligent machine communicate with people as it moves through shared public spaces? The answer, he suggested, depends upon much more than selecting attractive sounds. It requires understanding personality, context, expectation and human behaviour long before the first sound is ever designed.

    Like every project discussed earlier in the presentation, the design process began with strategy rather than sound. Before any recording or composition took place, the team explored what the robot represented, how people would encounter it and the personality it should express. Several distinct sonic directions were developed around different interpretations of the brand before being refined through successive design reviews and evaluated within the robot itself. Moore emphasised that successful Audio UX develops through continual iteration rather than moments of inspiration. Sounds that appear convincing inside a studio may behave very differently once reproduced by a moving robot navigating busy streets, restaurants and crowded pavements. Testing therefore becomes an integral part of the creative process rather than simply a means of checking technical performance.

    One of the most revealing aspects of the project concerned personality. Popular culture has encouraged audiences to expect robots to communicate through futuristic electronic sounds or highly expressive synthetic voices. Moore deliberately avoided both extremes. The ambition was to create a robot that felt warm, approachable and reassuring without pretending to possess human intelligence or emotional awareness. At the same time, the team resisted the temptation to rely upon recorded speech, recognising that a natural voice would create expectations that the technology could not consistently fulfil. Instead, they searched for a middle ground in which sound suggested character without imitating humanity. This balance between familiarity and honesty reflected one of the most thoughtful ideas running throughout the presentation. Good Audio UX should communicate clearly without misleading users about a product’s capabilities.

    Developing that personality required exploration rather than immediate certainty. Moore described creating several contrasting sonic directions, each expressing a different interpretation of the robot’s identity. Some embraced more mechanical qualities that acknowledged the machine’s physical presence. Others explored vocal-like synthesis capable of suggesting expression without becoming literal speech. A third direction employed simple sine-wave tones that created a calmer, softer and more abstract character. Rather than choosing a favourite instinctively, these alternatives became prototypes through which designers could observe how people responded emotionally to different sonic identities. The final design combined warmth, clarity and subtle expressiveness, producing a robot that felt approachable without becoming theatrical or sentimental. The process illustrated an important principle: successful sound design rarely emerges fully formed. It develops through comparison, evaluation and refinement.

    Attention then shifted from the robot’s overall personality to the design of individual interactions. Different situations demanded different styles of communication. Interactions inside restaurants, where staff loaded deliveries into the robot, prioritised efficiency through short, direct auditory cues that confirmed actions without interrupting the workflow. Encounters with members of the public required a gentler approach. Longer note durations, more relaxed phrasing and softer musical gestures created the impression of patience rather than urgency. In effect, the robot adapted its acoustic behaviour according to the social environment in which it operated, much as people instinctively alter their own behaviour between professional and public settings.

    A particularly memorable example centred on one of the simplest interactions imaginable: saying, “Excuse me.” Rather than relying upon recorded speech, Moore developed a brief auditory gesture that politely attracted attention before allowing the robot to continue its journey. The intention was not to surprise pedestrians or demand an immediate response. Instead, the sound functioned more like a courteous acknowledgement of another person’s presence. This small interaction captured a principle that extended throughout the presentation. Effective communication often depends upon restraint rather than intensity. Products should seek attention only when attention genuinely needs to be given.

    Safety presented a different set of priorities. Warning sounds must communicate immediately and unambiguously, leaving little room for ambiguity or interpretation. Here, Moore returned to ideas introduced earlier in the presentation. Instead of inventing unfamiliar sonic languages, the design frequently drew upon acoustic references that people already understood from everyday experience. Turn indicators, movement cues and other operational sounds retained familiar characteristics while remaining consistent with the robot’s wider sonic identity. This reduced the need for users to learn entirely new sonic conventions. Familiar sounds could be interpreted almost instinctively, allowing people to respond appropriately without consciously analysing what they had heard.

    Perhaps the most technically demanding challenge involved the robot’s continuous movement through public space. Moore explored several possible solutions, including humming, whistling and slowly evolving tonal textures that he described as “glowing.” Each communicated the robot’s presence in a slightly different way. Some attracted attention more effectively, while others blended more comfortably into the surrounding soundscape. Extensive user testing, including sessions involving blind participants, revealed that restrained harmonic complexity and carefully controlled modulation proved more effective than more elaborate alternatives. Yet Moore resisted the temptation to increase the amount of sound simply to improve awareness. His longer-term ambition was quite the opposite. Intelligent products should become quieter rather than louder. If a robot recognises that nobody is nearby, there may be no need for it to produce sound at all.

    This idea provides a fitting conclusion to Moore’s broader philosophy of Audio UX. The discipline is not concerned with filling products with attractive sounds or memorable sonic logos. It asks how technology can communicate clearly, respectfully and appropriately with the people who use it. Whether designing for autonomous robots, electric vehicles, smartphones or medical devices, the same principles continue to apply. Strategy comes before implementation. Communication matters more than novelty. Personality should remain authentic. Silence deserves to be designed as carefully as sound itself. When those ideas come together successfully, sound ceases to be decoration and becomes an essential part of the conversation between people and technology.

  • How Much Sound Does a Game Really Need? Gaetan Troutet on Casual Games, Creative Restraint, and Designing for the Real World

    Gaetan Troutet

    How much sound does a game really need?

    Most players never notice the sounds that have been deliberately left out of a game. During his online guest lecture for Edinburgh Napier University, Gaetan Troutet suggested that this is often the hallmark of successful sound design. Creating an effective soundtrack is rarely about filling every moment with audio. It is about deciding what genuinely deserves to be heard. Drawing upon his work developing casual games for Global Eagle Entertainment, he demonstrated how technical limitations, player behaviour and careful editorial judgement shape almost every creative decision. A single principle underpinned the discussion. Successful sound design depends as much upon restraint as invention.

    The environment in which Troutet’s games are played makes these decisions particularly demanding. Unlike many commercial titles developed for dedicated gaming hardware, his work must function across a diverse collection of in-flight entertainment systems installed on aircraft across the world. Some platforms provide comparatively modern hardware with generous storage and processing resources. Others continue to rely upon considerably older systems whose limited memory and bandwidth require soundtracks to be simplified before they can be deployed. The same game may therefore exist in several different technical versions, each shaped by the capabilities of the hardware on which it will eventually run. Even then, the hardware represents only part of the challenge. Every passenger experiences the soundtrack differently. Some use the headphones supplied by the airline, others connect their own, while many later encounter the same games on mobile devices with entirely different loudspeakers. Unlike a cinema or recording studio, there is no single reference listening environment. Troutet suggested that professional sound designers should accept this uncertainty rather than attempting to eliminate it. The objective is not to produce a soundtrack that sounds perfect under ideal conditions. It is to create one that continues to communicate effectively wherever it is heard.

    Although the lecture centred upon casual games, the questions Troutet raised apply to sound design far more generally. Every project exists within practical constraints, whether they involve memory budgets, processing power, production schedules or playback systems. Rather than viewing these restrictions as obstacles to creativity, Troutet argued that they often encourage clearer thinking. Once every sound occupies valuable storage, competes for the listener’s attention and requires implementation within a functioning game, designers become far more selective about what truly matters. Working as the sole audio practitioner within his development team reinforces that perspective. Troutet moves continually between creating sound effects, composing music, recording dialogue, implementing assets and collaborating with programmers and designers. Rather than treating these activities as separate disciplines, he presented them as interconnected parts of a single design process. Creative decisions influence implementation, technical limitations shape artistic choices and production realities affect every stage of development. Sound design therefore becomes inseparable from the wider process of building the game itself.

    One of the most thought-provoking moments in the lecture centred upon what appears to be a deceptively simple question. When a player performs an action, should that action always produce a sound? Many beginning designers instinctively answer yes. Buttons receive clicks. Menus receive confirmation tones. Every movement, selection, reward and transition appears to justify another layer of feedback. Troutet challenged this assumption directly. Rather than asking which sounds could be added, he encouraged students to ask which sounds genuinely improved the experience. Every additional sound competes for the listener’s attention. Every new cue alters the perceived importance of those surrounding it. Audio that initially appears informative can rapidly become repetitive, distracting or simply exhausting when heard hundreds of times during repeated play. Casual games make this question particularly important. Players often return to them repeatedly in relatively short sessions. Sounds that seem satisfying during the first few minutes may become irritating after dozens of repetitions. Troutet therefore described restraint as an active design decision rather than the absence of creativity. Silence is not an empty space waiting to be filled. It forms part of the overall balance of the soundtrack. Choosing not to add a sound may ultimately improve clarity far more than creating another effect.

    These same principles become particularly apparent in interface design, where audio functions less as decoration than as communication. Troutet encouraged students to think of interface sounds as messages directed towards the player rather than ornamental additions to menus and buttons. A confirmation tone, warning signal or navigation sound should communicate its purpose immediately, allowing players to understand what has happened without continually consulting the screen. One particularly memorable suggestion involved imagining the interface without any graphics at all. If a player were blindfolded and heard only the sounds, could they still distinguish success from failure, confirmation from cancellation, or navigation from selection? If the answer is yes, then the sounds are performing a genuine communicative role. If not, making them louder or more elaborate is unlikely to solve the underlying problem. Rather than treating interface sounds as decorative clicks or beeps, Troutet encouraged students to think of them almost as a spoken language. Every sound should communicate intention. Players should recognise whether an action has succeeded, failed or requires further input without consciously analysing what they have heard. Well-designed interface audio reduces cognitive effort. The player understands first and reflects afterwards. In this sense, interface sounds become part of the conversation between the game and the player rather than simply another layer of feedback.

    The same philosophy shaped Troutet’s approach to creating collections of related sounds. Rather than treating every effect as an independent recording selected from unrelated libraries, he described building what he called families of sounds. Interface elements, gameplay feedback and recurring actions share common characteristics, creating a recognisable sonic vocabulary throughout the game. Individual sounds may differ substantially in pitch, duration or function, though they continue to feel as though they belong together. Players may never consciously analyse these relationships, yet they often perceive the overall soundtrack as more coherent and easier to understand. Creating these relationships frequently meant recording original material rather than relying exclusively upon commercial sound libraries. Library recordings remain valuable resources, though bespoke recordings provide greater flexibility when developing a consistent sonic identity. Variations can be created from common source material, preserving subtle similarities that would be difficult to achieve using unrelated recordings gathered from multiple collections. The objective is not originality for its own sake. It is to ensure that every sound contributes towards a coherent listening experience rather than drawing attention to itself as an isolated event.

    Troutet consistently returned to the relationship between player experience and design judgement. Recording equipment, software and implementation techniques remained important, though they were never presented as ends in themselves. Every technical decision ultimately served the same objective: helping players understand, navigate and enjoy the game. Sound design therefore became an exercise in editorial judgement rather than accumulation. The important question was no longer how another sound might be added, but whether that moment genuinely deserved sound at all. Once that decision becomes the starting point, implementation, iteration and refinement begin to look rather different, forming the focus of the remainder of the lecture.

    Implementation forms the natural continuation of Troutet’s argument. Once the decision has been made that a sound genuinely deserves to exist, another set of questions immediately follows. When should it play? Under what conditions should it remain silent? How should it respond when players behave in unexpected ways? Troutet encouraged students to recognise that creating an individual sound is only one stage of the design process. A carefully recorded asset can still fail if it appears at the wrong moment, masks more important information or becomes repetitive through excessive triggering. Implementation therefore becomes an extension of sound design rather than a separate technical activity. Decisions about timing, variation and behaviour shape the player’s experience just as profoundly as the recordings themselves. Very few sounds remain unchanged after their first implementation. Once assets begin interacting with graphics, gameplay and player behaviour, weaknesses quickly become apparent. Sounds that worked well in isolation may feel intrusive within the finished game. Others disappear beneath music or gameplay effects, while some simply occur too frequently. Rather than treating these discoveries as failures, Troutet presented them as an expected part of development. Every implementation reveals more about how players actually experience the game, allowing successive revisions to refine the soundtrack until it supports interaction naturally.

    This willingness to revise also requires a particular creative mindset. Troutet observed that sound designers often invest considerable effort in creating individual recordings, making it tempting to defend them once they have been completed. Professional practice frequently demands the opposite approach. If a sound distracts players, interrupts the pacing of the game or simply fails to communicate effectively, attachment to the recording itself becomes irrelevant. During the lecture he summarised this philosophy with a familiar expression from creative practice: kill your babies. The phrase may sound severe, though the principle behind it is straightforward. The success of the overall experience matters more than preserving individual ideas. Removing or replacing a favourite sound is sometimes the decision that allows the remainder of the soundtrack to function more effectively. The willingness to edit critically therefore becomes every bit as important as the ability to create new material.

    The same philosophy extends beyond individual recordings into collaboration with the wider development team. Troutet repeatedly emphasised that sound design does not develop independently from programming, art or game design. Audio practitioners inherit decisions made elsewhere while simultaneously influencing the work of others. Effective collaboration therefore depends upon communicating design decisions in terms of the player’s experience rather than purely technical language. Requests for additional implementation features, changes to interface behaviour or modifications to gameplay become far easier to justify when they are framed around what players will understand, notice or enjoy. Communication, in this sense, becomes another aspect of sound design rather than an administrative task surrounding it. Professional organisation supports that collaboration in equally practical ways. Clear file names, consistent project structures and carefully maintained asset libraries rarely receive the same attention as recording or mixing, yet they influence every subsequent stage of production. Projects evolve over months or years, assets require continual revision and other members of the team must be able to locate the correct material quickly. Well organised sessions reduce confusion, simplify implementation and ultimately create more opportunities for genuinely creative work.

    Troutet also cautioned against becoming overly attached to particular software, plug-ins or recording equipment. Digital audio workstations continue to evolve, new tools appear regularly and production techniques inevitably change across a career. These developments undoubtedly influence professional practice, though they remain only means of achieving a larger objective. The more important questions concern what the player should hear, what information deserves emphasis and how audio contributes to the overall experience of the game. The same perspective shaped his comments on sources of inspiration. Commercial sound libraries, films and existing games all provide valuable references, though they should never replace careful design thinking. A distinctive soundtrack emerges through the relationships between sounds, the pacing of interaction and a clear understanding of the audience rather than through the novelty of any individual recording. Troutet consistently returned to the idea that sound design is fundamentally a process of making informed decisions rather than collecting techniques.

    Troutet repeatedly argued that sound should guide interaction rather than compete with it. Audio may reward success, reinforce important actions or draw attention towards changing events, though it should rarely distract players from the activity itself. This philosophy connects directly to the earlier discussions of restraint, interface communication and coherent families of sounds. Every element of the soundtrack exists to support understanding. Once a sound begins attracting attention to itself rather than to the player’s experience, its purpose deserves to be questioned. The measure of successful sound design is therefore not how much audio has been added to a game, but whether every element continues to justify its presence through the experience it creates for the player.

    The lecture concluded by returning, implicitly, to the same deceptively simple question that had shaped the discussion from the beginning. How much sound does a game really need? Troutet offered no universal formula. Different genres, audiences and platforms inevitably require different solutions. Instead, he encouraged students to replace assumptions with judgement. Does this sound communicate something important? Does it improve the player’s understanding? Does it strengthen the overall experience? If the answer is no, then adding more audio is unlikely to solve the problem. Careful omission often represents a stronger design decision than continual addition. Across examples ranging from airline entertainment systems to interface design, implementation and professional collaboration, Troutet consistently presented sound design as an exercise in thoughtful selection. The defining characteristic is judgement. Choosing which sounds deserve to exist, how they relate to one another and when they should remain silent requires an understanding of perception, interaction and communication that extends far beyond recording individual effects. Successful sound design is therefore measured not by the quantity of sounds within a project, but by how effectively those sounds help players understand, navigate and enjoy the worlds they inhabit.

  • How Do You Design Great Sound for Terrible Speakers? Tracy Bush on Creative Constraints, Game Audio, and Designing for the Real World

    Tracy Bush

    How do you design great sound for terrible speakers?

    Modern games present players with remarkably convincing sonic worlds. Dialogue responds naturally to changing situations, environments feel alive with movement and atmosphere, interfaces communicate information almost instinctively, and music adapts to the pace of play. Looking at contemporary productions, it is easy to imagine that these achievements are primarily the result of increasingly powerful technology. During his online guest lecture for Edinburgh Napier University, Tracy Bush suggested something rather different. Drawing upon a career that has included Blizzard Entertainment, Sony Online Entertainment, NCSoft and Sphero, he described how some of the most effective sound design emerges when technology imposes severe limitations. Small memories, limited processors, unpredictable playback systems and tiny loudspeakers do not simply restrict creativity. They force designers to think more carefully about what listeners genuinely need to hear.

    Bush’s own career reflected the rapid evolution of the games industry itself. Music had always formed an important part of his life, though his professional background began in information technology rather than audio. While working during the day, he spent evenings performing as a pianist in bars around San Francisco. After relocating to southern California, he joined Blizzard Entertainment in an IT role. His musical interests gradually became known throughout the company, leading colleagues to involve him in audio work whenever opportunities arose. Rather than following a carefully planned route into game sound, his career developed through a willingness to solve unfamiliar problems wherever they appeared. Looking back, Bush suggested that many people entered the industry in much the same way. Studios were small, responsibilities overlapped, and individuals frequently discovered new specialisms simply by becoming the person willing to tackle the next challenge.

    The games industry of the late 1990s differed substantially from the one students encounter today. Development teams were comparatively small, production pipelines remained fluid and many working practices were still evolving. Audio departments often worked alongside programmers, artists and designers in highly collaborative environments where formal boundaries between disciplines were less rigid than they later became. Bush described an atmosphere in which experimentation emerged naturally from everyday work. New hardware appeared rapidly, production tools changed continuously and every project seemed to introduce another set of technical problems that required fresh solutions. Experience remained valuable, though it rarely eliminated uncertainty.

    The computers on which players experienced those games introduced another level of unpredictability. Audio hardware varied enormously between systems, making consistent playback almost impossible to guarantee. Different sound cards reproduced music in noticeably different ways, while MIDI playback depended heavily upon whichever synthesis hardware happened to be installed inside an individual computer. A carefully balanced piece of music created inside the studio might sound dramatically different once it reached somebody else’s machine. Sound designers could control what left the studio. They could not control how it would ultimately be heard.

    This uncertainty extended well beyond music. Dialogue, sound effects and ambience all passed through hardware whose behaviour remained largely outside the control of the development team. Rather than designing for one predictable playback system, audio professionals found themselves designing for thousands of possible listening environments. Bush described this as one of the defining characteristics of early game audio. The question was rarely how a soundtrack sounded under ideal conditions. Instead, designers learned to ask whether it continued to communicate effectively when reproduced by equipment they had never encountered. The playback system itself became part of the design problem.

    Although contemporary technology has advanced enormously, the underlying challenge remains surprisingly familiar. Players now experience games through televisions, headphones, laptops, handheld consoles, mobile phones and increasingly varied listening environments, each introducing its own acoustic character. Perfect consistency remains elusive. The responsibility of the sound designer therefore extends beyond producing interesting sounds. It includes anticipating how those sounds will survive the journey from the studio to the listener.

    Bush also reflected upon the rapid transformation of production tools during this period. Early editing systems offered comparatively limited support for assembling large projects, requiring significant manual organisation and making complex revisions both time-consuming and potentially destructive. The arrival of Pro Tools transformed those workflows, allowing audio teams to edit non-destructively, manage increasingly complex sessions and collaborate more effectively. At much the same time, improvements in virtual sampling gave composers access to increasingly expressive orchestral sounds without requiring every revision to involve live performers. These developments expanded what small audio teams could realistically achieve while allowing creative ideas to evolve throughout production rather than becoming fixed at an early stage.

    The tools available to sound designers evolved just as quickly. Bush described middleware as another important step in that development. As implementation systems became more sophisticated, audio teams gradually assumed greater responsibility for how sounds behaved inside games rather than simply supplying recordings for programmers to trigger. Interactive playback, transitions and behavioural logic increasingly became part of the sound designer’s creative role. Technology expanded the possibilities available to audio departments, though it also broadened their responsibilities. Understanding implementation became almost as important as creating the sounds themselves.

    One observation from Bush’s time at Blizzard challenged another common assumption about technological progress. Greater technical capability did not necessarily encourage increasingly elaborate soundtracks. He reflected upon how musical direction gradually changed across successive projects, with later productions often favouring greater restraint rather than greater complexity. Earlier scores frequently relied upon dense orchestral textures intended to create scale and spectacle. Later work often achieved stronger dramatic results through simpler arrangements that allowed individual musical ideas greater space to breathe. Rather than filling every available moment with sound, composers became increasingly selective about where music should lead the player’s attention and where silence or restraint could prove more effective.

    The same principle appeared throughout sound design more generally. Memory budgets restricted how many sounds could be stored. Processor limitations reduced the number that could play simultaneously. Dialogue budgets limited the amount of recorded speech available to designers. Every technical restriction demanded choices. Which sounds genuinely communicated useful information? Which could be simplified without affecting the player’s experience? Which details would most influence the way a moment was perceived? Bush’s examples repeatedly suggested that successful sound design depends less upon including everything that is technically possible than upon identifying what is genuinely important for the listener.

    By this stage of the lecture, the discussion had established a way of thinking that extended well beyond the technology of any particular decade. New hardware, new software and new production methods continually alter the practical challenges facing sound designers, yet they rarely change the underlying task. Every project begins with a listener, a playback system and a collection of technical constraints that cannot simply be ignored. The role of the sound designer is to understand those conditions and create the most convincing experience possible within them.

    The relationship between creativity and constraint became considerably more tangible during Bush’s work with Sphero, where many of the assumptions underlying conventional game audio no longer applied. Working on licensed products featuring characters such as R2-D2, BB-8 and Lightning McQueen involved far more than transferring familiar techniques onto a different platform. Every sound would eventually emerge from a miniature loudspeaker housed inside a compact plastic enclosure containing motors, batteries, gears and electronic components. The finished product would be heard in kitchens, classrooms, living rooms and gardens rather than through carefully positioned studio monitors or high-quality headphones. Under those conditions, many established production practices simply ceased to be useful. The question was no longer how a sound performed inside the studio. It became how that sound survived once it reached the device for which it had actually been designed.

    Bush described changing his workflow to reflect that reality. Rather than completing the sound design and then testing it on the finished hardware, he monitored much of his work directly through the loudspeaker installed inside the product itself. Equalisation, dynamics, tonal balance and overall character were judged using exactly the same hardware that customers would eventually hear. The acoustic behaviour of the enclosure, the resonances introduced by the plastic casing and even the mechanical sounds generated by the internal motors became part of the design process. Instead of treating these characteristics as defects to be corrected afterwards, they became factors that shaped creative decisions from the beginning.

    The approach illustrates an important principle that extends well beyond embedded devices. Playback systems are never neutral. Every loudspeaker, pair of headphones, television or mobile phone colours the material passing through it. Sound designers often devote considerable attention to recording, editing and mixing, though the listening environment ultimately contributes just as much to the audience’s experience. Bush repeatedly returned to the importance of understanding where sounds will actually be heard. A design that performs beautifully on large studio monitors may communicate surprisingly little through the hardware used by most listeners. Successful sound design therefore depends not only upon creating interesting sounds, but also upon understanding the conditions under which those sounds will be experienced.

    Tiny loudspeakers presented another unavoidable challenge. Their physical dimensions simply prevented them from reproducing deep bass with any real authority. Attempting to force low frequencies through such hardware produced distortion long before it created convincing weight. Rather than attempting to overcome those physical limitations directly, Bush exploited the way listeners perceive sound. By introducing carefully controlled upper harmonics, he encouraged the auditory system to infer the presence of frequencies that the loudspeaker itself could not reproduce. The hardware remained unchanged, though the listening experience became noticeably richer.

    The solution depended upon psychoacoustics rather than brute force. Human hearing does not operate as a simple measuring device. Listeners continually reconstruct incomplete information, using harmonic relationships, timing cues and previous experience to build coherent auditory impressions. Bush’s work demonstrated how understanding those perceptual processes can prove more valuable than pursuing technically impossible specifications. The objective was never to reproduce frequencies that the loudspeaker could not generate. It was to create a convincing impression of fullness using the resources that remained available. Throughout the lecture, this distinction emerged repeatedly. Good sound design often depends less upon reproducing reality perfectly than upon understanding how listeners interpret what they hear.

    Sampling rates introduced another practical compromise. Embedded devices offered only a fraction of the storage and processing power available to contemporary games, requiring careful management of bandwidth and memory. Bush explained that these restrictions became particularly noticeable when working with robotic characters such as R2-D2, whose personality depends upon bright electronic vocalisations occupying the upper regions of the frequency spectrum. Lower sampling rates inevitably reduced the highest frequencies that could be reproduced accurately, making filtering and careful spectral management essential parts of the design process. Concepts that students often encounter as digital audio theory became everyday creative decisions affecting how expressive and recognisable the finished character would become.

    The material supplied by Lucasfilm also revealed how much organisation underpins apparently effortless performances. Bush did not receive complete scenes or finished sequences ready to be inserted into the product. Instead, he worked with an extensive collection of individual R2-D2 vocalisations drawn from the films. These recordings were not simply organised according to pitch or duration. Their emotional character proved considerably more important. Expressions of curiosity, excitement, concern, frustration and amusement were grouped together so that the robot’s responses could reflect changing situations while remaining faithful to the personality audiences already recognised.

    Randomisation played an important role, though not in the simplistic sense of allowing any sound to play at any time. Bush described carefully controlled systems that introduced variation without sacrificing recognisability. Human listeners identify repeated patterns remarkably quickly, yet behaviour that appears completely unpredictable can feel equally artificial. Convincing interactive audio therefore occupies a position between repetition and randomness. Familiar vocalisations return often enough to establish character, while subtle variations prevent those repetitions from becoming mechanical. The objective is not to surprise the listener continually, but to create the impression of a responsive and expressive personality.

    The same balance appears throughout interactive sound design. Footsteps, interface sounds, environmental ambiences and weapon effects all benefit from controlled variation rather than unlimited randomness. Collections of related recordings, small differences in pitch or timing and carefully managed playback logic often produce more convincing results than vast libraries of unrelated sounds. Bush’s examples demonstrated that believable behaviour frequently depends upon the relationships between sounds rather than the number of sounds available.

    As the lecture broadened beyond embedded devices, Bush argued that creating individual sounds represents only one part of a modern sound designer’s role. Interactive media introduces challenges that simply do not exist in linear forms such as film or television. A film editor knows exactly when every line of dialogue will be heard and how every scene will unfold. Games surrender much of that control to the player. Conversations may begin unexpectedly, be interrupted, or never occur at all. Players may spend hours exploring one environment while another moves through it in minutes. The soundtrack therefore cannot be constructed as a fixed sequence of events. It has to respond continuously to changing circumstances.

    Middleware transformed this aspect of production. Earlier generations of game development relied heavily upon programmers to implement even relatively modest audio behaviour. As middleware matured, sound designers gained much greater control over how sounds responded to events within the game itself. Playback logic, transitions, priorities and interactive behaviours increasingly became part of the sound designer’s creative responsibility. Recording remained an important part of the job, though implementation became equally significant. Designing how sounds behave proved just as important as designing the sounds themselves.

    This shift also changed the relationship between audio departments and the wider development team. Bush repeatedly emphasised that sound design does not exist in isolation. Programmers determine what information becomes available. Designers establish the systems that govern player behaviour. Writers shape dialogue, animators influence timing and movement, while artists define the visual environments within which sounds operate. Audio departments respond to all of these decisions while contributing their own expertise in return. Successful interactive soundtracks emerge through continual collaboration rather than from any single discipline working independently.

    One discussion during the lecture addressed the way sound professionals are perceived within development teams. Bush reflected on labels such as “the sound guy” or “the noise boy”, expressions that dramatically underestimate the breadth of contemporary audio practice. Modern sound designers contribute far beyond the creation of individual sound effects. They solve technical problems, shape interactive behaviour, collaborate across disciplines and influence how players ultimately experience the game. Titles such as Audio Director acknowledge that broader creative and technical responsibility.

    Questions from students later turned towards virtual reality, where many of these relationships become even more apparent. Convincing virtual environments depend upon much more than visual realism. Sound provides continuous information about distance, movement, scale and spatial relationships, allowing users to build coherent mental models of spaces extending beyond their immediate field of view. Carefully designed spatial audio therefore contributes directly to presence, orientation and immersion rather than acting as a decorative addition to the visual experience.

    Across subjects as varied as desktop games, embedded devices, robotic toys and virtual reality, Bush repeatedly returned to the same way of thinking. Every project began with an understanding of the available technology, the listening conditions and the perceptual abilities of the audience. The hardware changed dramatically throughout his career, though the questions facing the sound designer remained remarkably consistent. Rather than asking how to exploit every available technical capability, Bush continually asked what listeners actually needed to hear and how the available technology could communicate that experience most effectively.

    Across projects as different as Blizzard’s games, Sphero’s robotic products and emerging virtual reality systems, Bush consistently returned to the same set of design questions. Technology continued to change throughout his career, introducing new platforms, workflows and constraints, yet the underlying task remained remarkably stable. Successful sound design depended upon understanding how people listen, how technology behaves and how creative decisions bridge the gap between the two. Whether working with a full orchestral score, an interactive dialogue system or a miniature loudspeaker inside a robotic toy, the objective was never simply to produce impressive sounds. It was to create listening experiences that remained convincing under the conditions in which they would actually be heard.

  • How Can Sound Become an Interface? Professor Stephen Brewster on Non-Speech Audio, Multimodal Interaction, and Designing Beyond the Screen

    Professor Stephen Brewster

    What happens when looking at a screen is no longer the best option?

    Computing has become increasingly mobile. Phones accompany people through cities, workplaces, public transport systems, shops, festivals, and countless other environments. Yet much interaction design still assumes that users can devote their attention to a display whenever information needs to be communicated. During his online guest lecture for Edinburgh Napier University, Professor Stephen Brewster challenged that assumption. Drawing on decades of research in human-computer interaction, multimodal interfaces, auditory displays, sonification, and mobile computing, he explored a deceptively simple question. What happens when information is communicated through sound rather than vision?

    Brewster began by situating the discussion within a broader problem. Human beings possess multiple senses, though much digital technology continues to privilege vision above all others. Screens dominate contemporary computing. Menus, notifications, progress indicators, maps, messages, and data visualisations typically assume that users are willing and able to look. Yet many situations challenge this assumption. Someone cycling through traffic cannot continuously monitor a display. A pedestrian navigating a crowded city may already be dividing attention between multiple tasks. Bright sunlight can render screens difficult to read. Some users experience visual impairments. Others simply have more pressing demands on their attention than a device in their hand. Rather than treating these situations as exceptions, Brewster suggested they reveal a limitation in conventional interface design. If visual attention is unavailable, how else might information be communicated?

    This question has shaped much of his research. Rather than viewing sound as decoration or enhancement, Brewster approaches it as a communication channel. Sound can operate while users look elsewhere. It can communicate information rapidly. It can support accessibility. It can function alongside vision rather than competing with it. The goal is not to replace screens entirely. Instead, it is to make fuller use of the sensory capabilities people already possess. Multimodal interaction, as Brewster described it, involves designing systems that acknowledge how people actually experience the world rather than assuming that vision should always dominate.

    Mobile devices provided an especially important motivation throughout the lecture. Traditional desktop computing emerged within relatively controlled environments. Users sat at desks, faced screens, and focused primarily on a single task. Mobile computing transformed those assumptions. People now interact with technology while moving through complex environments filled with competing demands upon their attention. A larger display cannot solve every problem. In many situations, the challenge is not the quantity of visual information available. The challenge is finding ways to communicate information without requiring users to look at all. Brewster argued that interaction design should respond to these realities rather than simply shrinking desktop interfaces onto smaller screens.

    Attention emerged as a recurring concern throughout the lecture. Many interface designs implicitly assume that information should compete for attention whenever it becomes available. Notifications flash. Windows appear. Alerts demand immediate responses. Yet everyday life rarely operates in this way. People constantly manage multiple streams of information simultaneously. Conversations continue while walking. Music plays while working. Environmental sounds remain present while attention shifts elsewhere. Brewster’s work asks whether digital systems might learn from these patterns. Rather than repeatedly interrupting users, could information move fluidly between foreground and background depending upon circumstances? Sound appears particularly well suited to this challenge. Unlike visual displays, which generally require direct attention, auditory information can remain available while users focus elsewhere. The question is not simply whether sound can communicate information. It is whether sound can communicate information without constantly demanding attention.

    One reason sound becomes attractive in this context is its efficiency. Speech can communicate detailed information, though it requires time. A spoken message unfolds word by word. Non-speech audio can often communicate information much more rapidly. Brewster compared the relationship between speech and non-speech sound to the relationship between text and icons. A paragraph may describe an object in detail. An icon can often communicate a similar idea almost instantly. Carefully designed sounds can function in much the same way. Rather than reading information aloud, they communicate status, warnings, activity, trends, or relationships through concise auditory cues.

    Much of the lecture explored different approaches to designing these cues. One of the earliest involved earcons, structured auditory messages built from musical elements such as rhythm, pitch, timbre, and tempo. Unlike everyday sounds, earcons are abstract. Their meaning must be learned. Yet this abstraction also provides flexibility. Brewster demonstrated how simple auditory components can be combined to create larger structures capable of communicating increasingly complex information. A particular rhythm might signal an error. Changes in timbre or pitch might identify different categories of error. Much like language, the system develops a vocabulary from smaller building blocks. Users invest effort in learning the code, though once acquired it can support sophisticated communication through relatively simple sounds.

    Auditory icons take a rather different approach. Instead of relying upon abstract structures, they exploit familiar sounds drawn from everyday experience. Brewster discussed William Gaver’s influential SonicFinder project, which mapped computer operations onto recognisable environmental sounds. Selecting a folder might produce the sound of paper. Dragging an object across the desktop might generate a scraping sound. Deleting a file might end with breaking glass. Such sounds often require little training. Their meaning emerges from existing associations. Yet the approach also reveals interesting limitations. Everyday life contains only a finite number of obvious metaphors. As software functions become more specialised, finding intuitive sonic equivalents becomes increasingly difficult. What sound represents copying a file rather than moving it? What sound represents a menu hierarchy? Questions such as these expose the challenges that emerge when designers depend upon metaphor alone.

    A third approach, sonification, shifts attention away from interfaces and towards data. Here, numerical values are mapped onto auditory parameters such as pitch, rhythm, or timbre. Brewster compared the process to visualisation. Graphs provide rapid access to patterns that would be difficult to identify within tables of numbers. Sonification attempts to achieve something similar through listening. By converting data into sound, listeners can often identify trends, anomalies, peaks, and relationships that might otherwise remain hidden. Rather than replacing detailed numerical information, sonification provides an overview. It allows users to perceive the broader shape of a dataset before examining specific values.

    Questions from students helped illuminate this distinction further. One example involved pollen data transformed into sound through changing pitches. The goal was not to communicate precise measurements. Instead, listeners could quickly identify whether levels were increasing, decreasing, or remaining stable. Brewster argued that this reflects the real strength of sonification. A graph rarely succeeds solely through precision. It succeeds by revealing patterns. Sonification can achieve a similar outcome through auditory perception. Numerical detail remains available when required, though sound offers a rapid way of monitoring change over time.

    Several studies discussed during the lecture demonstrated how even relatively simple sounds can influence interaction. One experiment examined numerical data entry on mobile devices. Participants entered information using either large visual buttons or substantially smaller alternatives. Predictably, performance declined when the buttons became smaller. Yet when simple auditory feedback was added, performance improved dramatically. Users working with the smaller controls performed almost as well as those using larger buttons. The sounds themselves were uncomplicated. Their value lay not in complexity but in the additional information they provided. By reducing uncertainty during interaction, they made the task easier to perform.

    Another particularly elegant example involved progress indicators. Most software communicates progress visually through bars that gradually fill across a display. Brewster and colleagues explored whether similar information could be represented spatially through sound. As a task progressed, a sound moved around the listener’s head. Position communicated completion. Movement communicated change. Without looking at a screen, users could estimate how far a process had progressed and whether activity had stalled.

    During the discussion period, students questioned whether such displays might become intrusive. Brewster responded by drawing attention to forms of ambient awareness that already exist within everyday life. People rarely focus continuously on air-conditioning systems, distant traffic, rainfall, or background conversations. Such sounds remain available without demanding constant attention. Auditory displays, he suggested, can function in a similar way. Information remains present when required, fading into the background when it is not. This idea runs through much of his research. Sound is not always most effective when it occupies the foreground. Sometimes its greatest strength lies in supporting awareness without interruption.

    Spatial audio appeared repeatedly throughout the lecture as a particularly rich area for exploration. Rather than treating sound as something emitted from a single speaker, Brewster investigated how information might be organised around listeners in three-dimensional space. Progress indicators could move around the head. Calendar entries could occupy positions corresponding to times of day. Menu items could exist within an auditory environment rather than a visual one. These systems exploit the human ability to localise sound sources, transforming listening into a form of navigation. Information acquires location. Interaction becomes spatial rather than purely symbolic.

    Some of the most imaginative projects discussed during the lecture extended these principles into everyday environments. AudioFeeds transformed social media activity into ambient soundscapes. Twitter, Facebook, news feeds, and other information streams occupied different locations within auditory space, represented through distinct families of sounds. Rather than repeatedly checking a screen, users could maintain a broader awareness of activity through listening. Detailed information remained available when required, though constant checking became unnecessary.

    The significance of AudioFeeds extends beyond social media. The project raises broader questions about how digital information should occupy everyday life. Many contemporary systems assume that awareness requires direct inspection. Brewster’s work suggests alternatives. Awareness may emerge gradually. Information may remain peripheral until circumstances make it relevant. In this respect, auditory displays resemble many naturally occurring environmental sounds. People rarely monitor rainfall continuously, though they remain aware that it is raining. They do not focus constantly on traffic outside a window, though they often notice when conditions change. Sound supports forms of awareness that differ from the all-or-nothing relationship often associated with visual attention.

    Pulse extended these ideas into urban environments. During the Edinburgh Festival, geolocated tweets became spatial audio cues distributed around the city. The project transformed social activity into something that could be heard rather than viewed. Participants were not presented with lists of events ranked by popularity, nor were they required to consult maps repeatedly. Instead, they developed a sense of where activity was occurring through listening.

    One of the most interesting aspects of the project is that it occupied a space between navigation and exploration. Traditional navigation systems attempt to guide users towards predetermined destinations. Pulse encouraged discovery instead. Participants moved towards sounds that suggested activity, curiosity, or interest. Information became something encountered rather than simply retrieved. In doing so, the project demonstrated how auditory displays can support forms of engagement that differ substantially from conventional graphical interfaces.

    The lecture concluded with one of Brewster’s more recent ideas: musicons. Earcons require designers to construct sounds from scratch. Musicons instead draw upon music that listeners already know. Research revealed that people consistently identify particular moments within familiar songs as especially representative. Often these moments involve vocals, choruses, or distinctive melodic features. By extracting such fragments, it becomes possible to create recognisable auditory cues from a user’s existing music collection. The appeal lies partly in familiarity. Rather than learning a completely new auditory language, users rely upon associations they already possess. Recognition emerges from memory rather than training.

    Musicons reveal another recurring theme in Brewster’s work. Successful interfaces rarely begin from technology alone. They begin from existing human abilities. Earcons ask users to learn a new auditory language. Musicons exploit knowledge that listeners already possess. A few notes from a familiar song may be recognised almost instantly. Years of listening experience become part of the interface itself.

    Looking across the different projects discussed during the lecture, it becomes clear that Brewster is addressing a much larger question than how to design better sounds. The deeper issue concerns the relationship between people and technology. Modern computing frequently competes with the surrounding world for attention. Screens draw the eye away from streets, conversations, environments, and other people. Brewster’s work suggests that alternative relationships may be possible.

    Sound occupies a distinctive position within this discussion. It can communicate information while allowing users to continue looking elsewhere. It can support awareness without requiring constant inspection. It can reveal patterns within data, provide feedback during interaction, and create new forms of accessibility. Most importantly, it can coexist with other activities rather than replacing them.

    None of this means that sound should replace vision. Brewster repeatedly emphasised the value of multimodal design rather than sensory competition. Different senses possess different strengths. The challenge for interaction designers is understanding how those strengths can complement one another. Sound becomes most useful not when it attempts to imitate visual displays, but when it contributes capabilities that vision alone cannot easily provide.

    For many people, digital interaction has become almost synonymous with looking at screens. Brewster’s lecture offered a reminder that computing does not need to be confined to vision. Human beings hear, touch, move, and orient themselves within space. Designing for those abilities opens possibilities that extend far beyond the display. In that sense, the lecture was not really about sound alone. It was about recognising the full range of ways people experience the world.

  • Why Is Data So Quiet? Hugh McGrory on Sonification, Accessibility, and the Future of Information

    Hugh McGrory

    Why is data so quiet?

    Modern life is shaped by data. Governments collect it. Businesses depend upon it. Scientists analyse it. Social media platforms generate vast quantities of it every second. Increasingly, decisions about healthcare, transportation, education, finance, climate, and public policy are informed by information that exists primarily as data. Yet despite its growing importance, most people encounter data in remarkably similar ways. We see charts, graphs, dashboards, spreadsheets, maps, and visualisations. We are expected to look at information rather than listen to it.

    During his online guest lecture for Edinburgh Napier University, Hugh McGrory challenged this assumption. Drawing upon a career that has spanned animation, virtual reality, software development, data storytelling, and sonification, McGrory described a field that sits between sound, design, accessibility, and communication. Across a wide-ranging discussion that moved from GPS navigation to astronomy, podcasting, climate data, artificial intelligence, and urban infrastructure, he repeatedly returned to a deceptively simple question. If data is everywhere, why do we still experience almost all of it through our eyes?

    McGrory’s route into sonification was anything but conventional. Beginning in computer animation and experimental digital media, he later worked with medical imaging researchers at Yale University, where he encountered scientific data in a completely new way. Rather than using cameras to create images, researchers were transforming data into visual representations that scientists could study and interpret. Later work in virtual reality continued this fascination with information and how people interact with it. Yet throughout these experiences, one issue kept resurfacing. Data communication was overwhelmingly visual. Even the language reflects this bias. The field is known as data visualisation. Information is generally assumed to become meaningful once it has been converted into something that can be seen.

    For McGrory, this raises an obvious question. Why should vision carry so much of the burden? The field of sonification attempts to address this imbalance by exploring how information can be communicated through sound. Yet McGrory encouraged students to think beyond narrow academic definitions. The challenge is not simply turning data into audio. The challenge is deciding when sound might be a more useful way of communicating information. GPS navigation provides a useful example. Rather than forcing drivers to consult maps continuously, navigation systems deliver information precisely when it becomes relevant. A driver does not need every detail about the surrounding road network. They need to know when to turn left. Effective sonification follows the same principle. Its purpose is not to communicate everything. Its purpose is to communicate what matters. Throughout the lecture, McGrory repeatedly argued that the modern problem is rarely a lack of information. More often, it is an excess of information. The real design challenge lies in deciding what should reach people, when it should reach them, and how it can be communicated without demanding unnecessary attention.

    This perspective places sonification within a much broader discussion about interface design. Many of the tools through which people still interact with information were developed for a world in which data was far less abundant than it is today. Screens, keyboards, menus, and dashboards remain remarkably successful technologies, though they are not always suited to situations in which people are moving through complex environments while simultaneously performing other tasks. McGrory pointed towards cities as a particularly interesting example. Vast quantities of public information now exist concerning transportation systems, environmental monitoring, infrastructure, weather, traffic, and public services. Much of this information could potentially support everyday decision-making, yet most people never encounter it. One reason is that visual interfaces demand attention. Looking at a screen competes with countless other activities. Sound offers different possibilities. It can accompany movement, coexist with visual tasks, and communicate information without constantly demanding that people stop and look elsewhere.

    Questions of accessibility revealed why these issues matter so much. Much of McGrory’s work has involved collaboration with blind and visually impaired communities, experiences that challenged many of his assumptions about information design. Designers often assume that providing access to information is sufficient. The reality is considerably more complicated. Screen readers can successfully read large quantities of information aloud, though understanding the overall structure of that information remains difficult. McGrory compared this experience to attempting to complete a jigsaw puzzle without ever seeing the image on the box. Individual pieces are available, yet the broader picture remains difficult to grasp. A spreadsheet containing thousands of values can be read sequentially, though identifying patterns, relationships, and trends becomes far more challenging. Sonification offers one possible response to this problem. Rather than replacing detailed exploration, it can provide rapid overviews that help listeners understand the shape of information before investigating individual details.

    These experiences also led McGrory to question some established assumptions within sonification itself. Many projects focus heavily on transforming data into sound while paying comparatively little attention to context. Listeners are often presented with unfamiliar sounds and expected to interpret them independently. For McGrory, this represents a significant limitation. Communication rarely functions through raw information alone. Context, explanation, and narrative play equally important roles. Podcasting, journalism, and storytelling all demonstrate how audiences use framing to understand unfamiliar material. Projects such as the BBC’s Audiograph series combine sonification with narration, allowing listeners to understand not only what they are hearing but why it matters. This approach shifts attention away from sonification as a technical exercise and towards sonification as communication. Sound becomes one element within a broader process of explanation rather than an isolated solution expected to function independently.

    A particularly memorable example involved astronomy. McGrory discussed the work of a blind astronomer who uses sonification to explore stellar data, challenging assumptions about both astronomy and accessibility. At first glance, astronomy appears inseparable from visual observation. Popular images of galaxies, stars, and nebulae reinforce the assumption that astronomical knowledge depends upon sight. Yet the example revealed something much more interesting than accessibility alone. Rather than simply compensating for an inability to see, sonification provided an alternative way of engaging with information.

    This distinction became important throughout the lecture. Discussions of accessibility sometimes assume that non-visual approaches exist primarily to reproduce experiences that sighted people already have. McGrory encouraged a different perspective. Listening is not merely a substitute for seeing. It is a different mode of perception with its own strengths and limitations. Patterns that may be difficult to recognise visually can sometimes become more apparent through sound. Temporal relationships, repetition, variation, rhythm, and change often lend themselves naturally to auditory interpretation. For this reason, sonification can function as more than an accessibility tool. It can become an analytical tool. Different sensory approaches reveal different aspects of information. A graph may highlight one set of relationships. A sonification may reveal another. Rather than competing with visualisation, the two approaches can complement one another. The question is not whether data should be seen or heard. The question is what becomes possible when both options are available.

    Climate data provided another revealing case. Contemporary societies generate vast quantities of information about environmental change, though much of that information remains difficult for non-specialists to engage with meaningfully. Charts and graphs may communicate trends accurately, yet they do not necessarily encourage engagement. McGrory discussed projects that transform environmental datasets into musical or sonic experiences, creating opportunities for audiences to encounter information differently. Such approaches are not intended to replace scientific analysis. Rather, they create alternative routes into understanding. A sonification may not communicate every statistical detail, though it may encourage curiosity, emotional engagement, or reflection in ways that conventional visualisations struggle to achieve. Throughout the lecture, McGrory repeatedly returned to the idea that communication is not simply a matter of transmitting information. It is a matter of helping people connect with information in the first place.

    Underlying many of these examples was a broader argument about innovation and interdisciplinary thinking. McGrory repeatedly suggested that some of the most interesting developments emerge when disciplines that rarely interact begin to overlap. He cited a definition of innovation that particularly resonated with him: innovation happens when things that are separate become mixed. Sonification itself reflects this principle. The field draws simultaneously upon sound design, music, journalism, accessibility research, user experience design, data science, software development, artificial intelligence, and communication studies. No single discipline possesses all the answers. Progress emerges through collaboration between people approaching similar problems from different directions. This perspective also explains why McGrory consistently framed sound as part of larger design conversations rather than as an isolated specialism. Questions about how people receive information, understand systems, and engage with the world are not purely visual problems. They are design problems.

    These ideas become increasingly significant as emerging technologies continue to reshape everyday life. Artificial intelligence, conversational interfaces, spatial audio, augmented reality, and wearable computing all point towards futures in which information may become less dependent upon screens. While immersive visual technologies continue to evolve, audio already occupies a privileged position within contemporary life. Millions of people carry headphones throughout much of the day. Voice assistants have become commonplace. Podcasts reach global audiences. The infrastructure required for sophisticated auditory experiences already exists. For McGrory, the challenge is no longer technological feasibility. The challenge is learning how to design auditory experiences that are genuinely useful.

    Looking back across the lecture, perhaps the most striking aspect was the extent to which sonification emerged as a human problem rather than a technical one. Questions about mapping data to sound, selecting parameters, or designing auditory displays certainly matter, though they consistently led back to larger concerns about accessibility, communication, attention, understanding, and engagement. Again and again, McGrory encouraged students to think less about sound as an isolated discipline and more about what sound can contribute when combined with other ways of understanding the world.

    Data has become one of the defining materials of contemporary society. Increasingly, our institutions, technologies, economies, and daily lives depend upon it. Yet much of that information remains hidden behind interfaces designed primarily for looking rather than listening. For Hugh McGrory, this represents an enormous opportunity. Whenever information exists without sound, there is the possibility of creating new ways for people to experience, understand, and engage with it. More importantly, there is the possibility of discovering things that might otherwise remain hidden. Listening does not simply provide another route to the same destination. Sometimes it changes what can be found along the way.

    Perhaps that is why his central question continues to resonate long after the lecture ends. In a world increasingly shaped by data, why should our ears be left out of the conversation?

  • Who Are We Designing For? Professor Bruce Walker on Sound, Accessibility, and Human-Centred Design

    Bruce Walker

    Who are we designing for?

    At first glance, the answer appears obvious. Designers create products for users. Engineers build systems to help people accomplish tasks. Technology exists to solve problems. Yet during his online guest lecture for Edinburgh Napier University, Professor Bruce Walker repeatedly returned to examples suggesting that the answer is often more complicated than it first appears. Again and again, he described situations in which technically impressive systems failed to account for the realities of the people expected to use them. A solution might work perfectly according to engineering specifications while proving frustrating, distracting, or simply undesirable in practice. Across projects involving navigation, education, museums, accessibility, sonification, and auditory interfaces, Walker argued that successful design begins not with technology but with understanding human needs.

    Sound provided the central thread running through the lecture. Many people associate auditory interfaces with simple alerts and alarms. Computers beep. Phones ring. Vehicles issue warning tones. Yet Walker demonstrated that sound can serve far more sophisticated purposes. It can guide navigation, communicate data, support education, enable accessibility, assist decision-making, and provide entirely new ways of interacting with technology. These possibilities emerge not from adding sounds indiscriminately but from carefully considering what information people need, when they need it, and how they can use it effectively.

    This emphasis on understanding users before designing solutions appeared throughout the lecture. These broader questions became particularly visible in Walker’s discussion of the SWAN project, the System for Wearable Audio Navigation. The goal was deceptively simple. Could sound help people navigate when they could not rely on vision? Blind users are an obvious example, though Walker deliberately framed the problem more broadly. Firefighters moving through smoke-filled buildings, soldiers operating at night, divers underwater, and workers engaged in visually demanding tasks may all find themselves unable to look or unable to see. Rather than designing exclusively for a particular group, the project focused on a shared perceptual challenge.

    Approaching the problem in this way immediately changed the nature of the design process. Navigation might seem like a familiar activity, something most people perform every day without conscious thought. Yet once the team began analysing the task carefully, they discovered layers of complexity hidden beneath the surface. People need to know where to go, though they also need information about obstacles, changes in terrain, landmarks, hazards, and points of interest. Navigation is not simply a matter of moving from one location to another. It involves understanding an environment while continuously making decisions within it.

    The resulting system employed spatialised audio beacons that users could follow through space. Walker compared them to a virtual carrot suspended ahead of the listener. Rather than receiving a sequence of verbal instructions, users simply moved towards a sound source. When a turn became necessary, the sound shifted position accordingly. The concept appears elegant partly because it exploits abilities listeners already possess. Humans are remarkably good at locating sounds. Rather than teaching an entirely new interaction technique, the system builds upon existing perceptual skills.

    What makes the project particularly revealing, however, is the extent to which users shaped its development. Early assumptions frequently proved incorrect. Designers initially believed they should describe surface transitions in detail, informing users when they were moving from pavement to grass or from one surface to another. Users quickly pointed out that such information was often redundant. They already knew where they were standing. What mattered was knowing what was coming next. By listening carefully to users rather than insisting upon their original assumptions, the team produced a more efficient and less intrusive system.

    This pattern appeared repeatedly throughout the lecture. Successful auditory interfaces emerge through collaboration, observation, and evaluation rather than through technological enthusiasm alone. Walker repeatedly emphasised the importance of testing systems with real users performing real tasks. Maps of navigation paths, performance data, error rates, and subjective feedback all played important roles in understanding whether a design genuinely worked. The question was never whether a system could be built. The question was whether it improved people’s ability to accomplish what they were trying to do.

    Perhaps nowhere was this philosophy more apparent than in the team’s work with bone-conduction audio. Navigation systems often rely upon stereo or spatial audio, which typically requires headphones. Yet many users rejected conventional headphones immediately. Blind people rely heavily upon environmental sounds. Firefighters need to hear what is happening around them. Covering the ears solved one problem while creating another. Rather than treating this as an unavoidable limitation, the researchers reconsidered the entire system. Bone-conduction devices allowed spatial audio to be delivered without blocking environmental awareness. Once again, the solution emerged not from pursuing technology for its own sake but from understanding the realities of users’ lives.

    Walker extended these ideas beyond navigation into the design of auditory menus and interfaces. Modern computer systems contain large amounts of information organised visually through menus, icons, scroll bars, and navigation structures. Translating these elements into sound presents significant challenges. Simply reading everything aloud through text-to-speech quickly becomes inefficient and frustrating. Long contact lists, extensive music libraries, and complex software systems require more sophisticated approaches.

    Many of the solutions developed by Walker and his collaborators demonstrate an intriguing combination of technical ingenuity and psychological insight. Spearcons, for example, compress spoken words into highly abbreviated auditory cues that users rapidly learn to recognise. Spindex systems provide indexing sounds that help listeners move efficiently through large collections without hearing every item in sequence. Whispered speech can indicate unavailable menu items while preserving the overall structure of an interface. These innovations are clever, though their importance lies less in their novelty than in their effectiveness. Each emerged through extensive experimentation designed to determine what users actually understood and preferred. What makes this work particularly interesting is that it was never simply about making visual interfaces audible. Instead, it explored how auditory interfaces might exploit the strengths of listening itself. Users do not experience sound in the same way they experience vision, and successful interfaces acknowledge that difference rather than treating audio as a substitute for a screen.

    The lecture repeatedly highlighted a distinction between engineering and design. Engineering can produce systems that function correctly. Design concerns whether people will actually want to use them. Walker discussed several examples of technically impressive systems that failed precisely because they neglected this distinction. Some provided too much information. Others demanded excessive attention. Many ignored the broader contexts within which people operate. A system may be capable of delivering enormous amounts of information, though that does not necessarily mean people want to receive it.

    Questions of accessibility broadened these concerns further. Much of Walker’s work focuses on enabling participation rather than merely providing access. He described projects supporting education for blind students, making scientific information more accessible, and improving experiences within museums and aquariums. These examples revealed another recurring theme. Accessibility is not simply about removing barriers. It is about ensuring that people can engage meaningfully with experiences, opportunities, and ideas.

    His work within museums and aquariums illustrates this particularly well. Modern cultural institutions often provide physical access while leaving informational and emotional access largely unresolved. A blind visitor may be able to enter an aquarium, though the experience remains fundamentally visual. Standing before an enormous tank filled with whale sharks and rays offers little if the most important aspects of the exhibit remain inaccessible. Walker’s team explored ways of using tracking systems, sonification, narration, and auditory displays to communicate not only what was present but what was happening. The objective was not merely to describe the environment. It was to create opportunities for engagement, curiosity, and wonder.

    Educational projects revealed similar concerns. Throughout the lecture, Walker repeatedly returned to the distinction between access and participation. Providing information is only one part of inclusion. Students also need opportunities to explore, question, discover, and develop understanding independently. Whether working with scientific data, classroom materials, museum exhibits, or public spaces, the challenge remained remarkably consistent. How can information be presented in ways that support meaningful engagement rather than passive reception? Accessibility, in this sense, becomes a question of design quality rather than a specialised feature introduced at the end of a project.

    Underlying many of these projects is the field of sonification, the practice of representing information through sound. Walker described sonification as both a design challenge and a research problem. Any attempt to translate data into sound requires decisions about mapping, scaling, timing, context, and interpretation. Should temperature be represented through pitch, loudness, or tempo? How should complex information be organised so that listeners can understand it? These questions have no universal answers. Effective solutions depend upon understanding both the data and the people expected to interpret it.

    One reason sonification remains challenging is that many listeners have relatively little experience interpreting information through sound. Graphs, charts, and maps are familiar cultural forms. Auditory representations are far less common. Designers therefore need to balance learnability with expressiveness. A system may communicate information accurately while remaining difficult to interpret. Conversely, a system may sound appealing while conveying very little. Walker’s research repeatedly demonstrated that effective sonification emerges through iterative testing with users rather than through theoretical assumptions alone.

    Such challenges reveal why Walker remains sceptical of simplistic approaches to auditory design. Throughout the lecture, he criticised the tendency to reduce audio interfaces to collections of arbitrary beeps and alerts. Sound possesses enormous communicative potential, though realising that potential requires careful thought. Designers must consider attention, context, usability, aesthetics, cultural expectations, and human behaviour. A sound that performs well in a laboratory may fail completely in everyday life. A technically accurate representation may prove ineffective if users cannot interpret it.

    The phrase “beeps and bops” became a useful shorthand for this problem. Many technologies employ sound only at the most superficial level, relying upon alerts, warnings, and notifications while overlooking the richer possibilities of auditory interaction. Walker’s work points towards a broader conception of sound, one capable of supporting navigation, exploration, learning, communication, and discovery. The challenge is not simply adding sound to technology. It is designing meaningful auditory experiences.

    Towards the end of the discussion, Walker reflected on what he described as a “failure of imagination” in technology design. Sometimes designers struggle to imagine how people actually live with technologies. At other times, users struggle to imagine possibilities that do not yet exist. Successful innovation requires navigating both challenges simultaneously. Revolutionary technologies rarely emerge through user requests alone. Yet genuinely useful technologies also cannot emerge through engineering in isolation. Design becomes a process of bridging these perspectives.

    Looking back across the lecture, what emerges most clearly is not a story about auditory interfaces but a broader philosophy of design. Sound happens to be the medium through which Professor Walker explores these questions, though the underlying principles extend much further. Navigation systems, auditory menus, museum exhibits, educational technologies, sonification projects, and accessibility tools all reveal the same challenge. Technologies succeed not when they demonstrate technical sophistication but when they become meaningful parts of human activity.

    Perhaps this is why Walker repeatedly resisted framing accessibility as a specialised concern. The challenges faced by blind users, firefighters, drivers, students, museum visitors, and countless others often reveal broader truths about human interaction with technology. Designing for specific needs frequently produces insights that benefit everyone. When designers stop asking what a system can do and start asking what people need, entirely new possibilities begin to emerge.

    Throughout the lecture, examples ranging from spatial navigation to aquarium exhibits pointed towards the same conclusion. Successful technologies rarely begin with devices, software, algorithms, or interfaces. They begin with people. Understanding how people listen, learn, move, explore, communicate, and make decisions provides the foundation upon which everything else is built.

    For Professor Bruce Walker, the future of auditory interfaces does not lie in adding more sounds to the world. It lies in understanding how sound can help people navigate, learn, communicate, discover, and participate more fully in the experiences around them. The technology matters. The research matters. The engineering matters. Yet each ultimately serves a more fundamental question, one that quietly shaped the entire lecture from beginning to end: who are we designing for?

  • Why Game Sound Is Never Finished: Mariana Botero on Systems, Possibility, and Interactive Audio

    Mariana Botero

    What does a sound designer actually create?

    For much of the history of recorded media, the answer has seemed relatively straightforward. A sound designer creates sounds. Those sounds are edited, arranged, mixed, and eventually delivered to an audience in a finished form. Whether working in film, television, radio, or theatre, the result remains largely fixed. Every audience member encounters the same sequence of events in the same order. A soundtrack may be replayed thousands of times, though the sounds themselves do not change. Sound design, in this traditional sense, is largely concerned with creating and refining artefacts. The work may be extraordinarily complex, though the outcome is ultimately stable. Once completed, the audience receives the experience that the creators intended.

    Games operate according to a different logic. During her guest lecture at Edinburgh Napier University, Mariana Botero, Sound Designer at Criterion Games, repeatedly returned to a challenge that sits at the centre of interactive audio. Players are unpredictable. They may rush through environments that designers expected them to explore carefully. They may spend an hour in a location intended for a few minutes of gameplay. They may repeat actions endlessly, ignore carefully placed cues, or discover solutions that nobody anticipated. A sound designer may spend weeks refining a particular moment, only for players to experience it in a completely unexpected way. This unpredictability creates a fundamental difference between games and most other forms of media. Film sound designers can assume a degree of control over audience experience. A scene unfolds at a predetermined pace. Music enters at a precise moment. Dialogue arrives exactly when it is needed. Sound and image progress together through a carefully authored sequence. Games surrender much of that control. Designers can establish possibilities, though they cannot determine exactly how those possibilities will be experienced. Every player creates a slightly different path through the material.

    Botero illustrated this distinction through a deceptively simple analogy. Traditional sound design can resemble baking a cake. Once the ingredients have been combined and baked, they become a finished object. Individual components can no longer be separated. The audience receives the completed result exactly as intended. Interactive audio often works differently. The ingredients remain available. They can be rearranged, adjusted, recombined, and reshaped while the experience is unfolding. What players hear depends not only on what the designer created but also on what they choose to do. A player who rushes through an environment may hear one version of the experience. Another who explores every corner may encounter something quite different. Neither experience is incorrect. Both emerge from the same underlying system. At first glance, this may appear to be a technical distinction, though throughout the lecture it became clear that something more significant is taking place. Interactive audio challenges assumptions that have shaped sound practice for decades. Rather than creating a finished soundtrack, game audio designers increasingly create systems capable of generating many possible soundtracks. The question is no longer simply what a sound should be. The question becomes how a sound should behave.

    Implementation sits at the heart of this shift. Students often encounter implementation through software platforms such as Wwise and Unity, where attention naturally gravitates towards events, switches, parameters, states, and scripting. Botero encouraged a broader perspective. These tools are not merely technical requirements added after the creative work has been completed. They are part of the creative process itself. They provide mechanisms through which sound can respond to player actions, environmental conditions, narrative developments, and changing game states. Once audio becomes interactive, implementation ceases to be a final stage of production. It becomes one of the primary ways in which experiences are designed. Many of Botero’s examples reflected this movement from sounds to systems. Consider something as apparently mundane as footsteps. Few sounds occur more frequently in games. A footstep that appears a handful of times in a film may occur thousands of times during a single play session. What initially feels satisfying can quickly become repetitive. Players generate these sounds through their own behaviour, meaning designers cannot simply assume that repetition will remain unnoticed. Botero discussed several implementation strategies designed to address this issue. Different recordings can be selected randomly. Pitch and volume may vary subtly between repetitions. Heel and toe impacts can be separated and recombined dynamically. A relatively small collection of recordings suddenly produces a far wider range of perceived outcomes. Yet what makes these techniques interesting is not their technical sophistication. Their significance lies in how they reveal a different philosophy of authorship. Rather than crafting every individual event directly, designers create rules governing how events are generated. They design the behaviour of the system rather than the precise form of every outcome.

    Her observations about footsteps led naturally into a broader discussion about attention. Not every sound deserves equal prominence throughout an experience. Early in a game, clearly audible footsteps may help players understand movement and control. Later, those same sounds can begin to dominate the soundscape unnecessarily. As players become familiar with core mechanics, environmental details often become more valuable. A distant owl, subtle weather activity, a passing vehicle, or an unexpected environmental cue may contribute more to a sense of place than another clearly articulated footstep. This may seem like a relatively small design decision, though it reveals an important principle running throughout the lecture. Sound design is not simply about creating sounds. It is about shaping attention. Designers are constantly deciding what players should notice, what they should ignore, and how their awareness should be directed through an experience. Interactive audio therefore becomes inseparable from broader questions of perception.

    Several examples from Botero’s work on Star Wars Battlefront II illustrated this relationship particularly clearly. One challenge involved creating a convincing sensation of speed during space combat. Space provides surprisingly few visual reference points. Without roads, buildings, or passing landscapes, players can struggle to judge how fast they are moving. From a purely visual perspective, extraordinary speeds can sometimes feel unexpectedly slow. Audio therefore assumes a more active role. Changes in acceleration, orientation, proximity, and manoeuvring can all be reinforced through sound, encouraging players to feel faster than they actually are. Importantly, the objective is not deception. The objective is alignment between what players see, what they hear, and what they believe they are experiencing. Sound helps bridge the gap between game mechanics and player perception. A related challenge emerged around spatial awareness. Players navigating complex three-dimensional environments often need information that visual displays cannot always communicate efficiently. Botero described examples where reflections, pass-bys, and environmental responses helped players understand their relationship to surrounding structures. Those sounds functioned as navigational aids as much as sound effects. Players may never consciously notice them, though their absence would make environments feel less intelligible. Such examples reveal another important aspect of interactive audio. Sound is not simply representing the world. It is helping players interpret the world. A successful design often communicates information, directs attention, reinforces emotion, and supports decision-making simultaneously. The most effective implementation frequently becomes invisible. Players simply feel that the game responds naturally to their actions.

    The same principles appeared again in Botero’s discussion of interactive music. Traditional composition assumes a relatively stable structure. A beginning leads towards a middle before eventually reaching an ending. Although composers may create complex and highly sophisticated works, they generally retain control over the order in which events occur. Games rarely provide such certainty. Players may linger in one location, interrupt events unexpectedly, revisit spaces repeatedly, or trigger narrative developments in unusual orders. Music must therefore accommodate possibilities that cannot be fully predicted in advance. Rather than composing a single linear sequence, designers often create collections of musical elements capable of being reorganised dynamically while preserving coherence. Botero compared the process to building with Lego bricks. Individual pieces remain consistent, though their arrangement changes according to context. Introductory passages, transitions, loops, and endings can be connected in different ways while still feeling musically coherent. Once again, the designer is not creating a single outcome. The designer is creating a framework capable of supporting many outcomes. What matters is not only the material itself but also the relationships that determine how that material behaves under changing circumstances.

    This way of thinking extends beyond music. One particularly revealing example discussed during the lecture involved a student project built around a time-slowing mechanic. In a traditional medium, slowing time might simply involve applying predetermined processing at specific moments. Within an interactive environment, however, the relationship becomes dynamic. Real-time parameter controls allowed player actions to influence audio behaviour directly. As gameplay changed, the soundscape changed alongside it. The mechanic was not merely accompanied by sound. The mechanic became part of the sound design process itself. Audio no longer functioned as a layer added on top of gameplay. It became woven into the behaviour of the system. Examples such as these help explain why programming increasingly occupies an important place within contemporary game audio. Throughout the lecture, Botero described learning through experimentation, tutorials, practical projects, and professional experience. Programming was presented not as a replacement for creative thinking but as a means of expanding creative possibilities. The more deeply designers understand systems, the more effectively they can connect audio to player experience. Technical knowledge becomes valuable not for its own sake but for the opportunities it creates.

    One of the more interesting consequences of this shift is that game audio designers increasingly resemble system designers as much as traditional sound designers. Much of the language surrounding audio production still reflects assumptions inherited from film, television, and music. We often speak about creating sounds, mixing sounds, or arranging sounds. Botero’s examples repeatedly pointed towards a broader form of practice. Designers create relationships between sounds, player actions, environmental conditions, and game states. Their work involves determining how sounds respond, adapt, and evolve rather than simply deciding what those sounds should be. The resulting experience emerges through interaction between the player and the system. In this sense, implementation is not something that happens after sound design. Implementation is increasingly part of sound design itself.

    This spirit of exploration appeared throughout Botero’s account of her own development. Long before working professionally in games, she was already engaging deeply with sound through recording, listening, and experimentation. Growing up in Colombia, she developed a fascination with environmental sound that later evolved into a broader interest in audio design. Looking back across the lecture, one of the most striking aspects of her career trajectory is how consistently curiosity appears as a driving force. New tools, new techniques, and new technologies repeatedly emerge, though progress often begins with a simple willingness to investigate how things work. Throughout her account there was little sense of a fixed pathway into game audio. Instead, learning appeared as an ongoing process of exploration in which technical knowledge, creative practice, and experimentation continually informed one another.

    That same curiosity remains visible within professional practice. One of the most memorable examples came from Criterion’s “Sound Design Thursdays”. Team members temporarily step away from production work and undertake creative challenges built around unusual constraints. Designers exchange recordings, limit themselves to small collections of source material, or pass sounds between colleagues with each person applying a single transformation before handing them on. The outcomes can be surprising, humorous, and occasionally absurd. A camel recording may gradually evolve into something resembling a bird. Familiar sounds acquire entirely new identities. Yet beneath the humour lies a serious lesson about creativity. Unexpected results often emerge when designers deliberately create conditions that encourage experimentation. Limitations become opportunities. Constraints become creative tools. In many respects, these exercises mirror the philosophy underlying interactive audio itself. Both involve constructing systems, establishing rules, and allowing surprising outcomes to emerge from interactions between different elements. Neither depends upon complete control. Both depend upon creating environments in which interesting possibilities can arise.

    Looking back across the lecture, what emerges most clearly is a view of sound design that extends well beyond the creation of individual sounds. Footsteps, adaptive music, spatial cues, implementation systems, real-time parameters, and creative experiments all point towards the same conclusion. Interactive audio requires designers to think in terms of relationships rather than isolated assets. Sounds gain meaning through how they respond to players, environments, and changing circumstances. The challenge is no longer simply to create a soundtrack. The challenge is to create a framework capable of supporting many different soundtracks.

    Perhaps this is what makes game audio such a distinctive area of contemporary sound practice. Film sound designers craft experiences that audiences receive. Game sound designers craft possibilities that audiences help create. Every play session unfolds differently. Every player encounters a slightly different combination of events. Every interaction generates new relationships between sounds, systems, and behaviours. The sounds matter. What matters just as much are the rules that determine what those sounds might become.

    In game audio, the work is never truly finished. It simply waits for the player to decide what happens next.

  • Why Do the Sounds of Don’t Starve Feel So Alive? Matthew Marteinsson on Experimentation, Voice, and Play

    Matthew Marteinsson

    Many games strive for realism. They aim to reproduce the sound of the world as accurately as possible, carefully modelling spaces, materials, physics, and behaviours so that players feel immersed in a believable environment. Don’t Starve takes a rather different approach. Its world is filled with living scarecrows, walking trees, giant spiders, impossible creatures, and surreal landscapes that seem to have escaped from the pages of a dark storybook. Very little about it appears realistic in any conventional sense. Yet despite this, the game feels remarkably alive.

    Matthew Marteinsson’s guest lecture explored how that happened. Although the talk covered specific technical systems, recording techniques, production challenges, and implementation details, a broader idea repeatedly emerged beneath them. The sounds of Don’t Starve do not feel convincing because they imitate reality. They feel convincing because they remain connected to physical performance, playful experimentation, and a constant willingness to explore unexpected possibilities.

    During the early development of Don’t Starve, Marteinsson was effectively the sole audio designer working alongside two composers, with no dedicated audio programmer and no substantial audio department behind him. Development moved rapidly, content changed constantly, and there was little opportunity for elaborate production pipelines. Rather than treating these limitations as obstacles, the team repeatedly used them as opportunities to find simpler and more creative solutions. Constraints were not merely something to overcome. They actively shaped the character of the game’s sound world.

    The game’s character voices provide an excellent example. Traditional voice acting would have required large quantities of dialogue recording, scheduling actors, and continuously updating content as the game evolved. Such an approach was difficult to reconcile with the speed at which the project was being developed. Yet the characters still needed personality, emotional expression, and identities that players could immediately distinguish. Instead of using spoken language, Marteinsson turned to musical instruments. Inspired partly by the adults in Peanuts cartoons and partly by Peter and the Wolf, where different instruments represent different characters, each character in Don’t Starvereceived its own instrumental voice. Wilson’s distinctive muted trumpet became the starting point, with subsequent characters developing from their own carefully chosen instrumental identities. What began as a practical solution ultimately became one of the most recognisable features of the game.

    Human vocal performance appeared repeatedly as a creative tool throughout the lecture. Many memorable sounds originated not from extensive libraries or complex synthesis chains but from experimentation with the voice itself. The spiders, for example, were largely built from Marteinsson’s own vocal performances combined with processing. The Gobbler, one of the game’s most beloved creatures, began with attempts to gather suitable turkey recordings. After examining the animation, however, he found himself instinctively making a strange vocal sound that immediately felt more appropriate than any authentic turkey call. The library recordings were discarded and the vocal performance became the creature. As he noted, the deliberately exaggerated human performance communicated personality far more effectively than realism alone could have achieved.

    Realism and believability emerged as distinct ideas within Marteinsson’s approach to sound design. A perfectly accurate turkey recording might have sounded more realistic, though it may not have felt more alive. The Gobbler succeeds precisely because it occupies an unusual space between animal, caricature, and performance. Players are not simply hearing a creature. They are hearing a performance of a creature. The sound communicates character as much as biology.

    Personality often seemed more important than realism throughout the lecture. Many of the creatures in Don’t Starveexist within a visual world that is intentionally exaggerated, stylised, and slightly absurd. Conventional fantasy sound design might have felt strangely out of place. Marteinsson instead described grounding many creatures in a “weird reality”, where recognisable physical behaviours remain present but become filtered through performance, humour, and experimentation. Human vocalisations proved especially valuable in this regard. Audiences are extraordinarily sensitive to nuances in human expression. Even heavily processed vocal sounds can communicate intention, emotion, vulnerability, aggression, or curiosity in ways that are difficult to achieve through purely synthetic or animal-based recordings.

    Environmental audio presented an equally interesting challenge. Procedurally generated worlds create difficulties that traditional environmental sound design rarely encounters. Designers cannot assume where players will travel or which environments they will encounter. Marteinsson described a system that continuously examines the terrain surrounding the player, identifies the dominant biome types within the immediate area, and dynamically blends the corresponding ambiences. Grasslands, forests, marshes, and other environments continuously mix together according to what the player is actually seeing at that moment. Rather than creating a fixed soundtrack for a predetermined world, the system responds to the world being generated in real time.

    What makes this system particularly interesting is that the underlying idea remains remarkably simple. Players should hear the world they are looking at. Technical sophistication only becomes valuable when it strengthens the player’s experience. Systems matter not because they are complex but because they help players understand the world around them. Throughout the lecture, Marteinsson repeatedly demonstrated a preference for elegant solutions that serve a clear experiential purpose.

    The broader design philosophy became especially clear during the discussion following the lecture. Marteinsson argued that game audio should generally perform one of two functions: it should either build the world or inform the player. If a sound accomplishes neither, its value becomes questionable. Such a statement sounds straightforward, though it carries considerable implications for design practice. Many games accumulate audio over time, layering additional sounds onto already crowded mixes. The result can be confusion rather than clarity. Marteinsson instead advocates careful consideration of why a sound exists and what purpose it serves. Sound is not decoration. It is communication.

    Small details often became surprisingly important within this design philosophy. During the lecture, he discussed how player feedback during early access revealed complaints about a particular pickup sound. Some players even requested a dedicated option to disable it. Rather than immediately changing the sound itself, Marteinsson investigated further and discovered that the underlying issue was simply that the sound was mixed too loudly. Once its level was adjusted, the complaints disappeared. The lesson was not that players were wrong. Rather, it highlighted the importance of identifying the underlying problem rather than accepting proposed solutions at face value. Players are often very effective at identifying areas where something feels wrong. Determining why it feels wrong remains part of the designer’s responsibility.

    Recording sessions often sounded closer to scientific experiments than conventional sound production. Music boxes, improvised instruments, jelly, pudding, toys, unusual household objects, mines, and novelty items discovered in shops all found their way into Marteinsson’s recording collection. A music box originally intended for composing melodies eventually became the basis for the unsettling sounds associated with the Shadow Hand. A visit to a local mining museum produced unique underground ambience recordings for the game’s cave systems. Strange objects were collected not because a specific project required them, but because they might become useful in the future.

    Playfulness often appeared not as a break from the work but as part of the work itself. Making strange noises while watching an animation, experimenting with unusual objects, collecting sounds without a specific purpose in mind, or exploring unexpected combinations of recordings all reflect a willingness to follow curiosity wherever it leads. Listening to these stories, it became increasingly clear that creativity often depends upon creating opportunities for surprise. The value of an unusual object or recording does not necessarily become apparent immediately. A sound designer may encounter something intriguing, record it, store it away, and only discover its purpose years later.

    Technical decisions rarely appeared separate from creative ones during the lecture. Recording techniques, implementation systems, middleware, debugging tools, and production constraints were all discussed in detail. Yet none of these elements were treated as separate from creativity itself. Debug tools existed to facilitate experimentation. Procedural systems existed to strengthen immersion. Recording techniques existed to discover new forms of expression. Technology remained important throughout the talk, though it rarely appeared as the primary source of innovation.

    Reflections on game development brought many of the lecture’s themes together. Marteinsson acknowledged the challenges facing the industry, including long hours, instability, and periods of significant uncertainty. Yet his reflections consistently returned to enthusiasm, curiosity, and the joy of creating experiences that players genuinely care about. That optimism felt closely connected to the ideas that had surfaced throughout the lecture. The sounds of Don’t Starve emerged not from a search for perfection but from a willingness to experiment, adapt, collaborate, and occasionally embrace absurd ideas simply to see where they might lead.

    Perhaps that helps explain why the world of Don’t Starve feels so distinctive. Its sounds rarely seem trapped by expectations about what things ought to sound like. A spider may begin as a human vocal performance. A terrifying shadow creature may emerge from a modified music box. An iconic turkey may owe more to an impulsive noise made while watching an animation than to any field recording. Throughout the lecture, Marteinsson repeatedly demonstrated that memorable sound design often emerges when curiosity is allowed to guide the process.

    Rather than attempting to recreate reality exactly, Don’t Starve constructs a world that feels alive through performance, experimentation, and play. Many of the sounds discussed during the lecture began as accidents, improvisations, constraints, or strange ideas that simply seemed worth exploring. What emerged from that process was not merely a collection of sound effects but a coherent sonic world. Listening to the lecture, it became difficult to separate the sound of Don’t Starve from the spirit in which it was created. Both are defined by curiosity.

    In doing so, Marteinsson offered a useful reminder that some of the most memorable sounds are not discovered by following established rules. They emerge when designers remain willing to ask a simple question: what happens if we try this?

  • Speaking into Spaces: Lou Mallozzi on Language, Sound, and Public Architecture

    Lou Mallozzi

    We often think of speech as a way of transmitting information from one person to another. Someone speaks, someone listens, and meaning moves between them. Language therefore appears relatively straightforward. Words describe things, explanations clarify ideas, and speech communicates intentions. Yet speaking also does something else that often receives much less attention. Voices establish relationships. A person standing quietly in a crowded room occupies space in one way, though speaking suddenly reorganises that same environment. Attention shifts. Distances feel altered, while public and private boundaries become less stable. Speech does not simply move through spaces. It also changes them.

    This broader relationship between sound, language, and space formed a recurring concern throughout an online guest lecture delivered by Lou Mallozzi, a Chicago-based sound artist whose work moves across performance, installation, moving image, and public intervention. Although the projects discussed during the lecture varied considerably in form, common questions repeatedly surfaced beneath them. What happens once language stops functioning simply as communication? What happens when speech becomes material rather than message? How do voices alter relationships between people and places?

    Mallozzi described some of these ideas through the notion of an “architectonics of public speech”. Rather than treating speech as something moving cleanly between speaker and listener, this perspective considers speech as something capable of creating structures around it. Speaking establishes relationships, gathers attention, and reshapes environments around it. Architecture repeatedly appeared throughout the lecture, though not always in conventional ways. Buildings establish boundaries physically through walls, rooms, doors, and passages. Sound establishes different kinds of structures. Voices create temporary boundaries of attention, while amplified speech can reorganise atmospheres entirely. A room filled with silent individuals does not feel identical to one organised around speech. The physical dimensions remain unchanged, though experiences of proximity, distance, and social relationships begin shifting once voices enter the environment.

    These ideas became particularly visible in discussion of La Patria Patrisci Patri Partiti, translated as The Fatherland Gives Birth to Departed Fathers. The work emerged partly through reflections on Mallozzi’s father’s experiences growing up in fascist Italy before emigrating to the United States. Historical documents, fascist texts, and autobiographical material became intertwined within the work. Materials themselves were not simply presented directly. Mallozzi instead described subjecting language to processes of subtraction through the removal of letters taken from his own family name.

    Initially this process appears procedural, though the consequences become more complicated once language itself begins breaking apart. Certain words survive while others collapse into fragments. Meaning becomes unstable. Speech no longer functions simply as a transparent carrier of ideas. Language begins behaving differently once removed from its usual role. Words become sounds, rhythms, repetitions, and interruptions. Fragments acquire physical presence independent of straightforward meaning. Listeners are no longer simply absorbing information. They become aware of language itself as material.

    Performance introduced another layer to these relationships through carefully controlled microphone feedback generated from microphones positioned within the mouth itself. Feedback was not presented simply as an acoustic effect. Mouths function as resonant chambers, while rooms similarly become chambers of resonance. Internal and external forms of architecture begin reflecting one another. The body itself begins appearing as another form of acoustic space rather than simply the source from which speech emerges.

    Questions surrounding public space appeared again through works such as Outpost, Peers, and Didact, which approached different environments though repeatedly returned to similar concerns involving observation, attention, and collective listening. Outpost involved Mallozzi standing above public spaces equipped with a telescope and amplified microphone system while verbally describing individuals moving below in real time. Descriptions themselves often remained entirely ordinary. Someone reached into a pocket, adjusted clothing, or walked across an open area. Little of obvious importance occurred.

    Yet amplification transformed the situation entirely.

    Nothing new was being revealed. Individuals already knew what they were doing, while others nearby could already see it happening. Information itself therefore became almost irrelevant. Attention had become the important factor.

    Public spaces frequently depend upon a delicate balance between visibility and anonymity. Most of the time these arrangements remain unnoticed precisely because they operate so quietly. People move through stations, streets, galleries, and public squares while occupying an unusual position between presence and absence. We recognise that others exist around us, though frequently without examining them closely. Shared environments therefore involve a form of social agreement in which attention remains distributed and unstable. Outpost disturbed that agreement. Someone who had previously existed as part of the background suddenly became the centre of attention.

    Surveillance formed part of this discussion, though not entirely in familiar ways. Contemporary surveillance systems frequently operate quietly from concealed positions. Cameras observe without drawing attention to themselves. Outpostinstead exaggerated observation until it became impossible to ignore. The telescope almost became absurd in its visibility. Observation itself became public.

    Related works such as Peers and Didact returned to similar concerns through multiple simultaneous voices. Speech that would normally appear as private reading or individual statements instead became layered, collective, and unstable. Meanings drifted in and out of focus as voices overlapped, while explanations gradually became textures and descriptions became atmospheres. A single voice often carries assumptions of coherence and individuality. Multiple voices instead create shifting relationships where language repeatedly moves between intelligibility and abstraction.

    Later works extended these concerns into moving image and impossible tasks. Screenplay 1-1 involved watching films with the sound removed while attempting to verbally describe everything appearing on screen in real time. Such a task immediately creates difficulty. Language struggles to keep pace with visual experience. Camera movements, gestures, editing, and events unfold more rapidly than speech comfortably allows.

    Mallozzi described many projects of this kind as involving impediments or impossible conditions. Failure therefore ceases to become something requiring avoidance. Contemporary discussions surrounding creative work often assume that limitations are problems waiting to be removed. Better technologies frequently promise greater control, speed, and precision. The projects discussed throughout the lecture repeatedly suggested something slightly different. Constraints sometimes generate possibilities that unrestricted situations would never produce. Impossible tasks force unexpected decisions. Gaps emerge between intention and outcome. Material appears that nobody initially planned.

    Throughout the lecture, speech repeatedly occupied a position between structure and instability. Voices organise attention while simultaneously disrupting it. Language communicates while also becoming detached from communication itself. Sound establishes relationships while continually reshaping them.

    Perhaps one of the more striking ideas emerging from the lecture involved recognising that speaking rarely involves transmitting information alone. Voices connect bodies, spaces, histories, and listeners around them. Speech therefore becomes something more than information moving between individuals. Voices gather people together, establish temporary structures of attention, alter atmospheres, and occasionally make familiar environments seem unfamiliar again.

    Much of this activity normally passes unnoticed. Conversations emerge and disappear. Announcements briefly reorganise public spaces. Voices drift across rooms before fading into the background once more. Listening often feels passive, though Mallozzi’s work repeatedly suggested something rather different. Speaking changes environments, while listening changes our relationships with them. Spaces may appear stable around us, though many of the ways we experience them are continually being reorganised through sound.

  • Creating Sounds for Things We Cannot See: Kenny Young on VR, Music, and Guiding Attention

    Kenny Young

    Many forms of media depend upon controlling attention. Films decide where audiences look through editing, framing, and camera movement. Theatre guides attention through staging and movement. Conventional games frequently do something similar through interface design, visual effects, or camera behaviour. Important information rarely appears entirely by accident. Designers often decide where attention should go long before audiences realise those decisions are being made. Most of this guidance becomes invisible precisely when it works well. Players rarely stop to think about how often games quietly redirect their attention from one place towards another. Experiences simply feel natural. Objectives appear at appropriate moments, important events seem difficult to miss, and information arrives when required.

    Kenny Young’s guest lecture explored what happens once some of these assumptions begin disappearing. Virtual reality introduces a relatively simple change that gradually creates much larger consequences. Players control the camera continuously. Looking left means physically turning left. Looking upwards requires physically raising the head. Looking away from something important may simply mean missing it altogether. Initially this sounds like a relatively minor alteration, though the consequences begin spreading surprisingly far once control over attention starts shifting away from designers themselves.

    Imagine hearing something important happening behind you in a conventional game. Designers possess numerous methods for ensuring that players notice it. Cameras may shift automatically, indicators can appear around the screen, and control may even be briefly interrupted. Decades of game design have produced increasingly sophisticated methods for solving these problems. Virtual reality complicates many of these solutions. Fixed interface elements become intrusive, large overlays can weaken immersion, and information existing outside the player’s field of view can remain entirely unnoticed. Questions therefore begin emerging around how players discover important information once designers can no longer simply place it directly in front of them.

    Young suggested that sound changes role at precisely this point. Human vision behaves selectively. We actively choose where to direct our eyes and ignore much surrounding information. Hearing functions rather differently. Sounds continue arriving whether or not we intentionally seek them out. A player may choose not to look towards something important, though hearing something nearby can still trigger an immediate response. Sound therefore begins moving away from a supporting role attached to visible events and towards something more active.

    Players do not simply hear sounds in games. They gradually learn them. Initially a sound may exist only as another event occurring within a larger environment. Yet repeated exposure slowly changes its role. Through familiarity, sounds begin accumulating meaning. This process often happens without players consciously noticing it. A sound that initially appears neutral gradually becomes linked with expectations, actions, and outcomes. Eventually hearing the sound no longer involves interpreting something unfamiliar. Players instead recognise patterns they have already learned.

    Young discussed the familiar alert sound from Metal Gear Solid as an example. During early encounters players hear a brief cue alongside visual information, though repeated exposure gradually changes the relationship. Eventually players stop hearing the sound as a sound effect at all. Instead, it begins behaving almost like language.

    Language may not be entirely the right word, though the comparison becomes useful. Words themselves do not naturally contain meaning. People gradually learn relationships between sounds and ideas through repeated experience until recognition becomes almost immediate. Something similar begins happening within games. A short musical cue or brief sound effect acquires meaning through use rather than explanation. Players are not consciously translating sounds each time they hear them. Recognition simply becomes increasingly automatic.

    Nobody pauses a game to explain that a particular sound means danger. Players learn these relationships through repeated experience. Over time certain sounds become linked with expectations, actions, and outcomes until responses begin occurring almost automatically. Listening changes in subtle ways once these associations form. Players stop consciously analysing what they hear, as attention begins shifting before deliberate thought catches up. Sound therefore becomes something more than feedback occurring after an event. It starts creating expectations about what might happen next.

    Music introduces another layer to these learned relationships. Discussions around game music frequently focus on emotion, atmosphere, and immersion. Players may notice tension increasing during combat, emotional themes returning around familiar characters, or changing musical textures supporting movement through a world. Young explored another possibility entirely. Under certain conditions, music may also begin operating as information.

    Much of his work on Tethered explored whether these kinds of relationships could be developed within virtual reality environments. Strategy games already involve unusually large amounts of simultaneous information. Resources require management, environments continue changing, threats emerge unexpectedly, and events occur across multiple locations at once. Conventional interfaces frequently solve these problems visually. Players monitor maps, indicators, menus, and notifications distributed around the screen. Translating these expectations into VR introduced a more difficult question. How can players remain aware of a world once they can comfortably see only part of it at any given moment?

    Rather than functioning purely as atmosphere or emotional support, musical phrases could gradually become learned signals recognised through repeated interaction. Certain sounds became associated with changing conditions, important events, or emerging situations. Initially these sounds carried little meaning beyond existing as recognisable musical gestures. Over time something rather different happened. Players were not simply listening to music accompanying a world. They were gradually learning the world itself.

    Listening consequently begins developing an unusual relationship with navigation. Physical landmarks help people orient themselves within real environments, though players may also begin constructing sonic landmarks. Certain sounds become associated with places, behaviours, or changing conditions. Listening therefore starts becoming part of understanding how a world behaves. Particular musical phrases began functioning almost like landmarks within an environment. Certain combinations of sounds became associated with emerging threats or opportunities requiring attention. Over time players could respond before consciously thinking about what had changed. Listening therefore became intertwined with understanding the behaviour of the world itself.

    Examples such as these begin shifting the discussion slightly. Rather than asking whether music sounds appropriate or emotionally effective, another question begins appearing. How do people learn sonic environments? Under what circumstances do sounds stop behaving like sounds and begin behaving more like information? Underlying processes of this kind may already exist across many forms of game audio, even if virtual reality makes them easier to recognise.

    Extending these ideas into working systems introduced additional challenges. Sounds needed to remain distinctive while fitting comfortably alongside underlying music. Delays had to remain short enough that players still connected events with their causes, while multiple simultaneous events could create confusion or dissonance. Initial solutions often resolved one issue only to reveal another elsewhere. Technical constraints, musical decisions, and player behaviour continually interacted throughout development. Creative work therefore emerged less as a process of executing perfect ideas and more as a continual process of adjustment.

    Running throughout the lecture was a broader observation concerning the role of sound itself. Discussions surrounding game audio frequently emphasise realism, emotion, and atmosphere. These remain important concerns, though Young’s work suggested something slightly different. Once familiar methods for directing attention become less reliable, sound begins taking on responsibilities traditionally associated with cameras and interfaces.

    Virtual reality may therefore reveal something that has existed quietly within games for much longer. Sound has rarely functioned only as decoration or atmosphere. It has also shaped where players look, what they notice, and how they organise experiences around them.

    Perhaps the more interesting question is not whether sound helps players understand virtual worlds. It may instead involve asking how much of our experience has always depended upon sound guiding us in ways we barely notice. Once designers lose many familiar methods for directing attention, sound begins moving from the background towards the centre of interaction itself.