Sound Design at Edinburgh Napier University

Category: Sound Design

  • How Can Sound Become an Interface? Professor Stephen Brewster on Non-Speech Audio, Multimodal Interaction, and Designing Beyond the Screen

    Professor Stephen Brewster

    What happens when looking at a screen is no longer the best option?

    Computing has become increasingly mobile. Phones accompany people through cities, workplaces, public transport systems, shops, festivals, and countless other environments. Yet much interaction design still assumes that users can devote their attention to a display whenever information needs to be communicated. During his online guest lecture for Edinburgh Napier University, Professor Stephen Brewster challenged that assumption. Drawing on decades of research in human-computer interaction, multimodal interfaces, auditory displays, sonification, and mobile computing, he explored a deceptively simple question. What happens when information is communicated through sound rather than vision?

    Brewster began by situating the discussion within a broader problem. Human beings possess multiple senses, though much digital technology continues to privilege vision above all others. Screens dominate contemporary computing. Menus, notifications, progress indicators, maps, messages, and data visualisations typically assume that users are willing and able to look. Yet many situations challenge this assumption. Someone cycling through traffic cannot continuously monitor a display. A pedestrian navigating a crowded city may already be dividing attention between multiple tasks. Bright sunlight can render screens difficult to read. Some users experience visual impairments. Others simply have more pressing demands on their attention than a device in their hand. Rather than treating these situations as exceptions, Brewster suggested they reveal a limitation in conventional interface design. If visual attention is unavailable, how else might information be communicated?

    This question has shaped much of his research. Rather than viewing sound as decoration or enhancement, Brewster approaches it as a communication channel. Sound can operate while users look elsewhere. It can communicate information rapidly. It can support accessibility. It can function alongside vision rather than competing with it. The goal is not to replace screens entirely. Instead, it is to make fuller use of the sensory capabilities people already possess. Multimodal interaction, as Brewster described it, involves designing systems that acknowledge how people actually experience the world rather than assuming that vision should always dominate.

    Mobile devices provided an especially important motivation throughout the lecture. Traditional desktop computing emerged within relatively controlled environments. Users sat at desks, faced screens, and focused primarily on a single task. Mobile computing transformed those assumptions. People now interact with technology while moving through complex environments filled with competing demands upon their attention. A larger display cannot solve every problem. In many situations, the challenge is not the quantity of visual information available. The challenge is finding ways to communicate information without requiring users to look at all. Brewster argued that interaction design should respond to these realities rather than simply shrinking desktop interfaces onto smaller screens.

    Attention emerged as a recurring concern throughout the lecture. Many interface designs implicitly assume that information should compete for attention whenever it becomes available. Notifications flash. Windows appear. Alerts demand immediate responses. Yet everyday life rarely operates in this way. People constantly manage multiple streams of information simultaneously. Conversations continue while walking. Music plays while working. Environmental sounds remain present while attention shifts elsewhere. Brewster’s work asks whether digital systems might learn from these patterns. Rather than repeatedly interrupting users, could information move fluidly between foreground and background depending upon circumstances? Sound appears particularly well suited to this challenge. Unlike visual displays, which generally require direct attention, auditory information can remain available while users focus elsewhere. The question is not simply whether sound can communicate information. It is whether sound can communicate information without constantly demanding attention.

    One reason sound becomes attractive in this context is its efficiency. Speech can communicate detailed information, though it requires time. A spoken message unfolds word by word. Non-speech audio can often communicate information much more rapidly. Brewster compared the relationship between speech and non-speech sound to the relationship between text and icons. A paragraph may describe an object in detail. An icon can often communicate a similar idea almost instantly. Carefully designed sounds can function in much the same way. Rather than reading information aloud, they communicate status, warnings, activity, trends, or relationships through concise auditory cues.

    Much of the lecture explored different approaches to designing these cues. One of the earliest involved earcons, structured auditory messages built from musical elements such as rhythm, pitch, timbre, and tempo. Unlike everyday sounds, earcons are abstract. Their meaning must be learned. Yet this abstraction also provides flexibility. Brewster demonstrated how simple auditory components can be combined to create larger structures capable of communicating increasingly complex information. A particular rhythm might signal an error. Changes in timbre or pitch might identify different categories of error. Much like language, the system develops a vocabulary from smaller building blocks. Users invest effort in learning the code, though once acquired it can support sophisticated communication through relatively simple sounds.

    Auditory icons take a rather different approach. Instead of relying upon abstract structures, they exploit familiar sounds drawn from everyday experience. Brewster discussed William Gaver’s influential SonicFinder project, which mapped computer operations onto recognisable environmental sounds. Selecting a folder might produce the sound of paper. Dragging an object across the desktop might generate a scraping sound. Deleting a file might end with breaking glass. Such sounds often require little training. Their meaning emerges from existing associations. Yet the approach also reveals interesting limitations. Everyday life contains only a finite number of obvious metaphors. As software functions become more specialised, finding intuitive sonic equivalents becomes increasingly difficult. What sound represents copying a file rather than moving it? What sound represents a menu hierarchy? Questions such as these expose the challenges that emerge when designers depend upon metaphor alone.

    A third approach, sonification, shifts attention away from interfaces and towards data. Here, numerical values are mapped onto auditory parameters such as pitch, rhythm, or timbre. Brewster compared the process to visualisation. Graphs provide rapid access to patterns that would be difficult to identify within tables of numbers. Sonification attempts to achieve something similar through listening. By converting data into sound, listeners can often identify trends, anomalies, peaks, and relationships that might otherwise remain hidden. Rather than replacing detailed numerical information, sonification provides an overview. It allows users to perceive the broader shape of a dataset before examining specific values.

    Questions from students helped illuminate this distinction further. One example involved pollen data transformed into sound through changing pitches. The goal was not to communicate precise measurements. Instead, listeners could quickly identify whether levels were increasing, decreasing, or remaining stable. Brewster argued that this reflects the real strength of sonification. A graph rarely succeeds solely through precision. It succeeds by revealing patterns. Sonification can achieve a similar outcome through auditory perception. Numerical detail remains available when required, though sound offers a rapid way of monitoring change over time.

    Several studies discussed during the lecture demonstrated how even relatively simple sounds can influence interaction. One experiment examined numerical data entry on mobile devices. Participants entered information using either large visual buttons or substantially smaller alternatives. Predictably, performance declined when the buttons became smaller. Yet when simple auditory feedback was added, performance improved dramatically. Users working with the smaller controls performed almost as well as those using larger buttons. The sounds themselves were uncomplicated. Their value lay not in complexity but in the additional information they provided. By reducing uncertainty during interaction, they made the task easier to perform.

    Another particularly elegant example involved progress indicators. Most software communicates progress visually through bars that gradually fill across a display. Brewster and colleagues explored whether similar information could be represented spatially through sound. As a task progressed, a sound moved around the listener’s head. Position communicated completion. Movement communicated change. Without looking at a screen, users could estimate how far a process had progressed and whether activity had stalled.

    During the discussion period, students questioned whether such displays might become intrusive. Brewster responded by drawing attention to forms of ambient awareness that already exist within everyday life. People rarely focus continuously on air-conditioning systems, distant traffic, rainfall, or background conversations. Such sounds remain available without demanding constant attention. Auditory displays, he suggested, can function in a similar way. Information remains present when required, fading into the background when it is not. This idea runs through much of his research. Sound is not always most effective when it occupies the foreground. Sometimes its greatest strength lies in supporting awareness without interruption.

    Spatial audio appeared repeatedly throughout the lecture as a particularly rich area for exploration. Rather than treating sound as something emitted from a single speaker, Brewster investigated how information might be organised around listeners in three-dimensional space. Progress indicators could move around the head. Calendar entries could occupy positions corresponding to times of day. Menu items could exist within an auditory environment rather than a visual one. These systems exploit the human ability to localise sound sources, transforming listening into a form of navigation. Information acquires location. Interaction becomes spatial rather than purely symbolic.

    Some of the most imaginative projects discussed during the lecture extended these principles into everyday environments. AudioFeeds transformed social media activity into ambient soundscapes. Twitter, Facebook, news feeds, and other information streams occupied different locations within auditory space, represented through distinct families of sounds. Rather than repeatedly checking a screen, users could maintain a broader awareness of activity through listening. Detailed information remained available when required, though constant checking became unnecessary.

    The significance of AudioFeeds extends beyond social media. The project raises broader questions about how digital information should occupy everyday life. Many contemporary systems assume that awareness requires direct inspection. Brewster’s work suggests alternatives. Awareness may emerge gradually. Information may remain peripheral until circumstances make it relevant. In this respect, auditory displays resemble many naturally occurring environmental sounds. People rarely monitor rainfall continuously, though they remain aware that it is raining. They do not focus constantly on traffic outside a window, though they often notice when conditions change. Sound supports forms of awareness that differ from the all-or-nothing relationship often associated with visual attention.

    Pulse extended these ideas into urban environments. During the Edinburgh Festival, geolocated tweets became spatial audio cues distributed around the city. The project transformed social activity into something that could be heard rather than viewed. Participants were not presented with lists of events ranked by popularity, nor were they required to consult maps repeatedly. Instead, they developed a sense of where activity was occurring through listening.

    One of the most interesting aspects of the project is that it occupied a space between navigation and exploration. Traditional navigation systems attempt to guide users towards predetermined destinations. Pulse encouraged discovery instead. Participants moved towards sounds that suggested activity, curiosity, or interest. Information became something encountered rather than simply retrieved. In doing so, the project demonstrated how auditory displays can support forms of engagement that differ substantially from conventional graphical interfaces.

    The lecture concluded with one of Brewster’s more recent ideas: musicons. Earcons require designers to construct sounds from scratch. Musicons instead draw upon music that listeners already know. Research revealed that people consistently identify particular moments within familiar songs as especially representative. Often these moments involve vocals, choruses, or distinctive melodic features. By extracting such fragments, it becomes possible to create recognisable auditory cues from a user’s existing music collection. The appeal lies partly in familiarity. Rather than learning a completely new auditory language, users rely upon associations they already possess. Recognition emerges from memory rather than training.

    Musicons reveal another recurring theme in Brewster’s work. Successful interfaces rarely begin from technology alone. They begin from existing human abilities. Earcons ask users to learn a new auditory language. Musicons exploit knowledge that listeners already possess. A few notes from a familiar song may be recognised almost instantly. Years of listening experience become part of the interface itself.

    Looking across the different projects discussed during the lecture, it becomes clear that Brewster is addressing a much larger question than how to design better sounds. The deeper issue concerns the relationship between people and technology. Modern computing frequently competes with the surrounding world for attention. Screens draw the eye away from streets, conversations, environments, and other people. Brewster’s work suggests that alternative relationships may be possible.

    Sound occupies a distinctive position within this discussion. It can communicate information while allowing users to continue looking elsewhere. It can support awareness without requiring constant inspection. It can reveal patterns within data, provide feedback during interaction, and create new forms of accessibility. Most importantly, it can coexist with other activities rather than replacing them.

    None of this means that sound should replace vision. Brewster repeatedly emphasised the value of multimodal design rather than sensory competition. Different senses possess different strengths. The challenge for interaction designers is understanding how those strengths can complement one another. Sound becomes most useful not when it attempts to imitate visual displays, but when it contributes capabilities that vision alone cannot easily provide.

    For many people, digital interaction has become almost synonymous with looking at screens. Brewster’s lecture offered a reminder that computing does not need to be confined to vision. Human beings hear, touch, move, and orient themselves within space. Designing for those abilities opens possibilities that extend far beyond the display. In that sense, the lecture was not really about sound alone. It was about recognising the full range of ways people experience the world.

  • Why Record Everything? Ric Viers on Sound Effects Libraries, Creative Possibility, and Listening for Opportunity

    Ric Viers

    Why record everything?

    Many sound designers spend years learning how to remove unwanted sounds from recordings. They search for quieter locations, better microphones, cleaner signal paths, and more controlled recording environments. During his online guest lecture for Edinburgh Napier University, sound designer, recordist, publisher, and author Ric Viers approached the problem from a rather different direction. Again and again, he encouraged students to record more, not less. More locations. More variations. More experiments. More sounds that might initially appear useless.

    The advice runs against much conventional recording practice. Storage fills quickly. Editing becomes more demanding. Organisation becomes more complicated. Yet Viers argued that one of the greatest mistakes a sound designer can make is deciding too early what will or will not be useful. Throughout the lecture, he repeatedly returned to a simple idea: many of the most valuable sounds reveal their potential only later.

    The immediate context for the discussion was the creation of commercial sound effects libraries. Viers guided students through the process he uses when developing libraries for Blastwave FX, beginning with the choice of a topic, category, or theme. Some libraries focus on a specific class of sounds, such as footsteps. Others are organised around broader concepts, such as a zombie apocalypse, requiring everything from impacts and gunfire to environmental ambiences, destruction effects, creatures, weather, machinery, and countless other elements. Yet selecting a theme was only the beginning.

    Considerably more time, he suggested, is often spent researching than recording. Before microphones are unpacked, he studies films, television programmes, games, applications, and existing libraries to understand what has already been recorded, what is missing, and where opportunities may exist. Commercial sound libraries do not emerge from recording sessions alone. They emerge from identifying gaps. A successful library must offer something that people cannot already obtain elsewhere. Recording therefore begins with investigation. What sounds are difficult to find? Which sounds have become overused? Which categories remain poorly represented? Questions such as these help determine where effort should be directed.

    Planning extends far beyond selecting a subject. Viers described the creation of extensive scavenger lists containing every conceivable sound that might belong in a library. The exercise draws heavily upon what he called blue-sky thinking, an approach in which ideas are generated before they are evaluated. Impractical suggestions are welcomed. Expensive suggestions are welcomed. Unlikely suggestions are welcomed. The purpose is not to determine whether an idea is immediately achievable. The purpose is to widen the range of possibilities. Viers argued that ideas often develop through association. A suggestion that cannot be pursued directly may still help identify a different route towards the same goal.

    A recurring theme in the lecture was the cultivation of listening as a habit. Ideas for sounds are often collected long before any recording session begins. A strange resonance in a pipe. The texture of metal scraping against metal. An unusual mechanical vibration. A sound designer’s work, in his view, begins long before the recorder is switched on. Listening becomes a form of continuous observation. Ideas are captured in notebooks, mobile apps, or voice memos. Some notes describe specific sounds. Others record textures, qualities, or possibilities.

    One example illustrated this way of thinking particularly clearly. While dealing with a blocked drain, Viers became fascinated by the sound produced as liquid moved through the pipework. Most people would simply hear a drain. Viers heard something else. The sound possessed qualities that might later become useful in an entirely different context. He immediately made a note to revisit the sound in the future. What interested him was not the object itself. It was the texture. The eventual application remained unknown. The possibility was enough.

    This distinction between objects and textures appeared repeatedly throughout the lecture. Sound designers are often asked where particular sounds come from. Audiences frequently imagine a straightforward relationship between source and result. A door sound comes from a door. An engine sound comes from an engine. Viers described a different way of thinking. A useful recording is not necessarily valuable for what it is. It may be valuable for characteristics that become apparent only after editing, processing, layering, or transformation. Recording therefore involves collecting materials whose eventual use remains unknown rather than merely documenting objects.

    Many lectures on sound design focus heavily on equipment. Microphones, recorders, plug-ins, and software frequently dominate discussions. Viers spent surprisingly little time discussing technology in isolation. When he addressed recording practice, attention remained focused on listening. Before recording in any location, he advocated standing still and listening carefully to the environment. Air conditioning systems, insect activity, traffic patterns, aircraft, electrical noise, and countless other factors become relevant once attention shifts from simply hearing a location to actively analysing it.

    This process of scouting locations received considerable attention. Viers argued that many people move through environments without noticing their acoustic details. Recording requires a different form of awareness. Insects become important. Distant roads become important. Wind direction becomes important. Time of day becomes important. A location that appears perfect at one moment may become unusable an hour later. Successful field recording often depends less upon equipment than upon patience, observation, and preparation.

    This concern with awareness also explains his insistence on monitoring continuously through headphones. Microphones do not hear the world in quite the same way people do. Wearing headphones while moving through an environment reveals details that might otherwise remain unnoticed. Interesting sounds are often discovered rather than sought. What appears unremarkable at first may become compelling when heard through a microphone. Recording therefore becomes an ongoing process of discovery rather than simply the execution of a predetermined plan.

    A similar principle shaped his approach to recording itself. Whenever possible, he records multiple takes. Fast versions. Slow versions. Loud versions. Quiet versions. Different perspectives. Different performances. On one level, this provides insurance against technical problems. On another, it reflects a deeper belief about sound design. Sounds rarely remain confined to their original purpose. A recording made for one project may later become useful in another. A variation that seems unnecessary today may become exactly what a future project requires.

    Experience had also taught him how easily apparently successful recording sessions can fail. During one project involving emergency vehicles, extensive access was arranged at a fire station. Recordings were captured, equipment functioned correctly, and everything appeared successful. Only later did the team discover that powerful sirens had physically affected the recording medium itself. Material that seemed secure had effectively been lost. The story was not presented as a technological curiosity. It explained why professional recordists often develop habits that appear excessive to newcomers. Additional takes, backups, and redundancy emerge from experience rather than paranoia.

    A bee entered the Foley studio while Viers was working on an unrelated project. The original plan was simply to remove it and continue working. An intern suggested recording it instead. That decision eventually led to an entire library of insect sounds, combining recordings of flies, bees, crickets, and other insects with carefully performed Foley designed to represent insect movement. The significance of the story lies less in the insect itself than in the response. The opportunity was not planned. It appeared unexpectedly. Remaining open to such moments allowed a chance event to become the basis of a completely new collection.

    Questions of organisation formed another important part of the lecture. Recording more sounds creates a practical problem. How can those sounds be found again months or years later? Viers discussed the importance of cataloguing, naming conventions, metadata, and library management. Collecting large quantities of material is only useful if that material remains accessible. A sound hidden inside thousands of poorly organised files may effectively disappear. The ability to locate recordings quickly becomes part of the creative process itself.

    The same concern with organisation appeared in his discussion of large-scale sound design projects. One example involved the construction of a tornado sequence containing roughly 180 individual tracks. Projects of this scale quickly expose weaknesses in workflow. Tiny editing errors become difficult to locate. Artefacts become buried within hundreds of layers. Seemingly minor organisational decisions accumulate into major practical consequences. Preparation therefore serves creative goals. Time spent organising material makes experimentation easier later.

    Recording occupied only part of the process. Viers repeatedly returned to what happens afterwards. Sounds are collected, edited, organised, and transformed. Recordings function as materials that can be combined, layered, stretched, pitched, and manipulated into entirely new forms.

    One example involved the creation of a failing fluorescent light. Unable to find exactly the sound he wanted, Viers began experimenting with alternative sources. The eventual solution came from an unexpectedly small fragment of fruit being crushed. Through editing and transformation, the recording acquired the qualities required for the scene. The finished sound bore little resemblance to its source. Yet this was precisely the point. The identity of the source mattered less than the acoustic properties it contained.

    The same logic appeared in Viers’ discussion of so-called bad recordings. Students often expect professional sound design to involve strict distinctions between useful and useless material. Viers challenged that assumption directly. During the discussion, he argued that there is rarely such a thing as a completely bad sound. Recordings that fail in one context may become valuable in another. Noise, distortion, clipping, and other imperfections can sometimes serve as raw material for later experimentation.

    One example involved a recording that initially appeared unusable. Hidden within the material was the sound of a cat. Rather than discarding the recording, Viers began manipulating fragments of it through processing, layering, and transformation. Elements that seemed worthless in their original form became the basis of drones, textures, and entirely different production sounds. The value of the recording emerged through later use rather than immediate judgement.

    Discussion of careers and commercial practice returned to the same issue. Students often assume that success depends upon following established models. Viers argued almost the opposite. He encouraged students to develop their own interests, methods, and creative identities. Distinctive approaches create opportunities. If everyone records the same sounds in the same way, there is little reason for anyone to choose one library over another.

    Recording, editing, organisation, publishing, and marketing occupied much of the lecture.

    Running through all of them was the same underlying concern: how to recognise useful material before its eventual value becomes obvious. Throughout the discussion, Viers repeatedly challenged the assumption that the usefulness of a sound can be determined immediately. A recording that appears unremarkable today may become the foundation of a future project. A failed recording may later prove valuable once new tools become available. A sound collected for one purpose may eventually find a completely different use.

    Many people encounter the world as a collection of familiar objects and events. Viers encouraged students to listen differently. A drain becomes a source of textures. A mechanical vibration becomes source material for a creature or machine. A crushed piece of fruit becomes a fluorescent light. An unexpected insect in a Foley studio becomes the starting point for an entirely new library. A sound’s future use is often difficult to predict when it is first recorded.

    Sound effects libraries occupy an unusual position within creative practice. They are archives of past recordings, though they are also collections of future possibilities. Every recording preserves an opportunity whose eventual use remains unknown. Viers’ argument was not simply that sound designers should record more sounds. It was that they should remain open to possibilities that have not yet revealed themselves. A recorder captures a sound at a particular moment. What that sound eventually becomes often remains an open question.