Sound Design at Edinburgh Napier University

Category: Sonification

  • How Can Sound Become an Interface? Professor Stephen Brewster on Non-Speech Audio, Multimodal Interaction, and Designing Beyond the Screen

    Professor Stephen Brewster

    What happens when looking at a screen is no longer the best option?

    Computing has become increasingly mobile. Phones accompany people through cities, workplaces, public transport systems, shops, festivals, and countless other environments. Yet much interaction design still assumes that users can devote their attention to a display whenever information needs to be communicated. During his online guest lecture for Edinburgh Napier University, Professor Stephen Brewster challenged that assumption. Drawing on decades of research in human-computer interaction, multimodal interfaces, auditory displays, sonification, and mobile computing, he explored a deceptively simple question. What happens when information is communicated through sound rather than vision?

    Brewster began by situating the discussion within a broader problem. Human beings possess multiple senses, though much digital technology continues to privilege vision above all others. Screens dominate contemporary computing. Menus, notifications, progress indicators, maps, messages, and data visualisations typically assume that users are willing and able to look. Yet many situations challenge this assumption. Someone cycling through traffic cannot continuously monitor a display. A pedestrian navigating a crowded city may already be dividing attention between multiple tasks. Bright sunlight can render screens difficult to read. Some users experience visual impairments. Others simply have more pressing demands on their attention than a device in their hand. Rather than treating these situations as exceptions, Brewster suggested they reveal a limitation in conventional interface design. If visual attention is unavailable, how else might information be communicated?

    This question has shaped much of his research. Rather than viewing sound as decoration or enhancement, Brewster approaches it as a communication channel. Sound can operate while users look elsewhere. It can communicate information rapidly. It can support accessibility. It can function alongside vision rather than competing with it. The goal is not to replace screens entirely. Instead, it is to make fuller use of the sensory capabilities people already possess. Multimodal interaction, as Brewster described it, involves designing systems that acknowledge how people actually experience the world rather than assuming that vision should always dominate.

    Mobile devices provided an especially important motivation throughout the lecture. Traditional desktop computing emerged within relatively controlled environments. Users sat at desks, faced screens, and focused primarily on a single task. Mobile computing transformed those assumptions. People now interact with technology while moving through complex environments filled with competing demands upon their attention. A larger display cannot solve every problem. In many situations, the challenge is not the quantity of visual information available. The challenge is finding ways to communicate information without requiring users to look at all. Brewster argued that interaction design should respond to these realities rather than simply shrinking desktop interfaces onto smaller screens.

    Attention emerged as a recurring concern throughout the lecture. Many interface designs implicitly assume that information should compete for attention whenever it becomes available. Notifications flash. Windows appear. Alerts demand immediate responses. Yet everyday life rarely operates in this way. People constantly manage multiple streams of information simultaneously. Conversations continue while walking. Music plays while working. Environmental sounds remain present while attention shifts elsewhere. Brewster’s work asks whether digital systems might learn from these patterns. Rather than repeatedly interrupting users, could information move fluidly between foreground and background depending upon circumstances? Sound appears particularly well suited to this challenge. Unlike visual displays, which generally require direct attention, auditory information can remain available while users focus elsewhere. The question is not simply whether sound can communicate information. It is whether sound can communicate information without constantly demanding attention.

    One reason sound becomes attractive in this context is its efficiency. Speech can communicate detailed information, though it requires time. A spoken message unfolds word by word. Non-speech audio can often communicate information much more rapidly. Brewster compared the relationship between speech and non-speech sound to the relationship between text and icons. A paragraph may describe an object in detail. An icon can often communicate a similar idea almost instantly. Carefully designed sounds can function in much the same way. Rather than reading information aloud, they communicate status, warnings, activity, trends, or relationships through concise auditory cues.

    Much of the lecture explored different approaches to designing these cues. One of the earliest involved earcons, structured auditory messages built from musical elements such as rhythm, pitch, timbre, and tempo. Unlike everyday sounds, earcons are abstract. Their meaning must be learned. Yet this abstraction also provides flexibility. Brewster demonstrated how simple auditory components can be combined to create larger structures capable of communicating increasingly complex information. A particular rhythm might signal an error. Changes in timbre or pitch might identify different categories of error. Much like language, the system develops a vocabulary from smaller building blocks. Users invest effort in learning the code, though once acquired it can support sophisticated communication through relatively simple sounds.

    Auditory icons take a rather different approach. Instead of relying upon abstract structures, they exploit familiar sounds drawn from everyday experience. Brewster discussed William Gaver’s influential SonicFinder project, which mapped computer operations onto recognisable environmental sounds. Selecting a folder might produce the sound of paper. Dragging an object across the desktop might generate a scraping sound. Deleting a file might end with breaking glass. Such sounds often require little training. Their meaning emerges from existing associations. Yet the approach also reveals interesting limitations. Everyday life contains only a finite number of obvious metaphors. As software functions become more specialised, finding intuitive sonic equivalents becomes increasingly difficult. What sound represents copying a file rather than moving it? What sound represents a menu hierarchy? Questions such as these expose the challenges that emerge when designers depend upon metaphor alone.

    A third approach, sonification, shifts attention away from interfaces and towards data. Here, numerical values are mapped onto auditory parameters such as pitch, rhythm, or timbre. Brewster compared the process to visualisation. Graphs provide rapid access to patterns that would be difficult to identify within tables of numbers. Sonification attempts to achieve something similar through listening. By converting data into sound, listeners can often identify trends, anomalies, peaks, and relationships that might otherwise remain hidden. Rather than replacing detailed numerical information, sonification provides an overview. It allows users to perceive the broader shape of a dataset before examining specific values.

    Questions from students helped illuminate this distinction further. One example involved pollen data transformed into sound through changing pitches. The goal was not to communicate precise measurements. Instead, listeners could quickly identify whether levels were increasing, decreasing, or remaining stable. Brewster argued that this reflects the real strength of sonification. A graph rarely succeeds solely through precision. It succeeds by revealing patterns. Sonification can achieve a similar outcome through auditory perception. Numerical detail remains available when required, though sound offers a rapid way of monitoring change over time.

    Several studies discussed during the lecture demonstrated how even relatively simple sounds can influence interaction. One experiment examined numerical data entry on mobile devices. Participants entered information using either large visual buttons or substantially smaller alternatives. Predictably, performance declined when the buttons became smaller. Yet when simple auditory feedback was added, performance improved dramatically. Users working with the smaller controls performed almost as well as those using larger buttons. The sounds themselves were uncomplicated. Their value lay not in complexity but in the additional information they provided. By reducing uncertainty during interaction, they made the task easier to perform.

    Another particularly elegant example involved progress indicators. Most software communicates progress visually through bars that gradually fill across a display. Brewster and colleagues explored whether similar information could be represented spatially through sound. As a task progressed, a sound moved around the listener’s head. Position communicated completion. Movement communicated change. Without looking at a screen, users could estimate how far a process had progressed and whether activity had stalled.

    During the discussion period, students questioned whether such displays might become intrusive. Brewster responded by drawing attention to forms of ambient awareness that already exist within everyday life. People rarely focus continuously on air-conditioning systems, distant traffic, rainfall, or background conversations. Such sounds remain available without demanding constant attention. Auditory displays, he suggested, can function in a similar way. Information remains present when required, fading into the background when it is not. This idea runs through much of his research. Sound is not always most effective when it occupies the foreground. Sometimes its greatest strength lies in supporting awareness without interruption.

    Spatial audio appeared repeatedly throughout the lecture as a particularly rich area for exploration. Rather than treating sound as something emitted from a single speaker, Brewster investigated how information might be organised around listeners in three-dimensional space. Progress indicators could move around the head. Calendar entries could occupy positions corresponding to times of day. Menu items could exist within an auditory environment rather than a visual one. These systems exploit the human ability to localise sound sources, transforming listening into a form of navigation. Information acquires location. Interaction becomes spatial rather than purely symbolic.

    Some of the most imaginative projects discussed during the lecture extended these principles into everyday environments. AudioFeeds transformed social media activity into ambient soundscapes. Twitter, Facebook, news feeds, and other information streams occupied different locations within auditory space, represented through distinct families of sounds. Rather than repeatedly checking a screen, users could maintain a broader awareness of activity through listening. Detailed information remained available when required, though constant checking became unnecessary.

    The significance of AudioFeeds extends beyond social media. The project raises broader questions about how digital information should occupy everyday life. Many contemporary systems assume that awareness requires direct inspection. Brewster’s work suggests alternatives. Awareness may emerge gradually. Information may remain peripheral until circumstances make it relevant. In this respect, auditory displays resemble many naturally occurring environmental sounds. People rarely monitor rainfall continuously, though they remain aware that it is raining. They do not focus constantly on traffic outside a window, though they often notice when conditions change. Sound supports forms of awareness that differ from the all-or-nothing relationship often associated with visual attention.

    Pulse extended these ideas into urban environments. During the Edinburgh Festival, geolocated tweets became spatial audio cues distributed around the city. The project transformed social activity into something that could be heard rather than viewed. Participants were not presented with lists of events ranked by popularity, nor were they required to consult maps repeatedly. Instead, they developed a sense of where activity was occurring through listening.

    One of the most interesting aspects of the project is that it occupied a space between navigation and exploration. Traditional navigation systems attempt to guide users towards predetermined destinations. Pulse encouraged discovery instead. Participants moved towards sounds that suggested activity, curiosity, or interest. Information became something encountered rather than simply retrieved. In doing so, the project demonstrated how auditory displays can support forms of engagement that differ substantially from conventional graphical interfaces.

    The lecture concluded with one of Brewster’s more recent ideas: musicons. Earcons require designers to construct sounds from scratch. Musicons instead draw upon music that listeners already know. Research revealed that people consistently identify particular moments within familiar songs as especially representative. Often these moments involve vocals, choruses, or distinctive melodic features. By extracting such fragments, it becomes possible to create recognisable auditory cues from a user’s existing music collection. The appeal lies partly in familiarity. Rather than learning a completely new auditory language, users rely upon associations they already possess. Recognition emerges from memory rather than training.

    Musicons reveal another recurring theme in Brewster’s work. Successful interfaces rarely begin from technology alone. They begin from existing human abilities. Earcons ask users to learn a new auditory language. Musicons exploit knowledge that listeners already possess. A few notes from a familiar song may be recognised almost instantly. Years of listening experience become part of the interface itself.

    Looking across the different projects discussed during the lecture, it becomes clear that Brewster is addressing a much larger question than how to design better sounds. The deeper issue concerns the relationship between people and technology. Modern computing frequently competes with the surrounding world for attention. Screens draw the eye away from streets, conversations, environments, and other people. Brewster’s work suggests that alternative relationships may be possible.

    Sound occupies a distinctive position within this discussion. It can communicate information while allowing users to continue looking elsewhere. It can support awareness without requiring constant inspection. It can reveal patterns within data, provide feedback during interaction, and create new forms of accessibility. Most importantly, it can coexist with other activities rather than replacing them.

    None of this means that sound should replace vision. Brewster repeatedly emphasised the value of multimodal design rather than sensory competition. Different senses possess different strengths. The challenge for interaction designers is understanding how those strengths can complement one another. Sound becomes most useful not when it attempts to imitate visual displays, but when it contributes capabilities that vision alone cannot easily provide.

    For many people, digital interaction has become almost synonymous with looking at screens. Brewster’s lecture offered a reminder that computing does not need to be confined to vision. Human beings hear, touch, move, and orient themselves within space. Designing for those abilities opens possibilities that extend far beyond the display. In that sense, the lecture was not really about sound alone. It was about recognising the full range of ways people experience the world.

  • Why Is Data So Quiet? Hugh McGrory on Sonification, Accessibility, and the Future of Information

    Hugh McGrory

    Why is data so quiet?

    Modern life is shaped by data. Governments collect it. Businesses depend upon it. Scientists analyse it. Social media platforms generate vast quantities of it every second. Increasingly, decisions about healthcare, transportation, education, finance, climate, and public policy are informed by information that exists primarily as data. Yet despite its growing importance, most people encounter data in remarkably similar ways. We see charts, graphs, dashboards, spreadsheets, maps, and visualisations. We are expected to look at information rather than listen to it.

    During his online guest lecture for Edinburgh Napier University, Hugh McGrory challenged this assumption. Drawing upon a career that has spanned animation, virtual reality, software development, data storytelling, and sonification, McGrory described a field that sits between sound, design, accessibility, and communication. Across a wide-ranging discussion that moved from GPS navigation to astronomy, podcasting, climate data, artificial intelligence, and urban infrastructure, he repeatedly returned to a deceptively simple question. If data is everywhere, why do we still experience almost all of it through our eyes?

    McGrory’s route into sonification was anything but conventional. Beginning in computer animation and experimental digital media, he later worked with medical imaging researchers at Yale University, where he encountered scientific data in a completely new way. Rather than using cameras to create images, researchers were transforming data into visual representations that scientists could study and interpret. Later work in virtual reality continued this fascination with information and how people interact with it. Yet throughout these experiences, one issue kept resurfacing. Data communication was overwhelmingly visual. Even the language reflects this bias. The field is known as data visualisation. Information is generally assumed to become meaningful once it has been converted into something that can be seen.

    For McGrory, this raises an obvious question. Why should vision carry so much of the burden? The field of sonification attempts to address this imbalance by exploring how information can be communicated through sound. Yet McGrory encouraged students to think beyond narrow academic definitions. The challenge is not simply turning data into audio. The challenge is deciding when sound might be a more useful way of communicating information. GPS navigation provides a useful example. Rather than forcing drivers to consult maps continuously, navigation systems deliver information precisely when it becomes relevant. A driver does not need every detail about the surrounding road network. They need to know when to turn left. Effective sonification follows the same principle. Its purpose is not to communicate everything. Its purpose is to communicate what matters. Throughout the lecture, McGrory repeatedly argued that the modern problem is rarely a lack of information. More often, it is an excess of information. The real design challenge lies in deciding what should reach people, when it should reach them, and how it can be communicated without demanding unnecessary attention.

    This perspective places sonification within a much broader discussion about interface design. Many of the tools through which people still interact with information were developed for a world in which data was far less abundant than it is today. Screens, keyboards, menus, and dashboards remain remarkably successful technologies, though they are not always suited to situations in which people are moving through complex environments while simultaneously performing other tasks. McGrory pointed towards cities as a particularly interesting example. Vast quantities of public information now exist concerning transportation systems, environmental monitoring, infrastructure, weather, traffic, and public services. Much of this information could potentially support everyday decision-making, yet most people never encounter it. One reason is that visual interfaces demand attention. Looking at a screen competes with countless other activities. Sound offers different possibilities. It can accompany movement, coexist with visual tasks, and communicate information without constantly demanding that people stop and look elsewhere.

    Questions of accessibility revealed why these issues matter so much. Much of McGrory’s work has involved collaboration with blind and visually impaired communities, experiences that challenged many of his assumptions about information design. Designers often assume that providing access to information is sufficient. The reality is considerably more complicated. Screen readers can successfully read large quantities of information aloud, though understanding the overall structure of that information remains difficult. McGrory compared this experience to attempting to complete a jigsaw puzzle without ever seeing the image on the box. Individual pieces are available, yet the broader picture remains difficult to grasp. A spreadsheet containing thousands of values can be read sequentially, though identifying patterns, relationships, and trends becomes far more challenging. Sonification offers one possible response to this problem. Rather than replacing detailed exploration, it can provide rapid overviews that help listeners understand the shape of information before investigating individual details.

    These experiences also led McGrory to question some established assumptions within sonification itself. Many projects focus heavily on transforming data into sound while paying comparatively little attention to context. Listeners are often presented with unfamiliar sounds and expected to interpret them independently. For McGrory, this represents a significant limitation. Communication rarely functions through raw information alone. Context, explanation, and narrative play equally important roles. Podcasting, journalism, and storytelling all demonstrate how audiences use framing to understand unfamiliar material. Projects such as the BBC’s Audiograph series combine sonification with narration, allowing listeners to understand not only what they are hearing but why it matters. This approach shifts attention away from sonification as a technical exercise and towards sonification as communication. Sound becomes one element within a broader process of explanation rather than an isolated solution expected to function independently.

    A particularly memorable example involved astronomy. McGrory discussed the work of a blind astronomer who uses sonification to explore stellar data, challenging assumptions about both astronomy and accessibility. At first glance, astronomy appears inseparable from visual observation. Popular images of galaxies, stars, and nebulae reinforce the assumption that astronomical knowledge depends upon sight. Yet the example revealed something much more interesting than accessibility alone. Rather than simply compensating for an inability to see, sonification provided an alternative way of engaging with information.

    This distinction became important throughout the lecture. Discussions of accessibility sometimes assume that non-visual approaches exist primarily to reproduce experiences that sighted people already have. McGrory encouraged a different perspective. Listening is not merely a substitute for seeing. It is a different mode of perception with its own strengths and limitations. Patterns that may be difficult to recognise visually can sometimes become more apparent through sound. Temporal relationships, repetition, variation, rhythm, and change often lend themselves naturally to auditory interpretation. For this reason, sonification can function as more than an accessibility tool. It can become an analytical tool. Different sensory approaches reveal different aspects of information. A graph may highlight one set of relationships. A sonification may reveal another. Rather than competing with visualisation, the two approaches can complement one another. The question is not whether data should be seen or heard. The question is what becomes possible when both options are available.

    Climate data provided another revealing case. Contemporary societies generate vast quantities of information about environmental change, though much of that information remains difficult for non-specialists to engage with meaningfully. Charts and graphs may communicate trends accurately, yet they do not necessarily encourage engagement. McGrory discussed projects that transform environmental datasets into musical or sonic experiences, creating opportunities for audiences to encounter information differently. Such approaches are not intended to replace scientific analysis. Rather, they create alternative routes into understanding. A sonification may not communicate every statistical detail, though it may encourage curiosity, emotional engagement, or reflection in ways that conventional visualisations struggle to achieve. Throughout the lecture, McGrory repeatedly returned to the idea that communication is not simply a matter of transmitting information. It is a matter of helping people connect with information in the first place.

    Underlying many of these examples was a broader argument about innovation and interdisciplinary thinking. McGrory repeatedly suggested that some of the most interesting developments emerge when disciplines that rarely interact begin to overlap. He cited a definition of innovation that particularly resonated with him: innovation happens when things that are separate become mixed. Sonification itself reflects this principle. The field draws simultaneously upon sound design, music, journalism, accessibility research, user experience design, data science, software development, artificial intelligence, and communication studies. No single discipline possesses all the answers. Progress emerges through collaboration between people approaching similar problems from different directions. This perspective also explains why McGrory consistently framed sound as part of larger design conversations rather than as an isolated specialism. Questions about how people receive information, understand systems, and engage with the world are not purely visual problems. They are design problems.

    These ideas become increasingly significant as emerging technologies continue to reshape everyday life. Artificial intelligence, conversational interfaces, spatial audio, augmented reality, and wearable computing all point towards futures in which information may become less dependent upon screens. While immersive visual technologies continue to evolve, audio already occupies a privileged position within contemporary life. Millions of people carry headphones throughout much of the day. Voice assistants have become commonplace. Podcasts reach global audiences. The infrastructure required for sophisticated auditory experiences already exists. For McGrory, the challenge is no longer technological feasibility. The challenge is learning how to design auditory experiences that are genuinely useful.

    Looking back across the lecture, perhaps the most striking aspect was the extent to which sonification emerged as a human problem rather than a technical one. Questions about mapping data to sound, selecting parameters, or designing auditory displays certainly matter, though they consistently led back to larger concerns about accessibility, communication, attention, understanding, and engagement. Again and again, McGrory encouraged students to think less about sound as an isolated discipline and more about what sound can contribute when combined with other ways of understanding the world.

    Data has become one of the defining materials of contemporary society. Increasingly, our institutions, technologies, economies, and daily lives depend upon it. Yet much of that information remains hidden behind interfaces designed primarily for looking rather than listening. For Hugh McGrory, this represents an enormous opportunity. Whenever information exists without sound, there is the possibility of creating new ways for people to experience, understand, and engage with it. More importantly, there is the possibility of discovering things that might otherwise remain hidden. Listening does not simply provide another route to the same destination. Sometimes it changes what can be found along the way.

    Perhaps that is why his central question continues to resonate long after the lecture ends. In a world increasingly shaped by data, why should our ears be left out of the conversation?

  • Who Are We Designing For? Professor Bruce Walker on Sound, Accessibility, and Human-Centred Design

    Bruce Walker

    Who are we designing for?

    At first glance, the answer appears obvious. Designers create products for users. Engineers build systems to help people accomplish tasks. Technology exists to solve problems. Yet during his online guest lecture for Edinburgh Napier University, Professor Bruce Walker repeatedly returned to examples suggesting that the answer is often more complicated than it first appears. Again and again, he described situations in which technically impressive systems failed to account for the realities of the people expected to use them. A solution might work perfectly according to engineering specifications while proving frustrating, distracting, or simply undesirable in practice. Across projects involving navigation, education, museums, accessibility, sonification, and auditory interfaces, Walker argued that successful design begins not with technology but with understanding human needs.

    Sound provided the central thread running through the lecture. Many people associate auditory interfaces with simple alerts and alarms. Computers beep. Phones ring. Vehicles issue warning tones. Yet Walker demonstrated that sound can serve far more sophisticated purposes. It can guide navigation, communicate data, support education, enable accessibility, assist decision-making, and provide entirely new ways of interacting with technology. These possibilities emerge not from adding sounds indiscriminately but from carefully considering what information people need, when they need it, and how they can use it effectively.

    This emphasis on understanding users before designing solutions appeared throughout the lecture. These broader questions became particularly visible in Walker’s discussion of the SWAN project, the System for Wearable Audio Navigation. The goal was deceptively simple. Could sound help people navigate when they could not rely on vision? Blind users are an obvious example, though Walker deliberately framed the problem more broadly. Firefighters moving through smoke-filled buildings, soldiers operating at night, divers underwater, and workers engaged in visually demanding tasks may all find themselves unable to look or unable to see. Rather than designing exclusively for a particular group, the project focused on a shared perceptual challenge.

    Approaching the problem in this way immediately changed the nature of the design process. Navigation might seem like a familiar activity, something most people perform every day without conscious thought. Yet once the team began analysing the task carefully, they discovered layers of complexity hidden beneath the surface. People need to know where to go, though they also need information about obstacles, changes in terrain, landmarks, hazards, and points of interest. Navigation is not simply a matter of moving from one location to another. It involves understanding an environment while continuously making decisions within it.

    The resulting system employed spatialised audio beacons that users could follow through space. Walker compared them to a virtual carrot suspended ahead of the listener. Rather than receiving a sequence of verbal instructions, users simply moved towards a sound source. When a turn became necessary, the sound shifted position accordingly. The concept appears elegant partly because it exploits abilities listeners already possess. Humans are remarkably good at locating sounds. Rather than teaching an entirely new interaction technique, the system builds upon existing perceptual skills.

    What makes the project particularly revealing, however, is the extent to which users shaped its development. Early assumptions frequently proved incorrect. Designers initially believed they should describe surface transitions in detail, informing users when they were moving from pavement to grass or from one surface to another. Users quickly pointed out that such information was often redundant. They already knew where they were standing. What mattered was knowing what was coming next. By listening carefully to users rather than insisting upon their original assumptions, the team produced a more efficient and less intrusive system.

    This pattern appeared repeatedly throughout the lecture. Successful auditory interfaces emerge through collaboration, observation, and evaluation rather than through technological enthusiasm alone. Walker repeatedly emphasised the importance of testing systems with real users performing real tasks. Maps of navigation paths, performance data, error rates, and subjective feedback all played important roles in understanding whether a design genuinely worked. The question was never whether a system could be built. The question was whether it improved people’s ability to accomplish what they were trying to do.

    Perhaps nowhere was this philosophy more apparent than in the team’s work with bone-conduction audio. Navigation systems often rely upon stereo or spatial audio, which typically requires headphones. Yet many users rejected conventional headphones immediately. Blind people rely heavily upon environmental sounds. Firefighters need to hear what is happening around them. Covering the ears solved one problem while creating another. Rather than treating this as an unavoidable limitation, the researchers reconsidered the entire system. Bone-conduction devices allowed spatial audio to be delivered without blocking environmental awareness. Once again, the solution emerged not from pursuing technology for its own sake but from understanding the realities of users’ lives.

    Walker extended these ideas beyond navigation into the design of auditory menus and interfaces. Modern computer systems contain large amounts of information organised visually through menus, icons, scroll bars, and navigation structures. Translating these elements into sound presents significant challenges. Simply reading everything aloud through text-to-speech quickly becomes inefficient and frustrating. Long contact lists, extensive music libraries, and complex software systems require more sophisticated approaches.

    Many of the solutions developed by Walker and his collaborators demonstrate an intriguing combination of technical ingenuity and psychological insight. Spearcons, for example, compress spoken words into highly abbreviated auditory cues that users rapidly learn to recognise. Spindex systems provide indexing sounds that help listeners move efficiently through large collections without hearing every item in sequence. Whispered speech can indicate unavailable menu items while preserving the overall structure of an interface. These innovations are clever, though their importance lies less in their novelty than in their effectiveness. Each emerged through extensive experimentation designed to determine what users actually understood and preferred. What makes this work particularly interesting is that it was never simply about making visual interfaces audible. Instead, it explored how auditory interfaces might exploit the strengths of listening itself. Users do not experience sound in the same way they experience vision, and successful interfaces acknowledge that difference rather than treating audio as a substitute for a screen.

    The lecture repeatedly highlighted a distinction between engineering and design. Engineering can produce systems that function correctly. Design concerns whether people will actually want to use them. Walker discussed several examples of technically impressive systems that failed precisely because they neglected this distinction. Some provided too much information. Others demanded excessive attention. Many ignored the broader contexts within which people operate. A system may be capable of delivering enormous amounts of information, though that does not necessarily mean people want to receive it.

    Questions of accessibility broadened these concerns further. Much of Walker’s work focuses on enabling participation rather than merely providing access. He described projects supporting education for blind students, making scientific information more accessible, and improving experiences within museums and aquariums. These examples revealed another recurring theme. Accessibility is not simply about removing barriers. It is about ensuring that people can engage meaningfully with experiences, opportunities, and ideas.

    His work within museums and aquariums illustrates this particularly well. Modern cultural institutions often provide physical access while leaving informational and emotional access largely unresolved. A blind visitor may be able to enter an aquarium, though the experience remains fundamentally visual. Standing before an enormous tank filled with whale sharks and rays offers little if the most important aspects of the exhibit remain inaccessible. Walker’s team explored ways of using tracking systems, sonification, narration, and auditory displays to communicate not only what was present but what was happening. The objective was not merely to describe the environment. It was to create opportunities for engagement, curiosity, and wonder.

    Educational projects revealed similar concerns. Throughout the lecture, Walker repeatedly returned to the distinction between access and participation. Providing information is only one part of inclusion. Students also need opportunities to explore, question, discover, and develop understanding independently. Whether working with scientific data, classroom materials, museum exhibits, or public spaces, the challenge remained remarkably consistent. How can information be presented in ways that support meaningful engagement rather than passive reception? Accessibility, in this sense, becomes a question of design quality rather than a specialised feature introduced at the end of a project.

    Underlying many of these projects is the field of sonification, the practice of representing information through sound. Walker described sonification as both a design challenge and a research problem. Any attempt to translate data into sound requires decisions about mapping, scaling, timing, context, and interpretation. Should temperature be represented through pitch, loudness, or tempo? How should complex information be organised so that listeners can understand it? These questions have no universal answers. Effective solutions depend upon understanding both the data and the people expected to interpret it.

    One reason sonification remains challenging is that many listeners have relatively little experience interpreting information through sound. Graphs, charts, and maps are familiar cultural forms. Auditory representations are far less common. Designers therefore need to balance learnability with expressiveness. A system may communicate information accurately while remaining difficult to interpret. Conversely, a system may sound appealing while conveying very little. Walker’s research repeatedly demonstrated that effective sonification emerges through iterative testing with users rather than through theoretical assumptions alone.

    Such challenges reveal why Walker remains sceptical of simplistic approaches to auditory design. Throughout the lecture, he criticised the tendency to reduce audio interfaces to collections of arbitrary beeps and alerts. Sound possesses enormous communicative potential, though realising that potential requires careful thought. Designers must consider attention, context, usability, aesthetics, cultural expectations, and human behaviour. A sound that performs well in a laboratory may fail completely in everyday life. A technically accurate representation may prove ineffective if users cannot interpret it.

    The phrase “beeps and bops” became a useful shorthand for this problem. Many technologies employ sound only at the most superficial level, relying upon alerts, warnings, and notifications while overlooking the richer possibilities of auditory interaction. Walker’s work points towards a broader conception of sound, one capable of supporting navigation, exploration, learning, communication, and discovery. The challenge is not simply adding sound to technology. It is designing meaningful auditory experiences.

    Towards the end of the discussion, Walker reflected on what he described as a “failure of imagination” in technology design. Sometimes designers struggle to imagine how people actually live with technologies. At other times, users struggle to imagine possibilities that do not yet exist. Successful innovation requires navigating both challenges simultaneously. Revolutionary technologies rarely emerge through user requests alone. Yet genuinely useful technologies also cannot emerge through engineering in isolation. Design becomes a process of bridging these perspectives.

    Looking back across the lecture, what emerges most clearly is not a story about auditory interfaces but a broader philosophy of design. Sound happens to be the medium through which Professor Walker explores these questions, though the underlying principles extend much further. Navigation systems, auditory menus, museum exhibits, educational technologies, sonification projects, and accessibility tools all reveal the same challenge. Technologies succeed not when they demonstrate technical sophistication but when they become meaningful parts of human activity.

    Perhaps this is why Walker repeatedly resisted framing accessibility as a specialised concern. The challenges faced by blind users, firefighters, drivers, students, museum visitors, and countless others often reveal broader truths about human interaction with technology. Designing for specific needs frequently produces insights that benefit everyone. When designers stop asking what a system can do and start asking what people need, entirely new possibilities begin to emerge.

    Throughout the lecture, examples ranging from spatial navigation to aquarium exhibits pointed towards the same conclusion. Successful technologies rarely begin with devices, software, algorithms, or interfaces. They begin with people. Understanding how people listen, learn, move, explore, communicate, and make decisions provides the foundation upon which everything else is built.

    For Professor Bruce Walker, the future of auditory interfaces does not lie in adding more sounds to the world. It lies in understanding how sound can help people navigate, learn, communicate, discover, and participate more fully in the experiences around them. The technology matters. The research matters. The engineering matters. Yet each ultimately serves a more fundamental question, one that quietly shaped the entire lecture from beginning to end: who are we designing for?

  • Making Waves: Dr Nina Schaffert on Sonification in Rowing

    Dr Nina Schaffert, a postdoctoral researcher at the University of Hamburg, delivered an engaging online lecture on the role of sonification in high-performance rowing. The session provided valuable insights into how sound can serve as an acoustic feedback mechanism to enhance elite athletes’ performance.

    Sofirow in use

    Biomechanical Feedback in Rowing

    Dr Schaffert outlined the importance of biomechanical diagnostics in elite rowing, where mobile measurement devices capture dynamic and kinematic parameters such as forces applied by athletes, boat speed, and acceleration. These data points are critical in supporting coaches as they refine technique and optimise training regimens.

    Traditionally, this feedback is presented visually, often through graphical displays. However, focusing on a screen while rowing is impractical, especially in changing outdoor conditions. Dr Schaffert noted that rowers naturally rely on acoustic cues, such as water splashes and boat movement, to assess their performance. Building on this, her team explored whether artificially generated sonification could provide real-time auditory feedback to support technique adjustments.

    What is Sonification?

    Sonification converts data into sound, allowing information to be communicated through auditory cues instead of visual representations. This method is particularly useful in situations where visual attention is occupied, enabling real-time feedback without requiring the user to look at a screen. Unlike traditional auditory feedback, which relies on verbal instructions or pre-recorded sounds, sonification generates dynamic audio based on real-time data, making it an interactive form of feedback.

    Different approaches exist within sonification. Parameter mapping sonification, the most commonly used, assigns data values to sound properties such as pitch or volume. Model-based sonification creates sounds based on physical models of movement, mimicking natural acoustic responses. Audification translates raw data directly into sound waves, making patterns perceptible through listening rather than visual analysis.

    Dr Schaffert’s research applies parameter mapping sonification, translating rowing boat acceleration into sound. This makes subtle movement variations audible, allowing athletes to refine their technique.

    Sonification in Rowing: Communicating Movement Through Sound

    In rowing, acceleration varies across the stroke cycle. A rowing stroke consists of two primary phases: drive and recovery. The key transitions—catch, where the oar enters the water, and finish, where it exits—significantly affect acceleration. Sonification maps these variations to sound, enabling athletes to perceive them intuitively.

    Dr Schaffert’s team tested this approach during on-water training with the German national rowing team. The system transformed real-time acceleration data into sound sequences delivered via loudspeakers or earphones. By listening to these sounds, rowers identified inconsistencies in their strokes, particularly during the recovery phase. Adjusting their technique in response to the sound led to smoother movement and increased boat speed.

    Beyond Rowing: Applications in Other Fields

    Sonification has been successfully applied in various domains beyond rowing. In sports training and performance enhancement, it has been used in speed skating, swimming, tennis, and golf. In speed skating, auditory feedback helps maintain optimal rhythm and stride length. In swimming, stroke consistency has been improved by mapping stroke rate and force to auditory signals. In tennis, racket movement has been sonified to enhance swing accuracy. In golf, putting and swing techniques have benefited from auditory cues linked to club speed and angle.

    Beyond sports, sonification supports medical rehabilitation, scientific research, and accessibility. In stroke recovery, auditory feedback aids movement coordination, while rhythmic cues improve gait stability for individuals with Parkinson’s disease. Prosthetic limb users refine control and movement patterns through sonified feedback. In scientific analysis, space telescope data has been converted into sound to reveal celestial phenomena, earthquake data has been sonified to detect tremors, and MRI and EEG data have been made audible for brain activity analysis. Sonification also enhances accessibility, with screen readers and navigation tools providing auditory cues for visually impaired users, while complex graphs and charts are transformed into sound for auditory data interpretation.

    Sofirow: Acoustic Feedback for Rowers

    To apply sonification in training, Dr Schaffert’s team developed Sofirow, a system designed to provide real-time auditory feedback based on biomechanical data. It measures boat acceleration with a micro-electromechanical sensor, converts the data into sound, and transmits it wirelessly to rowers and coaches.

    Sofirow translates acceleration changes into distinct sound variations, allowing rowers to hear their boat’s motion in real time. The system communicates key performance indicators, including boat speed, acceleration, and deceleration. If a rower moves too abruptly during recovery, the sound reflects this instability, prompting a smoother execution. Conversely, an efficient stroke produces a stable, consistent sound.

    A crucial function of Sofirow is improving the recovery phase. The system highlights when a rower disrupts the boat’s glide by moving too forcefully, allowing them to adjust their approach for minimal drag. Timing at the catch is another focal point, ensuring strokes are synchronised to maintain momentum without unnecessary deceleration.

    The system was tested during multiple training sessions with the German junior and senior national rowing teams. Sonification was introduced in alternating intervals, with sections of training both with and without sound. Results demonstrated that when auditory feedback was present, rowers achieved a more consistent technique and increased boat speed. Acceleration data revealed smoother transitions and reduced deceleration at key points in the stroke cycle.

    Athletes found the auditory feedback intuitive and effective in improving coordination. Dr Schaffert presented recordings of Sofirow’s output, demonstrating how variations in movement execution could be heard through pitch and tone changes.

    Future Possibilities for Sonification in Sports

    Dr Schaffert highlighted the expanding role of sonification in sports science, where advancements in machine learning, real-time data processing, and interactive feedback systems are transforming athletic training. One area of development is cycling performance, where real-time auditory cues on pedalling mechanics have been shown to improve efficiency and endurance. By integrating wearable sensors that monitor cadence and power output, sonification enables cyclists to make immediate adjustments to optimise their form, reduce fatigue, and maintain consistent performance over long distances.

    In racket sports such as squash and tennis, researchers have explored how auditory feedback can assess and refine shot precision. Systems that analyse racket-ball impacts can generate sound cues to help players adjust their stroke technique. This feedback allows athletes to develop greater control and consistency in their shots without relying solely on visual analysis. Similarly, in rowing and endurance sports, sonification can reinforce correct pacing by providing rhythmic auditory signals that help athletes synchronise movement with optimal stroke or stride rates, improving efficiency and reducing energy waste.

    The integration of wearable sonification technology is opening new possibilities for personalised training. Smart garments embedded with motion sensors can detect movement patterns and muscle activation, translating this data into sound cues that guide athletes in refining technique. These advancements are particularly relevant in sports requiring precise biomechanics, such as swimming, weightlifting, and gymnastics. With continued progress in real-time data processing, sonification could become a standard training tool, offering immediate and adaptive feedback to help athletes improve performance, prevent injuries, and achieve greater consistency in their movement execution.