5 takeaways:

➀ What is data sonification? Jordan Wirfs-Brock, a former data journalist and Ph.D. candidate at the University of Colorado, defines data sonification as simply “any technique for communicating data with sound. These are charts that you can use your ears to listen to,” she said. “Sonification sits within this broader field called auditory display, which is just any way of communicating information with sounds.” While that may feel esoteric or intimidating to some journalists, she points out that most people are familiar with “sonic logos” and “auditory icons” (such as the crumbling sound your computer trash can makes when you put something in Recycling) even if they don’t know those terms.

➁ Why use data sonification? In addition to the “gee whiz” factor, data sonification paired with visuals can delight audiences and reinforce learning, Wirfs-Brock said. “It can be super sticky,” she said. “When you have more than one sensory modality together, it actually helps people remember information better.” Because sound is inextricably tied to time, journalists “can invite people to linger,” Wirfs-Brock said. “It opens up this whole avenue into slower interactions with data.” It also makes data accessible to people who are blind or have impaired vision.

➂ Which newsrooms are using data sonification? The New York Times created a visualization-sonification combination to convey the difference a bump stock made in the Las Vegas shooting. “This is a really effective way to use tempo to illustrate a really crucial piece of information,” Wirfs-Brock said. “You experience it in a more visceral way when you’re able to hear the sound … they could have used some gunshot sound but they didn’t. They used this really simple tapping sound because I think it’s emotional enough and visceral enough just to hear it the way that it is.” Michael Corey of Reveal used tempo, volume and pitch to convey the change in the frequency and strength of earthquakes in Oklahoma before and after the oil and gas boom. “What’s really nice about this one is it establishes a before and after,” Wirfs-Brock said. “Sonification can be really effective for communicating state changes,” old normal vs. new normal.

➃ How numbers and sounds align. Tempo, volume and pitch are examples of “sonic parameters,” ways of mapping sound elements to data values, Wirfs-Brock said. Other ways of conveying different information include using minor or major key, complexity and layering, looping and repetition and timbre. “If you think about the difference between a flute playing a set of notes and a violin playing a set of notes, they can play the exact same notes, same pitch, same loudness, but they’re going to sound very different,” Wirfs-Brock said. “You could have multiple dimensions of your data that are mapped onto different instruments.” For example, USA Today used lower and higher-pitched instruments to convey driest years compared to heavy rainfall years in an investigation into how climate change has changed precipitation patterns. Wirfs-Brock used instruments, particularly drums, to communicate the crescendo of unemployment claims for Marketplace.

➄ How to train listeners. “We’ve been looking at bar charts, maybe since elementary school,” Wirfs-Brock said, and some people still don’t understand them, but “sonification literacy is way, way, way behind.” So she and other journalists work to help audiences listen and understand. “It’s not like visualization, where there’s all kinds of best practices and established rules. … it’s a much newer field and so there’s more ground to cover. And this is fun and exciting and liberating because you can actually be very creative, but it’s also really challenging.” One of the better-established sonic parameters is mapping quantity onto the pitch – if the number gets higher, the pitch gets higher.

Other ways of guiding listeners along include:

• Follow people’s natural auditory intuition – i.e. big bass drum equals big number.
• Incorporate spoken word as a way of labeling or marking sonifications – for instance, someone saying “2018” to mark the year when the sonification passes that point in the data.
• On mobile and web presentations, pairing visualizations or text labels with the sonification.

Speaker: 

Jordan Wirfs-Brock, Ph.D. Candidate, Information Science, University of Colorado Boulder

This program was funded by the Evelyn Y. Davis Foundation. NPF is solely responsible for the content.