Researchers working on innumerable scientific investigations throughout the world continue to present their findings in new and interesting ways, often apart from the usual charts and graphs. Some have turned to animation, others to interactive graphics and some to the medium of sound — a process called sonification.

Jens Hegg of the University of Idaho has collaborated with several musical composers to turn the migration of young chinook salmon into a musical score — although it doesn’t exactly have a beat you want to dance to.

In the first video on this page, you can close your eyes and imagine that you are standing at the mouth of the Snake River facing upstream, Jens advises in an email. To get the full effect, you need to listen with headphones. Fish moving on the Upper Snake River are represented by notes that sound the farthest away, the Clearwater River somewhat closer and the Lower Snake River closer still.

“The ocean is at your back, so as they enter the ocean, the washy sound sounds as if it is either behind you or directly between your ears,” Jens told me.

“Each river has it’s own set of tones that build a chord,” he continued. “The YouTube video uses the same WAV recording every time, but the actual sonification program randomly assigns fish to a new tone each time it is played, so that the music actually changes and is slightly different each time it is played while maintaining the same meaning.”

I have to confess that I’ve listened to this recording more than a dozen times and I’m still trying to visualize the movement in my mind. The map on the video actually helps with the understanding, but that’s not the movement of fish on the river that I’m trying to visualize. Jens said the program was set up so that fish in the Upper Snake are heard in the right ear; fish in the Lower Snake are heard in the left ear; and fish in the Clearwater are heard in the center.

Some sonification efforts result in music that is quite enjoyable to listen to. See Water Ways, Jan. 1, 2016. But if the point is to convey information, then the underlying music can sometimes be a distraction.

Jens told me that his experiments with sound originated in a roundabout way, somewhat out of desperation, as he tried to design his doctoral dissertation to meet the cross-disciplinary requirements for the program in Water Resources Science and Management.

“I had been told that the geology/ecology combination I had used for my master’s in the same program was not quite interdisciplinary enough, so I was in search of other possibilities,” he said.

Jens had read a newspaper profile about Jonathan Middleton of Eastern Washington University, who was creating music from protein data. He was finding that the people could discern variations in complex protein structure more easily with sonic rather than visual clues.

Jens always enjoyed music. He even writes his own songs. (See second video on this page.) Combining music and scientific data provided a fascinating challenge. “Nobody could say this wouldn’t be interdisciplinary,” he noted, “and it incorporates something I enjoy already, so it seemed like something worth pursuing.”

The data needed to create the sonic composition comes from Jens’ extensive study of salmon migration based on the ear bones of fish, called otoliths. Otoliths are composed of chemicals that build up over time as a fish grows. Rivers have their own chemical signatures, which are captured in the otoliths, so the movement of salmon can be determined by the chemical record stored in their ear bones.

Understanding the timing of salmon migration can help researchers figure out why some populations are more successful than others, especially as climate change shifts the timing of streamflows and alters the temperature and dissolved oxygen levels.

Mounds of otolith data were converted to notes with the help of Middleton at Eastern and Ben Luca Robertson at the University of Virginia. Middleton had developed a software program to help researchers turn their data into sound. Courtney Flatt of Northwest Public Broadcasting separated out some of the individual sounds in a piece she produced for Earthfix. Listen below.

Part of Jens’ research was to see if people could tell when the sound pattern changed, thus discerning the movement of fish from one place to another. The complexity of the sound reduced the ability of listeners to distinguish transitions, according to a new report by Jens and his collaborators in the journal Heliyon. Check out the additional sounds and animations in the “supplementary content” at the end.

It also turned out that people were able to describe the changes more accurately if they were not watching a related animation. One reason could be that the visual clues caused people to divide their attention, focusing less on the sound.

“We have a long way to go before a sonification of a large number of fish can clearly indicate movement,” Jens told me in his email. “Our paper shows that people can accurately distinguish movement of individuals played alone, two at a time, or three at a time. But we haven’t spent as much time optimizing the sonification for a large number of fish.

“Knowing how it is set up helps in interpreting it,” he said. “On repeated listenings, you can hear times where larger numbers of fish are all moving at once from one place to another. These are the kinds of trends we want to highlight in terms of understanding salmon migration timing.”

Share on Facebook