Tag Archives: Sonification

Amusing Monday: Movement of music captures climate discord

Using music to describe measurable changes in climate — and expressing the anxiety caused by the ongoing changes — is one approach to the climate problem that has been engaging scientists and musicians alike.

I’ve been following several methods of converting data to sound, which approximates music in some ways (Water Ways, Jan 16, 2017). But the Climate Music Project in San Francisco starts with a nearly complete musical composition and allows the data to alter the sound in remarkable ways.

Composer Erik Ian Walker had been writing and recording music for 30 years when he joined the Climate Music Project in 2015, collaborating with scientists and technicians to explore musical approaches to climate change.

“I welcomed the invitation to write and perform ‘Climate’ for CMP because I feel very strongly about the necessity to communicate the urgency of stopping the negative effects of human-caused climate change,” Erik said in an interview on CMP’s website. “Being a composer, this was the best use of my talents to do something. I also like the intersection of science and music very much, so it was a good fit….

“Decisions that had to be made were whether the climate data was going to be the music (sonification), or whether the data was going to alter music composed before the data collided with it,” he continued. “We chose the latter, as that was the more interesting scenario for a dramatic rendering…

“The hardest part was composing a ‘theme’ and framework that would not devolve too fast as the data we were using began to change the music,” he said. “There is a subjective response of the ear, outside of prescribed numbers, that gauges where ‘double’ of something is, for example. So, we had to find an ‘end point’ of the piece, where the greatest degree of climate change would be, hear what that would sound like, and work backward from there.”

The result is shown in the first video on this page, which shows the piece accompanied with dynamic charts and graphs. In fact, if you happen to be in San Francisco on Sept. 19, you can see and hear a CMP performance of “Climate” at the Exploratorium in the Embarcadero waterfront district.

The piece is about 30 minutes long and offers two scenarios: one in which humans continue on the current path of pumping massive amounts of greenhouse gases into the atmosphere and another in which major changes are made to keep the rise to less than 3.6 degrees F. — the goal of the Paris Climate Accord.

Reporter John Metcalfe describes in CityLab how the melodic movement begins to shift as the calendar reaches the start of the industrial revolution.

“Weird distortions like twinges in a stretched-out cassette tape arrive in the late 1900s as Earth’s energy balance is jolted out of whack,” he writes. “Looking into the future, the music then turns darker and frenetic in the decades post-2017 — the beat and pitch racing, the melody discordant and churning, and the planet’s temperature soaring into an irreversible heat hell.”

Besides the first video, enjoy the following samples of music from two different time periods offered by CMP on Vimeo:

Stephan Crawford, who started the Climate Music Project, explains how he came up with the concept of creating music that can help people experience climate change in an emotional way in an article by Alessandra Potenza in The Verge magazine. The second video on this page provides an idea of how the collaboration works for those involved with the project.

The difference between Erik Ian Walker’s “Climate” and sonifications of data — which certainly have their place — is that you can become immersed in the music, enjoying even the dark parts for their emotional impact. To sample and purchase Erik’s “normal” music go to Bottom Feeder Records’ webpage.

The third video is a promo of the Climate Music Project from two years ago.

Amusing Monday: Using musical notes to describe salmon migration

Researchers working on innumerable scientific investigations throughout the world continue to present their findings in new and interesting ways, often apart from the usual charts and graphs. Some have turned to animation, others to interactive graphics and some to the medium of sound — a process called sonification.

Jens Hegg of the University of Idaho has collaborated with several musical composers to turn the migration of young chinook salmon into a musical score — although it doesn’t exactly have a beat you want to dance to.

In the first video on this page, you can close your eyes and imagine that you are standing at the mouth of the Snake River facing upstream, Jens advises in an email. To get the full effect, you need to listen with headphones. Fish moving on the Upper Snake River are represented by notes that sound the farthest away, the Clearwater River somewhat closer and the Lower Snake River closer still.

“The ocean is at your back, so as they enter the ocean, the washy sound sounds as if it is either behind you or directly between your ears,” Jens told me.

“Each river has it’s own set of tones that build a chord,” he continued. “The YouTube video uses the same WAV recording every time, but the actual sonification program randomly assigns fish to a new tone each time it is played, so that the music actually changes and is slightly different each time it is played while maintaining the same meaning.”

I have to confess that I’ve listened to this recording more than a dozen times and I’m still trying to visualize the movement in my mind. The map on the video actually helps with the understanding, but that’s not the movement of fish on the river that I’m trying to visualize. Jens said the program was set up so that fish in the Upper Snake are heard in the right ear; fish in the Lower Snake are heard in the left ear; and fish in the Clearwater are heard in the center.

Some sonification efforts result in music that is quite enjoyable to listen to. See Water Ways, Jan. 1, 2016. But if the point is to convey information, then the underlying music can sometimes be a distraction.

Jens told me that his experiments with sound originated in a roundabout way, somewhat out of desperation, as he tried to design his doctoral dissertation to meet the cross-disciplinary requirements for the program in Water Resources Science and Management.

“I had been told that the geology/ecology combination I had used for my master’s in the same program was not quite interdisciplinary enough, so I was in search of other possibilities,” he said.

Jens had read a newspaper profile about Jonathan Middleton of Eastern Washington University, who was creating music from protein data. He was finding that the people could discern variations in complex protein structure more easily with sonic rather than visual clues.

Jens always enjoyed music. He even writes his own songs. (See second video on this page.) Combining music and scientific data provided a fascinating challenge. “Nobody could say this wouldn’t be interdisciplinary,” he noted, “and it incorporates something I enjoy already, so it seemed like something worth pursuing.”

The data needed to create the sonic composition comes from Jens’ extensive study of salmon migration based on the ear bones of fish, called otoliths. Otoliths are composed of chemicals that build up over time as a fish grows. Rivers have their own chemical signatures, which are captured in the otoliths, so the movement of salmon can be determined by the chemical record stored in their ear bones.

Understanding the timing of salmon migration can help researchers figure out why some populations are more successful than others, especially as climate change shifts the timing of streamflows and alters the temperature and dissolved oxygen levels.

Mounds of otolith data were converted to notes with the help of Middleton at Eastern and Ben Luca Robertson at the University of Virginia. Middleton had developed a software program to help researchers turn their data into sound. Courtney Flatt of Northwest Public Broadcasting separated out some of the individual sounds in a piece she produced for Earthfix. Listen below.

      1. Tracking Salmon With Musical Notes

Part of Jens’ research was to see if people could tell when the sound pattern changed, thus discerning the movement of fish from one place to another. The complexity of the sound reduced the ability of listeners to distinguish transitions, according to a new report by Jens and his collaborators in the journal Heliyon. Check out the additional sounds and animations in the “supplementary content” at the end.

It also turned out that people were able to describe the changes more accurately if they were not watching a related animation. One reason could be that the visual clues caused people to divide their attention, focusing less on the sound.

“We have a long way to go before a sonification of a large number of fish can clearly indicate movement,” Jens told me in his email. “Our paper shows that people can accurately distinguish movement of individuals played alone, two at a time, or three at a time. But we haven’t spent as much time optimizing the sonification for a large number of fish.

“Knowing how it is set up helps in interpreting it,” he said. “On repeated listenings, you can hear times where larger numbers of fish are all moving at once from one place to another. These are the kinds of trends we want to highlight in terms of understanding salmon migration timing.”

Amusing Monday: Science is music when data becomes sound

Nearly everyone who deals in scientific information learns to read simple charts and graphs to help visualize the data. As a reporter, I’m often looking for the right graph to bring greater meaning to a story. In a similar way, some people have been experimenting with rendering data into sound, and some of the more musically inclined folks have been creating songs with notes and musical scales.

As with graphs, one must understand the conceptual framework before the meaning becomes clear. On the other hand, anyone can simply enjoy the music — or at least be amused that the notes themselves are somehow transformed from observations of the real world.

The first video on this page, titled “Bloom,” contains a “song” derived from microorganisms found in the English Channel. The melody depicts the relative abundance of eight different types of organisms found in the water as conditions change over time. Peter Larsen, a biologist at the U.S. Department of Energy’s Argonne National Laboratory in Illinois, explains how he created the composition to Steve Curwood, host of the radio program “Living on Earth.”

      1. Living on Earth

Continue reading