“Envelope Follower to MIDI CC” is a Max/MSP patch and stand-alone application for converting audio files into MIDI CC messages in real-time. Just load an audio file, play it, and you’ll be sending MIDI CC data based upon the amplitude envelope of the audio file. You have controls over MIDI data scaling, sampling rate, audio playback speed, and strict playback duration for mapping projects.
The impetus for the tool comes from not being satisfied with available options to convert audio amplitude into MIDI data for Digital Audio Workstation mapping projects. As a teacher, I wanted to create a resource to enable student mapping that leverages their own digital tools.
I made the software available as a repository on github. You don’t need to Max/MSP to run; I made a basic stand-alone application as well and the link is included on github.
If you dig the free software, please listen, share, and follow my music on Spotify.
The creation of five sound sculptures centered around fire-affected areas of the 2020 Holiday Farm fire near Blue River, OR. The work was part of the Soundscapes of Socioecological Succession (SSS) project that was funded through a Center for Environmental Futures, Andrew W. Mellon 2021 Summer Faculty Research Award from the University of Oregon.
Over the summer, I produced five sound sculptures centered around fire-affected areas of the 2020 Holiday Farm fire. The work was part of the Soundscapes of Socioecological Succession (SSS) project that was funded through a Center for Environmental Futures, Andrew W. Mellon 2021 Summer Faculty Research Award from the University of Oregon.
Through field recording fieldwork, local wood sourcing, and custom electronic design, the five (5) sound sculpture prototypes were one way to generate a unique auditory experience aimed at the general public. The work was designed to unpack the sounds and scenes of wildfires in natural and human-systems and to document the regenerative succession of coupled social and ecological processes.
Video 1. Sound sculpture C prototype. Burnt cedar wood and audio sourced from fire affected area near Blue River, OR.
Socioecological systems emerge from interdependent processes through which people and nature self-organize across space and time (Gunderson and Holling, 2002). STEM-centric studies of socioecological dynamics miss literal and metaphorical connections between people and nature, which are difficult to quantify and to communicate. To address this limitation, the sound sculptures test a new approach to capture SSS as a qualitative record of collective response to catastrophic wildfire.
Like a slice of tree ring that marks age and time, the field recordings of audio in visits to fire-affected areas connotes a slice of succession activities. Sound recordings of the area are meant to capture multiple scenes and ecological voices, filtered through a raw material from the sites themselves.
Video 2. Sound sculpture D prototype. Wood and audio sourced from fire affected area near Blue River, OR.
Our sonic environment is polluted by man both in its content and its reflections. This is certainly true even for field recordists who venture further and further into the wild to break free from the noise pollution of a passing airplane, a highway’s din, or even audible underground activity such as fracking (One Square Inch, 2021). Treating site-specific wood as an acoustic resonator — a filter that distorts as much as it renders sound audible — casts a shadow onto the sounds it projects. The physical material acts as a filter upon the sound. The wood slightly changes the spectrum of sound by boosting or cutting the amount of different frequencies in the sound.
Our University of Oregon team expanded previous research by sampling the rich SSS at fire-affected sites, including soundscape field recordings, recorded interviews, and collecting “hazard tree” waste material. These materials offer a document of the resiliency of the landscape and illustrate how forest disturbance can set back human-defined sustainable development goals regionally. The development of the five sound sculptures are just one means to inform the public and inspire collective action towards sustainable futures.
Video 3. Sound sculpture E prototype. Wood and audio sourced from fire-affected area near Blue River, OR.
Audio field recordings were captured during two site visits to fire-affected areas on June 16, 2021 and July 2, 2021. The second visit was to H.J. Andrews forest and an interview and tour with Mark Schulze (H.J. Andrews Experimental Forest Director) Bailey Hilgren and I used a few field recording setups, and which mostly consisted of Bailey recording with a Zoom H6 using on-board mics and I recording with a Sound Devices 633 field mixer and three mics: Sennheiser MKH 30-P48 and MKH 50-P48 microphone in mid-side configuration and a LOM Uši Pro omnidirectional microphone. The Zoom recordings were captured at 96k-24bit, and the 633 recordings were captured at 192k-24bit. During the second visit, we were able to setup “tree ears” that consisted of two Uši Pro mics taped to a tree and a LOM Geofón low frequency microphone, and which we left recording for several hours in the H.J. Andrews forest (see Figure 2). Bailey organized all the audio recordings using the Universal Category System (UCS). The system is a public domain initiative for the classification of sound effects. While we chose not to make the 30+GB of audio files as a publicly available archive, we have made the audio categorization spreadsheet publicly available (SSS metadata spreadsheet).
Figure 1. Field recording setup at fire affected site.
Figure 2. “Tree ear” field recording configuration.
During the technical design phase, some secondary research questions were asked. Which audio exciter/transducers work best on non-flat, raw wood surfaces? Which exciters are the most cost-effective solution for an array of speakers?For fabrication of installing wood as pieces on a wall, can I cost-effectively source sturdier materials than aluminum posts?
Figure 3. Sound sculpture prototypes depicting standoffs and speakers.
I tested a few different models: waterproof transducer, round and square exciters, and distributed mode loudspeakers. I also tested different speaker formats: 10W 8ohm, 20W 4ohm, and 20W 8ohm. Unfortunately, the desired power outputs, 25-30W, models of exciters were consistently sold out throughout the project, therefore I was unable to equally distribute testing across similar power outputs. From experience more than a scientific A/B test, I found that the more flexible options for attaching to wood surfaces were the Dayton Audio DAEX25Q-4 Quad Feet 25mm and the Dayton Audio DAEX32SQ-8 Square Frame 32mm Exciter, 10W 8 Ohm. Generally, I realized that in order to get decent output in both frequency response and gain, the low-end of $15-20/transducer seems about right. I do not recommend anything below 10W for this type of work. Getting a stereo image was not important and would be difficult given the size of wooden pieces. I valued volume and minimizing visual distraction, so speakers were meant to be placed behind or under the sculptures. I doubled speakers whenever I used 10W drivers.
Figure 4. Recording log loader moving hazard tree material
Audio 1. Log loader field recording (see Figure 4)
For standoffs, I sourced variable size stainless steel standoff screws used in mounting glass hardware which worked extremely well on the river wood sound sculpture (Figure 5).
Figure 5. Stainless steel standoffs, 10W 8ohm speakers, and custom electronics board on sound sculpture D prototype.
I sourced audio amplifiers on sale for under $10 each, where $15 is normal pricing. The TPA3116D2 2x50W Class D stereo amplifier boards have handled well on previous projects, and finding them cheaply with the added volume control and power switch were a great addition for fine-tuning amplification in public spaces.
Normally powering the amplifiers and audio boards is where the real cost comes in, and I was happy to learn that Sparkfun’s Redbaord Arduino’s can now handle upwards of 15VDC, so I went with their MP3 Player Shield and Redboard UNO in order to split VDC power between the amplifier and board (12V, 2A power supplies were adequate for the project and transducer wattage).
Figure 6. Custom electronics board consisting of MP3 player shield, Arduino UNO Redboard, 2x50W class D amplifier, and power split for up to 15VDC.
Figure 7. Recording site near Eagle Rock along the McKenzie River.
I modified the outdated MP3 player code on Arduino to dynamically handle any number of tracks and named audio files, such that one doesn’t need to rename audio files in the convention “track001.mp3” “track002.mp3”.Whatever audio files are uploaded onto the SD cards, the filenames simply need to be placed into an array at the top of the code uploaded to the board. Thus, when powered on, the sound sculpture will play an endless loop of the uploaded audio files found on the SD card.
***For those interested in the Arduino code running on the MP3 players, I have made the code publicly accessible as a repository on Github.
Figure 8. Full electronics module example: 12V 2A power supply, MP3 player shield, Sparkfun Redboard Arduino, TDA 2x50W stereo amplifier, single 10W exciter.
Video 4. Sound sculpture A prototype. Wood and audio sourced from fire-affected area near Blue River, OR.
Selecting the audio for sound sculptures came through discussions with Bailey around ecological succession, the interviews conducted, and the types of audio that was captured and categorized. We chose four audio bins (categories) to work with: animals, soundscape or ambient, logging or construction, and scientific or interviews. Again, Bailey created a categorical spreadsheet of audio files within these four bins.
Video 5. Sound sculpture A prototype. Wood and audio sourced from fire-affected area near Blue River, OR.
Constructing the sound sculptures involved imagining public space and the materials. There are two pieces for wall, one for hanging, one for a pedestal, and one for the ground. The sculptures are stand alone pieces that simply require AC power for showing. See below for a gallery of stills of these works.
CONCLUSION
By activating sourced raw materials (e.g., “hazard tree” wood) with acoustic signals stemming from local sites, the sound sculptures amplify the regional and collective voice of wildfire succession even as it outputs a modified version of the input sound.
The process of developing sound sculptures led to additional ideas for iteration or for incorporating the sculptures within a larger-scale project. For example, in our interviews with Ines Moran and Mark Schulze, we found out about “acoustic loggers,” battery operated, weather-proof audio field recorders that record audio based upon a timer. We ordered one such acoustic logger for the project, an Audio Moth; however, the Audio Moth order did not arrive after the completion of the project. Working these into the project through sampling fire-affected sites would create a unique dataset.
The sound sculptures can be stand-alone works. We appreciated the modular approach to the design, and we could continue the modular approach or tether sound objects together. Future work could involve spatializing audio across multiple sculptures similar to previous sound artwork, like Wildfire and Awash.
For the sound sculptures themselves, there is gain control on speaker-level but not on the line output of the players. We could add buttons for increasing/decreasing volume on the MP3 boards to better manage levels, and if we want to provide an interactive component to the works, we could buttons for cycling through tracks on sound sculptures.
Listening to our environment is essential. In 2015, The United Nations Educational, Scientific, and Cultural Organization (UNESCO) formed a “Charter for Sound” to emphasize sound as a critical signifier in environmental health (LeMuet, 2017). By continuing to incorporate sonic practices (bioacoustics, sound art, field recording) into our work with the environment, we create more pathways to experiencing and understanding the planet we live on.
References / Resources
Gunderson, L.H., Holling, C.S., 2002. Panarchy: Understanding Transformations in Human and Natural Systems, Panarchy understanding transformations in human and natural systems. Island Press. https://doi.org/10.1016/j.ecolecon.2004.01.010
The sound sculptures were made possible through a 2021 Center for Environmental Futures Andrew W. Mellon Summer Faculty Research Award in collaboration with Lucas Silva (ENVS).
A shout out to Thomas Rex Beverly from whom I got the idea about recording using the “tree ears” configuration.
The sonic impact of radio on the Americana landscape is profound. Fireside chats, Radio theatre, Payola, DJs, drive-ins, elevator Musak, waiting room noise––the vast consumerism and reach of radio continues to this day. Yet, what happens when we smash two artists (Kenny G and Nickelback) together into one spectral stretched fantasy using the transmission medium that gave life to their careers? Are we doomed to phase out our history with background noise? Or are we undulating with the beat of cultural reclamation and signification? Sending us adrift inside the electrical coils of the radio, Lying in Fireflies Besides Brown Curls and other original compositions attempt to recount a personal connection to memory, lust, and the power of radio to receive a new transmission.
Relay of Memory was exhibited at the Edith Langley Barrett Art Gallery, Utica, NY. The exhibition was supported by funds from the Oregon Arts Commission.
Chromatic Shift is an audio plugin (AU and VST3) that uses three interpolating delays to pitch shift audio.
The range of the plugin is two octaves below and two octaves above the original audio.
A window delay (5ms to 50ms set outside playback) changes audio quality at more extreme settings. Default is 20ms.
Chromatic Shift was built using the JUCE Framework (C++).
Wildfire is a 48-foot long speaker array that plays back a wave of fire sounds across its 48-foot span at speeds of actual wildfires. The sound art installation strives to have viewers embody the devastating spread of wildfires through an auditory experience.
The work was installed at the Edith Langley Barrett Art Gallery in Utica, New York. The work ran Sept. 19 – Dec. 8, 2019 as part of a solo art exhibition entitled, “Impact! works by Jon Bellona.” Wildfire was part of SPRING/BREAK Art Show in NYC March 3 – 9, 2020 curated by Megan C. Austin and Ashlie Flood.
Wildfire was made possible through the University of Oregon Center for Environmental Futures and the Andrew W. Mellon Foundation. The Impact! exhibition was supported by funds from the Oregon Arts Commission. Additional support made possible from the Edith Langley Barrett Art Gallery.
Public Final Report for University of Oregon Center for Environmental Futures.
I mixed-down the forty-eight foot, 16-channel work into stereo (2-channel). An embedded Spotify player is below, but the stereo version may be found on Apple, Amazon, YouTube or wherever you stream music.
The exhibition is organized by the Barrett Art Gallery at Utica College. The exhibition is supported by funds from the Oregon Arts Commission. The Edith Langley Barrett Art Gallery programs are made possible with support from the Utica College School of Arts and Sciences and private contributions.
Media File Reported to Oregon Arts Commission (2019)
Awash depicts the life, color, and environment of the High Desert. The kinetic sound sculpture emanates audio from the region, while flowing as a singular mechanical wave overhead. Awash, like the High Desert, is shaped by many forces interacting in complex ways; the work is ecological – physical movement interacts with sonic vibrations. Sounds interact with the physical environment. Visual elements intermingle with acoustic elements.
The High Desert Museum commissioned Harmonic Laboratory for the work as part of its Desert Reflections 2019 exhibit. The exhibit went on to win the 2019 Charles Redd Center for Western Studies Award for Exhibition Excellence. Press release for the award.
Jon Bellona, electronics, software, field recording, Skinner church organ recording, sound editing, visual design, engineering
John Park, kinetics, visual design, engineering, schematics
Jeremy Schropp, field recording, visual design, engineering
Kevin Davis, Skinner church organ recording
City Synth transforms the city of Eugene into a musical instrument. By transmitting video feeds from multiple locations to a central location downtown that interprets movement and color into sound, the cityscape becomes a soundscape. The project offers the community a playful way to learn about and interact with the gigabit network in downtown Eugene. The project was a month long installation housed in downtown Eugene, OR. Project partners include South Eugene Robotics Team (SERT) and XS Media. The project was funded by a Mozilla Gigabit Community Fund Grant.
Artists involved in the project:
Jon Bellona, sound design, coding, project lead
John Park, coding and visual design
Jeremy Schropp, interface design and construction
Dimension Chorus is a VST plugin modeled after the Roland Dimension D® chorus. The effect includes a shared LFO (with phase flipped on opposite channel). The dimension knob controls a mix of inverted, high-pass filtered signal, which is sent to its opposite channel further spreading the stereo source.
Installation:
For Mac OS, go (Cmd-G) to ~/Library/Audio/Plug-ins to see your “Component” folder (for AU plugins) or “VST” folder (for VST plugins). Move the plugin file to the appropriate folder. You may also go to /Libary/Audio/Plug-ins too. Either place can be used to install plugins. Most DAWs will scan and recognize both locations.
If you dig the free software, please follow my music.