April 26, 2016

Intro to Sound and Data Sonification

Definitions

Acoustics: (1) The branch of physics concerned with sound. (2) The properties of a concert hall with respect to the way sound interacts with it.

Psychoacoustics: The branch of psychophysics that studies the sense of hearing. Psychoacoustics defines, qualifies and quantifies sensations in relation to the stimuli (sounds) that cause them.

Electroacoustics: The intersection of acoustics and electronics. Electroacoustics studies the conversion of sound into an electronic signal (called transduction), the manipulation of the electronic signal, and the conversion of the signal back into sound (also transduction).

Sound: a mechanical vibration transmitted through a medium (usually air) to the ear, with an amplitude and frequency capable of being perceived by the auditory system.


IF A TREE FALLS IN THE WOODS with no one around, it does make a sound.

IF A TREE FALLS ON THE MOON, even with someone around, it does not make a sound. (Sound does not travel in a vacuum.)

Bats produce ultra-sound (sound too high for humans to hear). Elephants produce infra-sound (sound too low for humans to hear).


Signal: any other vibration or energy variation that does not fit the definition of sound, even if the vibration or variation represents a sound. We commonly refer to electric and digital signals.

Analog Signal: A smoothly varying signal. In other words, a direct analog” for sound. An electric signal is an analog signal. The grooves on an LP are also a type of analog signal. A cassette tape stares an analog signal magnetically.

Digital Signal: A signal that varies in discreet steps. A digital signal can be created is created by sampling an analog signal at regular interval, called the sampling rate. A digital signal is like a rasterized image: It is a series of numbers, or each number (or sample”) representing the intensity of a signal at a given moment in time. (Whereas a raster image is a series of numbers each representing the color of an image at a given point on the screen or page.) A digital signal can be stored in a variety of ways, including magnetically (on a digital audio tape, or DAT), and optically (on a CD). A digital signal can be transmitted electrically (in a computer chip, or on a specially built electric cable) or optically (fiber optics).


Analog-to-Digital Conversion (ADC): The process of sampling an analog signal in order to create a digital signal.

SOUNDELECTRIC (ANALOG) SIGNALDIGITAL SIGNAL

Digital-to-Analog Conversion (DAC): The process of converting a series of samples to a continuously varying (analog) electric signal.

DIGITAL SIGNALELECTRIC (ANALOG) SIGNALSOUND

(NOTE: technically, in the diagrams above, the transition from electric to sound is transduction.”

Sample: (1) An individual number in a digital signal. The sample represents the intensity of a signal at a given time. The sample is to a digital signal as a pixel is to a digital image. (2) The length of time it takes for one sample to go by (depends on the sampling rate, but is usually a small fraction of a millisecond). (3) An entire bit of digitally recorded sound, stored as a series of numbers (A.K.A. a stored digital signal). This is also the popular usage of the term.

Sampling Rate (in Samples per Second): (1) The rate at which an analog signal is sampled in order to create a digital signal. The most common sampling rate is 44,100 samples per second. This is the rate used by CD players. Other common sampling rates are 22,050 samples per second and 48,000 samples per second. Sampling rate is analogous to the resolution of a raster image.


How Acoustic Parameters of Sound Map to Perception (and possible data types)

physical parameter perceptual parameter possible data mapping (Q=Quantatiative, O=Ordinal, N=Nominal)
Frequency (Hz) Pitch, or height” QON
Intensity Loudness (Q)ON
Waveform (spectrum) Tone Color (Q)(O)N
Intensity+Frequency+Waveform in Time Timbre (Q)(O)N

Notice, the term volume” is not used. Loudness” and Intensity are more precise. Volume” is used in psychoacoustics to refer to an esoteric characteristic of sound, which could be described as its fullness.” We will avoid the term volume” for now.


More Definitions

Noise:

  1. any undesirable, uncomfortable or dangerous sound. Sound pollution refers to this meaning. This is the popular meaning.
  2. The opposite of signal. Parasitic vibrations accompanying a signal that interfere with its clear transmission. The Signal-Noise ratio” refers to this meaning.
  3. an a-periodic sound (a sound without a definable frequency, hence without a definite pitch). This is the opposite of Musical Sound.”

Musical sound or pitched sound: a periodic sound (a sound with a definable frequency, hence with a definite pitch).

Unpitched sound: Noise (definition 3.)


Auditory Perception

As graphic perception must be taken into account when designing scales for visualization, auditory perception must be taken into account when designing scales for sonification of data.

One notable example of how auditory perception should influence scale design is in the construction of pitch scales.

For a discussion, see: http://blog.ericmarty.com/7/perceptually-uniform-pitch-loudness-scales-for-data-sonification


Sonification Examples

General Interest

http://www.huffingtonpost.com/mark-ballora/sound-the-music-universe_b_2745188.html

Simple Mapping of Single Variables

Nick Bearman: temperature to pitch (map mousover)

altitude to pitch (map mouseover)

integers mapped to integer frequency bins
Sorting algorithms (computer science) - scanned to frequency (integers mapped to integer frequencies)

Redundant Mapping of Single Vairables

price to speed+pitch+loudness
https://www.foreignaffairs.com/audios/2015-07-22/sound-economy

Multiple 1:1 Mappings

Listen to wikipedia: Hatnote

Arctic Ice

Sonification of LHC data

Complex Mappings of Single Variables

Brian Foo http://brianfoo.com

smog levels to granular synthesis parameters http://www.theatlantic.com/technology/archive/2012/09/soundscapes-of-smog-researchers-let-you-hear-the-pollution-of-cities-literally/262152/


Previous post
Visualization Tools Built on D3 The in-house family of higher-level tools built on top of Mike Bostock’s D3. Mike Bostock developped D3 at the Stanford Visualization Group, led by

© Copyright 2015 Éric Marty