February 24, 2016

# Vega Visualization Grammar

Vega is a visualization grammar. You can read about Vega, its relationship to D3, and the family of tools built on top of Vega in my last post: The D3 - Vega Stack”. This post is an introduction to Vega 2.5.

Vega is a declarative format for creating, saving, and sharing interactive visualization designs. A designer declares the elements of a visualization (using the Vega grammar) in a visualization specification, in JSON format, something like this:

{
"data":[...],
"scales":[...],
"axes":[...],
"marks":[...]
}

Vega does the rest. Vega Runtime can interpret the specification and render it directly in the browser using either SVG or HTML Canvas. (Or, a simple command line application can convert it directly to an SVG file.) An online Vega Editor that shows the spec and the visualization it produces side-by-side makes it easy to write Vega. Check out the examples in the Vega Editor to see some real Vega specs and the visualizations they produce. (Like this bar chart)

Conceptually, the Vega grammar separates the elements of the visualization into these semantic areas:

 DATA The data to visualize DATA TRANSFORMS Grouping, stats, projections, etc. SCALES Mappings of data to visual parameters GUIDES Axes & Legends to visualize Scales MARKS Graphic elements representing actual data

In addition, SIGNALS are dynamic variables that drive interactive behaviours.

The full Vega grammar is described in the wiki. Here is my basic Vega 2.5 grammar cheat sheet:

## Top Level“Visualization Properties” (container properties)

• name (optional) - name for this visualization
• width - width of chart
• height - height of chart
• viewport (optional) - [width, height] of scrollable window onto chart
• padding (optional) - margins
• background (optional) - background color
• scene (optional) - stroke and fill the entire scene

## Other Top Level Properties (chart properties)

• data - data to visualize. See Data

• scales (optional) - Scale transform definitions. See Scales

• axes (optional) - Axis definitions. Axes are the labels (tick marcs, etc.) that show the scales on the visualization

• legends (optional) - Legend definitions. See Legends

• marks - Graphical mark definitions. Marks are the main graphical and text elements of the visualization.

• signals(optional) - Signals are dynamic variables or interactive events

## scales - properties

• name - unique name for the scale
• type - type of scale
• ordinal (ordinal),
• date/time (time or utc), or
• quantitative (linear, log, pow, sqrt, quantile, quantize, or threshold)
• domain - The domain of the scale, representing the set of data values
• domainMin - Min value for scale domain (quantitative scales only)
• domainMax - Max value for scale domain (quantitative scales only)
• range - The range of the scale, representing the set of visual values
• domainMin - Min value for scale range (quantitative scales only)
• domainMax - Max value for scale range (quantitative scales only)
• reverse - flip scale range
• round - round scale range to integers

other properties whose usage varieas according to scale type:

• points - distribute ordinal values uniformly
• padding - apply spacing around ordinal points
• clamp - clamp out-of-range data to the ends of the scale domain
• nice - force scale to use human-friendly values (whole numbers, minutes, hours, etc.)
• exponent - set exponent (for exponential scales only)
• zero - force scale to include zero (quantitative scales only)

## axes - properties

• type - type of axis: x or y
• scale - name of the scale for this axis
• orient - axis orientation: top, bottom, left or right (e.g. right to put a y axis on the right side.)
• title (optional) - title text for the axis
• titleOffset - offset (in pixels) from the axis at which to place the title
• format (optional) - formatting pattern for axis labels (number formats, etc.)
• fomatTyle (optional) - (time, utc, string or number)
• ticks - number of ticks, for axes showing quantitative scales
• values - instead of specifying number of ticks, explicitely set each tick value
• subdivide - number of minor ticks between main ticks (e.g. 9 = decimal subdivision)
• tickPadding - padding between ticks and text labels
• tickSize - size of all ticks
• tickSizeMajor - size of only the major ticks
• tickSizeMinor - size of only the minor ticks
• tickSizeEnd - size of only the end ticks
• offset - offset betwwen axis and edge of the main data rectangle
• layer - draw axes in front (default) or back of the data
• grid - draw grid lines (true or false)
• properties - use for custom axis styling

## legends - properties

Legends link to named scales. At least one of the size, shape, fill or stroke parameters must be specified

• size, shape, fill and/or stroke — scale name determining size, shape, fill or stroke of a data item in the visualization (at least one must be specified)
• orient — position of legend within the scene: right (default) or left
• offset - horizontal offset of legend (in pixels) from the data rectangle
• title (optional) — legend title
• format (optional) - formatting pattern for legend labels (number formats, etc.)
• values (optional) - Explicitly set the visible legend values
• properties - use for custom legend styling

## marks - properties

Marks are the basic visual buiding blocks of a visualization. A mark” is a prototype graphic object duplicated and varied for each data point. (e.g. one single rectangle mark generates an entire bar graph.)

• type - type of mark: rect, symbol, path, arc, area, line, rule, image, text or group.
Also, the special group type can contain other marks, plus local scales, axes and legends. See Group Marks
• name (optional) - unique name for the mark instance (can be used for css styling)
• description (optional) - desciption of mark or comment
• from — data this mark set should visualize
• data — name of the data set to use.
• transform (optional) — array of data transformations to apply
• properties - object containing sets of mark properties
• enter — set of properties to apply when data is processed for the first time and a mark instance is newly added to a scene
• exit (optional) — set of properties to apply when the data linked to a mark instance is removed, and so the mark instance is disappearing. Seldom used.
• update (optional) — set of properties to apply to already existing mark instances, when needed (such as when data changes or after a hover).
• hover (optional) — set of properties evaluated when pointer hovers over a mark instance. At the end of the hover, the update property set is triggered.

Within each set, properties are defined in "name":"value" pairs, where "name" is the property and "value" is either a value reference or a production rule. See Marks for full documentation.

• key (optional) — data field to use as a unique key for data binding for dynamic data
• delay (optional) — transition delay (milliseconds) for mark updates. Used for animation.
• ease (optional) — transition ease function: linear, quad, cubic, sin, exp, circle, and bounce. See here for documentation. (default = cubic-in-out)

## signals - properties

Signals are dynamic variables that drive interactions.

For a description of how Signals work in Vega, see https://github.com/vega/vega/wiki/Signals

February 23, 2016

# The D3 / Vega“stack”

D3, written by Michael Bostock, with Vadim Ogievetsky and Jeffrey Heer at the Stanford Vizualization Group (now the University of Washington Interactive Data Lab is a visualization library now widely used as the basis for many of the most powerful online visualizations. According to Heer, it is intentionally designed as a low-level system.

During the early design of D3, we even referred to it as a visualization kernel” rather than a toolkit” or framework.” In addition to custom design, D3 is intended as a supporting layer for higher-level visualization tools.1 — Jeff Heer

There are numerous higher-level (i.e. easier, more conceptual) third-party tools built on top of D3, including pretty sophisticated high-level GUIs like Plot.ly, simpler drag-and-drop editors like Raw and tools leveraging D3 in computing environments like R and Python.

But the creators of D3 have built their own in-house stack” of increasingly higher-level languages and tools, built on D3, known as the Vega” family of tools.

### low level: Data Visualization“Kernel”

D3 (“Data Driven Documents”) - a JavaScript library for manipulating documents based on data. Users write JavaScript code to produce SVG graphics driven by data.

### low-mid level: Visualization Grammar

Vega by the creators of D3 (Interactive Data Lab). A visualization grammar. User language: JSON. User writes a visualization specification” as a JSON file. The Vega renderer draws draws the visualization from the spec in SVG or HTML Canvas in the browser. Online editor: http://vega.github.io/vega-editor/

(You can run Vega locally with Vegaserver)

### mid level: Visual Analysis Grammar

Vega-lite by the creators of D3. This simplified version of Vega serves as a visual analysis grammar. User writes a visualization specification” with minimal styling options as a JSON file. Vega-lite then generates a full Vega spec. Online editor: https://vega.github.io/vega-editor/?mode=vega-lite

### High level data exploration (GUI)

Voyager built on top of Vega-lite. This web-based GUI automatically builds a set of recommended visualizations from your dataset. Use Voyager to explore your data and possible visualizations.

Pole✭ (Polestar) built on top of Vega-lite. Web-based Drag-and-drop GUI for quickly building visualizations from your dataset. Online editor: http://uwdata.github.io/polestar/#
Compare to Tableau

### Full GUI’s (no coding)

Lyra A full GUI design environment built on Vega. Currently in Beta, and buggy. Try it here

### State of the Stack

Pole✭ (Polestar), Voyager and Lyra are not yet bug free and ready for prime time, but they represent important pieces of the design stack. Voyager is a visualization recommendation engine that helps organize data exploration, saving loads of time in the process. Polestar is a welcome edition to the list of drag-and-drop visualization builders.

Lyra, in particular, represents an important category of high level design tool that just hasn’t existed yet - a full GUI design environment for data-driven graphics. It still has a long way to go to become the Illustrator of data graphics, but it’s heading in that direction. Once Lyra (or something like it) reaches maturity, then we will see another surge in the prevelance of high quality data visualizations that parallels the surge that the introduction of D3 brought us.

The basis for these advances is Vega, the declarative visualization grammar. Compared to coding in D3, Vega can be learned by non-coders quite easily. No background in JavaScript, or any programming language, is needed. At the same time, Vega is rich enough to be expressive, and not just easy to use.

Vega is well documented on the wiki. Mostly for those like me who want a birds-eye view of Vega, I’ve assembled a Vega 2.2 cheat sheet, which you can read in my next post.

1. Jeff Heer : https://github.com/vega/vega/wiki/Vega-and-D3, edited May 1, 2014, accessed Aug 26, 2015

November 9, 2015

# European Premier of Party Pieces

Back in 2013, in honor of the 100th birthday of John Cage, the Forum of Contemporary Music Leipzig [FZML] asked me and 124 other composers to collectively write an exquisite corpse composition based on an idea of John Cage’s. Each of us wrote one page of manuscript, passing on only the last bar to the next composer. The sequence of composers was chosen by a random process involving tossing coins and consulting the I Ching, one of Cage’s favorite tools.

125 Party Pieces was premiered in New York by Ensemble Either/Or in October 2013 for the finale of the international Cage 100 Festival. The 125 manuscripts were exhibited at Galerie für Zeitgenössische Kunst, Leipzig, Germany in August-Spetember 2013.

Party Pieces will get its European premier in Leipzig on January 20, 2016 (tickets).

Thanks to Pierre Boulez for his patronage of this project and the German Federal Cultural Foundation for the funding.

September 28, 2015

# Perceptually Uniform Pitch-Loudness Scales for Data Sonification

Musical scales are based on perceptual pitch intervals corresponding to frequency ratios, not differences. (e.g., a ratio of 2.0 gives the pitch interval called the octave.“) The deciBell scale (dB) is a log scale for sound intensity (or pressure), where a ratio of 2.0 = 6dB.

One of the challenges in producing a good visualization is the construction of appropriate visual scales that do not bias the interpretation of the data. When constructing color scales, for example, it’s important to base them on how we perceive the colors, not simply on raw RGB intensity levels or raw hue values. (See my previous post, as well as Gregor Aisch’s article on the subject.)

An analogous problem in sonification is the construction of perceptually linear frequency and intensity scales. Directly mapping data to frequency and intensity (sound pressure level) produces misleading results. To begin with, frequency perception is logarithmic and best represented as pitch (frequency on a log scale).1 But not only is intensity perception logarithmic (hence the log deciBell scale), it is also highly frequency dependent. Consider this map of perceived loudness as a function of frequency and intensity (data from the current ISO standard, derived from multiple empirical studies):

Phon levels are perceptually equal loudness levels across frequencies. Phon differences represent perceptually equal loudness differences. At 1000 Hz, the phon and dB scales are identical. For a 100Hz tone and a 1000Hz tone to sound at equal loudness, the 100 Hz tone must have a much higher SPL, since the human ear is much less sensitive to low frequencies.

Figure 1. ISO226:2003 Equal loudness contours. Produced with Matlab and Plotly.

Take a hearing test like the ones used to collect data for those phon curves.

Just as in visual perception, perceived brightness is dependent on frequency (hue) and not just light intensity, in auditory perception, perceived loudness is also highly dependent on frequency (and not just sound intensity). That means that if we simply construct a linear pitch scale at a constant intensity (or sound pressure, or gain” in digital audio terms), then some pitches will sound much louder than others, making them sound more important” and biasing the interpretation of the data. So, we need to construct a perceptually uniform pitch-loudness space” analogous to a perceptually uniform color space” (such as CIELAB).

This 2D pitch-loudness space based on ISO226:2003 shows equal perceived differences in pitch on the X axis (as log Frequency), and equal perceived differences in loudness (phons) on the Y axis. The colored contours show the SPL required to produce a desired loudness (phon) level at that frequency.

Figure 2. SPL contours for given frequency and loudness level, derived from ISO226:2003. Produced with Matlab.

Let’s hear that. First, here’s a sine tone sweeping from low to high across the audible frequency range, at a single Sound Pressure Level (dB). This sound would be a horizontal line on the first graph, and a contour line on the second graph. Notice how the sound is hard to hear at first (in the low range) and at the end (high range).

Listen with decent headphones in a quiet room for best results. Turn the volume up to comfortably loud level.

31.5 Hz to 12,500 Hz, equal SPL:

Next, the same sweep, but with the SPL constantly following the 80 phon equal loudness contour.” That’s a contour line on the first graph, and a horizontal line on the second graph.

31.5 Hz to 12,500 Hz at equal loudness
(using 80 phon equal loudness contour):

Notice the low and high sounds are easier to hear. If you listen on good reference” speakers in a recording studio environment, the sine tone would sound equally loud across the whole range.

Using that second graph of pitch-loudness space, you can now imagine constructing more complex scales, such as a diagonal line that goes from low+loud to high+soft, or divergent scales analogous to these divergent color scales (code by Gregor Aisch). (We might want to take into account the Sone scale of loudness to construct certain scales in 2D pitch-loudness space, but that’s a topic for another day…)

Note: All of this is only valid for sounds that are pure tones” (i.e. sine tones). If we want to construct objective, linear scales for sonification using complex tones we will need a higher-dimensional timbre-space, and one that is perceptually uniform—something like this (Hiroko et al. Perceptual distance in timbre space”) or this (Hoffman et al. Feature-Based Synthesis for Sonification and Psychoacoustic Research”). But that’s a topic for yet another day.

## Software Tools

The ISO226 standard (2003 revision) represents the current most widely accepted representation of perceptually uniform pitch-loudness space (for sine tones). This is a commercial standard, and you need to purchase it in order to read the complete document. Here are a couple of tools that implement it:

1. There is a good Matlab function called iso226(), written by Christopher Hummersone that outputs SPL in dB for given frequency and phon level(s). The function implements the mathematics in the ISO226:2003 and uses Matlab’s shape preserving pchip method for interpolation.

2. For my own purposes, I wrote an abstraction for Cycling74′s Max environment called can.phon2dB that does the same thing, using a pre-calculated lookup table. The lookup table is in in cycling74′s jitter matrix” format and is well suited for real-time applications. (I’m using it to build a sonification of microbial populations—more on that in a later post…)

can.phon2dB.zip — The zip file contains 3 files:

• can.phon2dB.maxpat - the abstraction
• can.phon2dB.maxhelp - the help patch
• iso226-2003.jxf.jit - the matrix file (64MB)

The .jxf.jit file must be in your search path. This lookup table has a resolution of 1 Hz and 0.25 phon, which is about the same as or less than the just-noticeable difference” for pitch and loudness, respectively.

## The Mathematics: Formula for deriving SPL from Phons

Read on if you’re interested in the math involved in constructing scales from the equal loudness curves, or on practical tools for sonification. The formula is taken from ISO226.

The sound pressure level $L_{p}$ of a sine tone at frequency $f$ at perceived loudness level $L_{N}$ is:

$L_{p}=(${10}A{f} ) dB $-L_{U}+94$ dB

where

$A_{f}=4.4710^{-3}( 10^{0.025L_{N}}-1.14 )+[ 0.410^{( -9 )} ]^{_{f}}$

and

• $T_{f}$ is the threshold of hearing for frequency $f$, in dB;
• $_{f}$ is the exponent for loudness perception at frequency $f$;
• $L_{U}$ is a magnitude of the linear transfer function normalized at 1000 Hz;

The values of $T_{f}$, $_{f}$ and $L_{U}$ are given in the following table:

Frequency $f$ Loudness perception exponent $_{f}$ Transfer function magnitude $L_{U}$ Threshold of hearing $T_{f}$
20 Hz 0.532 -31.6 dB 78.5 dB
25 Hz 0.506 -27.2 dB 68.7 dB
31.5 Hz 0.480 -23.0 dB 59.5 dB
40 Hz 0.455 -19.1 dB 51.1 dB
50 Hz 0.432 -15.9 dB 44.0 dB
63 Hz 0.409 -13.0 dB 37.5 dB
80 Hz 0.387 -10.3 dB 31.5 dB
100 Hz 0.367 -8.1 dB 26.5 dB
125 Hz 0.349 -6.2 dB 22.1 dB
160 Hz 0.330 -4.5 dB 17.9 dB
200 Hz 0.315 -3.1 dB 14.4 dB
250 Hz 0.301 -2.0 dB 11.4 dB
315 Hz 0.288 -1.1 dB 8.6 dB
400 Hz 0.276 -0.4 dB 6.2 dB
500 Hz 0.267 0.0 dB 4.4 dB
630 Hz 0.259 0.3 dB 3.0 dB
800 Hz 0.253 0.5 dB 2.2 dB
1000 Hz 0.250 0.0 dB 2.4 dB
1250 Hz 0.246 -2.7 dB 3.5 dB
1600 Hz 0.244 -4.1 dB 1.7 dB
2000 Hz 0.243 -1.0 dB -1.3 dB
2500 Hz 0.243 1.7dB -4.2 dB
3150 Hz 0.243 2.5 dB -6.0 dB
4000 Hz 0.242 1.2 dB -5.4 dB
5000 Hz 0.242 -2.1 dB -1.5 dB
6300 Hz 0.245 -7.1 dB 6.0 dB
8000 Hz 0.254 -11.2 dB 12.6 dB
10,000 Hz 0.271 -10.7 dB 13.9 dB
12,500 Hz 0.301 -3.1 dB 12.3 dB

To produce smooth curves, we must interpolate between the 29 data points. The graphs and software tools in this post all interpolate using Matlab’s custom built-in pchip method, which is a shape preserving PCHIP (Piecemeal Cubic Hermite Interpolating Polynomial). It is similar to using bezier curve interpolation, but with a tighter fitting curve that does not overshoot” actual data points, so that the curves shape is preserved.

The ISO226 specification also provides the formula for deriving Phons from SPL, which uses the same tables for $T_{f}$, $_{f}$ and $L_{U}$.

1. There is another, empirically derived scale for frequency perception called the Mel scale, which in some contexts can be more appropriate, but the ordinary logarithmic frequency scale is probably best in most cases. The Mel scale seems to capture the way we perceive frequency best in the absence of any musical” context. But if we listen to a linear scales constructed from small equal steps, (where we can easily compare each interval with the adjacent intervals) then scales based on the simple mathematical definition of pitch (equal frequency ratios = equal perceptual difference) produce more even results than ones based on the Mel scale, especially for anyone with musical experience. Listen to an example of the Mel scale here if you’d like to judge for yourself: http://www.sfu.ca/sonic-studio/handbook/Mel.html

August 24, 2015

# Global Ocean Surface Waves, Visualized

Here is an exquisite vizualization of ocean surface waves by Cameron Beccario @cambecc. The animation below is just one visualization possible with his tool, Earth. Visit earth.nullschool.net to play with wind, current and temperature data. Scroll down to read more about Earth.

The color scale shows the peak wave period (time between crests) for ocean swells. Longest period waves (up to 25 seconds) are brighter cyan, and chopy, short period waves approach black. The animation on top is a representation of the surface wave vector field,” and not an animation of actual surface waves.

Wave conditions (updated every 3 hours). Scroll down to see more.

### Earth: an Excellent Example of Good Design

Earth is an excellent example of how good design can produce visualizations that engage a public audience and do a better job of representing data. A good data visualization begins by formulating the task as a Design Problem. The design problem that Earth sets out to solve might be worded like this:

Represent near real-time global weather forecast in an accurate yet holistic way that captures geographical context.”

Earth is on view now through Fall 2015 at Point.B Studio in Port Orford, Oregon, as part of the exhibition, Gegenshein.

You might add and is visually successful as fine art,” since one of Baccario’s intended uses of Earth is to generate fine art prints and other materials for exhibition. But a really good design can produce museum quality work in any case, even if it has a functional purpose.

Beccario decided that what was required was not a single visualization but a tool to generate visualizations of a choice of datasets from a variety of perspectives. The design of the resulting tool, Earth, flows from that requirement. It uses a visually efficient, limited, and unified design language and intuitive interface to allow users to display a very large range of possible visualizations.

Relief map of the moon, color shaded, with rainbow scale. (Image credit: NASAs Goddard Space Flight Center/DLR/ASU.) See Drew Skau’s visual.ly blog post: Dear NASA: No More Rainbow Color Scales, Please”

One small component of that design language is the carefully constructed set of color scales. Color scales are often an afterthought in scientific visualizations, often relying on default rainbow” scales. These naïve color scales can cause intelligibility problems and bias the interpretation of the data. For example, the colors in the standard rainbow scale are not of equal perceived brightness; therefore some bands of information can be perceived as more important than they are, and we can perceive boundaries between color bands that are not necessarily there in the data.1 Beccario designed more perceptually relevant color scales with the help of the online tool ColorBrewer. (There are many other similar tools out there. See also New York Times’ designer Gregor Aischs excellent article on color scales.) Beccario’s color scale for wave periods is a perceptually even brightness gradient of a single hue (cyan), so any boundaries we see in the map are real boundaries, an not artifacts of the color scale.

Beccario's color scale for wave period is a perceptually linear brightness gradient of a single hue.

I encourage everyone, including scientists, to explore Cameron Beccario’s tool Earth at their leisure, to discover the difference good design can make in data visualization.

Watch Beccario talk about his design process for Earth at the Graphical Web 2014 conference:

1. Drew Skau, Dear NASA: No More Rainbow Color Scales, Please.” blog post, Visually. http://blog.visual.ly/rainbow-color-scales/