Composing data visualisations

How does one create new data visualisations? Apart from the art, is there a science to it?

Let’s explore a few popular charts. We have the vertical bar graph small-vertical-bar or the horizontal bar graph small-horizontal-bar. The stacked bar small-stacked-bar. The variwide or Marimekko chart small-variwide. The waterfall small-waterfall. The scatterplot small-scatterplot. The treemap small-treemap. And so on.

The first thing you’ll observe is that all of these are a series of rectangles. (We’re treating the dots on the scatterplot as little squares.) The only thing that varies across these charts is the position and size of the rectangles – and the colour as well.

That gives us a hint. Perhaps there are many ways of creating visualisations just by changing the position, size and colour of rectangles. For example the horizontal bar graph small-horizontal-bar can be constructed as follows:

  • The x position is constant for each rectangle. It starts at zero.
  • The width is proportional to the value of the series
  • The y position is proportional to the index of the values (1,2,3,…)
  • The height is constant for each of the bars
  • The colour is constant too.

Whereas, if we look at a horizontal stacked bar small-horizontal-stack, then:

  • The x position is proportional to the cumulative value of the series.
  • The width is proportional to the value of the series
  • The y position is constant at zero
  • The height is constant for each of the bars
  • The colour is based on the index of the values (distinct colours labelled 1,2,3,…)

Generalising this, we can construct a table like this that shows the structure of various visualisations:

Chart x width y height colour
Vertical bar chart index constant constant value constant
Stacked bar index constant cumulative value index
Waterfall index constant cumulative value constant
Scatterplot value constant value constant index
Horizontal bar chart constant value index constant constant
Variwide cumulative value constant value constant

That leads to a line of thought: what if we tweaked this table? Would we get new visualisations that might be interesting?

Let’s experiment with a few.

waterfall-variwideWhat if we took the waterfall chart, and made the constant widths proportional to value, instead? The waterfall chart shows a cumulative series of values (e.g. percentages). This new chart – a cascade chart – allows us to depict each bar’s relative importance as well as value.

boxesWhat if we kept the width, height and y constant, and just let the x values vary as the index? It would just be a row of boxes. But we’d have the option of colouring them with a value. This could be useful when showing performance along a discrete series (e.g. attendance by weekday).

boxesWhat if we allowed the x, y, width, height and colour to vary with a different value? The graph looks like a scatterplot, but every dimension here – position, size,  colour, even aspect ratio – indicates some informational measure.

This chart can, for example, show the position and spread of two metrics. For example, if the X-axis were sales, and the Y-axis were price, each bar could be the distribution of price and sales in a branch, with the colour indicating growth of the branch.

Just using the combinations discussed above, there are 75 possible types of visualisations – many of which are meaningful in different circumstances. And this is just using rectangles.

What we’ve done here is mapped data to attributes of a visualisation. This is part of a generalised approach to graphics, similar to that covered by Leland Wilkinson’s Grammar of Graphics and implemented in libraries like ggplot2 or D3. Once we establish that basic concept – that a chart is a mapping of attributes to data – the variety of charts you’ll be able to create is unlimited, and you move from being a user of charts to a composer of data-driven visualisations.

Tracking computer usage

CIHL (a consortium of universities in Andhra Pradesh) offers a masters course in information technology. As part of that, the computer usage for volunteering student was tracked for 7 weeks. The raw data shows how long each application was used.

We visualised the total usage of the top applications by student.

msit-computer-usage

Before we go on to the results, a few words about the visualisation.

  • Each row is one application. They are sorted by usage.
  • Each column is one student. The width of the column is proportional to their usage. They are sorted by the amount of time spent on computers.
  • Each cell shows %time spent on the application. For example, 20% means that student spent 20% of her time on that application.

This is similar to the heatgrid we saw last month, but with a difference – the widths of the columns are not constant, and represent the hours of usage. This means that the colour represents not just the % usage by a student – it has an additional significance. The amount of purple ink used in each row is the total hours of usage of the application.

Now for what we found.

Browsers are clearly the most popular application, with people spending 25-50% of their time on the browser. Firefox is the most popular browser, followed by Chrome. Only 3 students used IE as their main browser.

Microsoft Word emerged as the second most popular application. This is what students submitted their assignments in.

VLC was the next most popular, ignoring the time spent on Windows Explorer. While their coursework did require them to view a number of videos, an analysis of the window titles showed that the percentage of course-related videos were in a minority. This also provided us with a number of interesting movie recommendations that has kept us busy last month.

Two games made their way into the top applications list: Half Life and Warcraft III. While only 4 students were serious gamers, the time they spent on this was significant. The student spending maximum time on the PC spent almost 20% of time on games, with another 30% on movies. (We were yet to investigate whether this had a positive or negative effect on grades.)

Chat applications did not show significant usage. IPMsg was the most popular, with up to 0.5% of time being spent on this. Google talk was used by fewer people, but those that used it spent up to 3% of time on it.

But the strangest observation was regarding two students, both of who spent about 10% of their time looking at screen savers. One of them was, in fact, a blank screen saver. We have still not been able to figure out what exactly they were up to.

Charting one-dimensional data linearly

How many ways are there of looking at series of data? Consider this rainfall data:

chart-table

We have rainfall for every district in Tamil Nadu for every month over the last 5 years. That’s 60 data points per district. How many ways are there of plotting it?

In this post, we’ll look at 10 ways you can represent a simple series – in a straight line.

Data Bars

chart-bars

These are a quick way of plotting bar graphs within the cells. The eye is naturally drawn to numbers with large values. It’s an easy way of locating big numbers, and in particular, to compare data across series. But it isn’t very easy to find trends within a series.

Colour scales

chart-gradient

These shade each cell with a colour gradient. Red for low, green for high. While they’re much worse at exact comparisons, they’re much better at helping identify trends – both within a series and across.

Heatmap

chart-heatmapThe colour scales can be shrunk without much loss of information if we’re more interested in the trend than in the numbers.

This heatmap is a compact way of comparing information over time, and across districts. Reading left-to-right, the patterns of growth, decline or seasonality can be observed. Reading top-to-bottom, patches of high or low that cut across data series become evident.

This is a simplified one-dimensional version of the traditional heatmap which typically shows data in two dimensions.

Bar chart

chart-barchart

If being able to compare quantities within a series becomes important, one can use bar charts instead.

The bar chart shown here is a variant of the traditional bar chart. It does away with the horizontal and vertical axes, as well as the labels, and just shows the bars.

This is an example of a micro-chart, the most classic example of which is the sparkline. Microsoft has introduced a number of these micro-charts in Excel 2010. This is one of the significant upgrades in Excel 2010’s charting capabilities.

Sparkline

chart-sparkline

Sparklines are among the earliest microcharts, initially created by Edward Tufte. They are the equivalent of line graphs, but without the labels and axes.

These make it very easy to compare trends within a series. However, comparing across series may not be easy. In fact, it would not be possible at all unless the sparklines are drawn to scale.

Sparklines are, however, particularly effective in representing growth or decline trends in a compact fashion.

Trendline

chart-sparkline-trend

A trendline overlays sparklines with a trend. This may be a moving average, a best-fit line (e.g. linear regression), etc.

The high variability of sparklines can be smoothened out through the trendlines, making it slightly easier to spot long-term trends.

This is particularly useful when the data shows multi-seasonal patters (e.g. a weekly as well as a monthly pattern), and we want to bring out both effects in the same chart.

Streamgraph

chart-streamgraph

A streamgraph is identical to a sparkline, except that instead of the height representing the value, it is the width of the graph that represents the value.

These are also referred to as stacked graphs. They are particularly effective when visualising multiple series one on top of another. See Lee Byron’s Last.fm listening history for an example of effective use of this graph.

These are most effective in identifying which series is dominant at a given point in time, and how the series grows or dies around that point.

Horizon graph

chart-horizon

The horizon graph expands the resolution of sparklines. First, it uses an absolute scale, differentiating between positives and negatives. Negatives are coloured red, and positives are coloured green. These are then folded.

The chart is then folded repeatedly, and uses colour intensity in conjunction with height to show the value. Panopticon, who created Horizon Graphs, have a good introduction to the use and construction of these graphs.

Like heatmaps, these are useful in spotting horizontal and vertical trends, but using an absolute rather than a relative scale.

Jitter plot

chart-jitter

Jitter plots are useful ways of visualising the density and frequency of a data series. They plot the values horizontally, rather than vertically. That is, the x-axis is the value rather than the y-axis. The y-axis just spreads the points around randomly to minimise the overlap.

This is useful in comparing frequency data. For example, here, it is clear that no rainfall is the most frequent state. It can also been seen that Cuddalore typically has many months with little rainfall.

When the data density becomes too high, however, jitter plots are not as effective.

Box plot

chart-boxplot

In such cases, box-plots make for a better display. Invented by John Tukey in 1977, these summarise a data series using just five numbers: the minimum, the lower quartile, the median, the upper quartile and the maximum.

The box represents the area where 50% of the observations lie. The horizontal line represents the full range of values in the series. The vertical line is the median. Half the values lie to the left, and half to the right.

While this plot appears simplistic, it often is much more robust (i.e. safe to use for a wide variety of datasets).