Head in the Clouds
...or why I am not a fan of word clouds
One of the questions that occasionally we get asked is how to present data from unstructured free text.
And sometimes then we get asked, "Do you do a word cloud?"
To be honest we get asked this less and less and I think as people get more used to data visualisation we could see this end completely.
It is well known, to those who know me well, that I am not and never have been a fan of word clouds, but why? and also why do people want to use them?
The Mullets of the Internet
I do love this, I am biased I know, but I think it is a fab description from 2005 of the faddish adoption of word clouds (tag clouds) back in the day.
What are they?
Words are taken from unstructured (or structured) text and shown in a form of a grouping of words, typically larger or smaller based on the number of times the word is used.
Whats the issue?
If we think about the purpose of data visualisation. Data visualisation takes complex data and turns this into visuals that allow the user to see the information they need quickly, to spot trends, to see data represented in different ways and to tell a story about the data.
Telling a story is an important part of data visualisation and I am afraid word clouds simply do not do this, they strip out the sentences, the narrative and try to replace this rich (unstructured) data with "tags" based on how often a single word is said.
I will demonstrate with a bad example; Macbeth, a classic:
Methought I heard a voice cry, ‘Sleep no more! Macbeth does murder sleep: the innocent sleep, Sleep that knits up the ravelled sleeve of care, The death of each day's life, sore labor's bath, Balm of hurt minds, great nature's second course, Chief nourisher in life's feast.
The book is complex, with lots going on and many different viewpoints and narratives, understanding this requires that you read the book or at least a detailed synopsis of the major themes.
Or could you just read the word cloud...
Are we surprised that the main word used in Macbeth, a story about Macbeth is "Macbeth"?
They provide no context to the sentiment in the text
Words are just displayed with no background, no context of how the person felt at the time.
We will often use the same words when we are asked a question that are in the question, if you ask someone about Teamwork people may say "we are great at teamwork" or "teamwork could be improved" or "teamwork is ok"
So in the word cloud we will see "Teamwork" dominating but all the other words will be based on the individual - great, awesome, fab, good, ok, done well and amazing all suggest good things, you just won't get that in a word cloud.
Their design means you see something that potentially is not there
Some words are bigger than other words because they contain more letters - sounds obvious right?
If a short word has a large count and a long word a medium-sized count you run the risk that you lose the scale between the words simply because the word is bigger!
Look at how "CadburysDairyMilk" is so much larger than "Milka" or "Trident" below - are you confident that this is because it is statistically larger or just because it has more letters.
This is not data science, you cannot see the facts
Here is a word cloud representing towns in the UK by population size, what does this tell us?
We can see that Liverpool and Sheffield are the largest here, but we instantly lose any real frames of reference.
How much larger is the population of Liverpool than Sheffield?
What about Hull? is it the same as Derby?
Quote from Jacob Harris from the NY Times
So, why do people use them?
I think it is because of the following:
Unstructured text takes a long time to read so people want a fast solution
Word clouds were everywhere once and therefore well known and associated with a fast solution
Most people do not think about data visualisation issues
We all have biases that things like word clouds play on
Let's look at those biases and how we respond to word clouds:
I do not have enough meaning
We have already said that word clouds are sparse when it comes to actual meaning so what happens - "We tend to find stories and patterns even when looking at sparse data"
Some of the biases at play:
Clustering Illusion - The clustering illusion is the tendency to erroneously consider the inevitable "streaks" or "clusters" arising in small samples from random distributions to be non-random
Illusory correlation - In psychology, illusory correlation is the phenomenon of perceiving a relationship between variables even when no such relationship exists
I have too much information
free text by it's nature is a lot of information so what happens? - "Visually striking or bizarre things stick out more than non-bizarre, less striking things"
Word clouds are designed to be visually striking
Some of the biases at play:
Von Restorff effect - The Von Restorff effect, also known as the "isolation effect", predicts that when multiple homogeneous stimuli are presented, the stimulus that differs from the rest is more likely to be remembered
I need to act fast
We have already said that in business we default to needing to act quickly, reading lots of free text is time-consuming.
This is one of the reasons word clouds are chosen because - "We favour simple-looking options over complex ones"
So, what is better?
Hopefully, by now you are in agreement that we do not need a word cloud in our life if we are looking for insights into any large unstructured free text,
So, if not word clouds then what?
Stay tuned - our series on data visualisation is going to cover this. so watch out for the next blog and keep an eye on our videos for more information!
The People Experience Hub
A tech-led, human approach to measuring and improving people experience at work, helping you understand your people better.
If you want to find out more about how we can help you or want to chat about Employee Engagement, Employee Experience, People Analytics or Coffee then drop us a note on email@example.com or fill in the contact form on the website.