When you hear the term “big data”, what comes to mind?
“statistics...results...unbiased...evidence...facts...answers...proof”
For many of us, when we hear “data”, we think “facts”. And not just any facts – statistics, numbers, quantitative results…aka “trustworthy” facts.
It is a version of this thinking that pervades many organizations where a quantification bias, “valuing the measurable over the immeasurable” (Wang, 2016), drives us to invest heavily in big data systems and to respond to disappointing results by gathering more data.
Meanwhile, this behaviour continues to fail us. Big data lets us down.
Why doesn’t big data stand up to its reputation?
In large part, because we forget that big data does not come from a big bang, or some other force beyond the scope of human influence. We are big data.
We are its contents, its creators and its curators:
Data consists of the thoughts, attitudes, beliefs, behaviours, and other activities of humans that are constantly changing
The environments we live in, from which data are extracted, are continuously evolving
Because we cannot extract ourselves from the data:
The data we choose to collect is influenced by (conscious and unconscious) human motivations
What we collect determines what can be discovered
How we collect data shapes what that data is and can be
We like patterns, and the more data we collect, the more we prioritize data that fits the growing set
Big data is not free of human bias, but its use is often missing human intelligence
Big data uses machine learning to define large, varied sets of data; detect patterns, and report results as insights. These reported “insights” are based on models reflecting patterns and relationships in the data; and on organizational processes and priorities identified by stakeholders.
It is this approach to deriving insights that can give us a reassuring sense of objectivity, and completeness – while actually missing a critical aspect of understanding the data reported: context.
Context is the WHY – why are we seeing these results? how did these relationships come to be? Why are people making these choices? Behaving in these ways?
Insight is context-dependent. The why is essential to decision-making.
How can we make better use of big data?
#1: Start with the big questions
Ask the big, scary questions. Get clear about what you really want to find out, and why. Ask why, and then ask why again (and again).
#2: Give numbers a backdrop
Treat big and thick data as accomplices. There is never a time when qualitative inputs won’t add depth, meaning, and improved insights to quantitative results. Put in the time now. The resources you think you’re saving by “just sending a survey” and “pulling the numbers” is an illusion – you’ll pour much more time and effort into finding answers when big data fails to deliver a silver bullet.
#3: Quit needing to be right
We are all prone to confirmation bias. Beware of solidifying, and even intensifying toxic assumptions within the organization when big data is used to confirm the status quo and other hypotheses without reflection.
#4: Work that courage muscle
The promise of big data is not a lie, we just aren’t using it to its potential. Instead of seeking to confirm, use big data to challenge and explore thinking, results, and assumptions that reflect the organizational status quo. Be a scientist! Make a point of trying to prove your hypothesis wrong.
#5: Discover questions you haven’t asked
Using big data is not just about finding answers to the big questions. Very often, what we discover leads to new questions, which are sometimes even more useful than the originals. Get into the habit of asking “why is that?” when you find something interesting in a report. Learn to look at data differently by putting data into graphics that show patterns. Data visualization doesn’t have to be complicated – it can be as simple as an Excel graph that compares two data sources.
#6: Human-centricity > data-centricity
Many organizations make decisions and optimize performance for metrics. Sure, we use metrics to report on overall business success, and as one measure of decision-making efficacy. But people are at the crux of this “optimization” and neglecting the actual experiences involved in these processes strips us of the moral reflections that rationalize our actions. People first, then numbers.
#7: Cultivate data enthusiasts
Data science is not reserved for the professional elite. By empowering everyone to think about and dig into big data, we gain better, more real-time access to the human context big data so desperately needs. Having data enthusiasts can enable a different kind of collaboration and generate collective insight.