The Problem With Big Data (It's Not Me, It's You)

Big data. There, I said it.

For the past little while, I have watched – in astonishment – the amount of times a senior marketing professional (both from the agency and brand side) have spouted off about the opportunity of big data in this brave new world. As they continue to speak, it becomes abundantly clearer that they probably don’t even know what big data is. By my estimation, the vast majority of people extolling the virtues of big data seem to think that it’s just like the data we have known to date… but more of it. That’s not big data… that’s more data. That’s just a lot more of the same data. As these marketing professionals continue on their verbal admiration of everything that big data will bring to the industry, you quickly begin to realize that it’s nothing more than jargon to these people.

The problem with big data.

The problem with big data is that most marketers suck (big time) at the small data… or the data they’re currently using. How many different flows of data do they have (email lists, web analytics, search engine marketing campaigns, traditional ad campaign results, research and more)? Do those individual streams of data flow into a larger ocean? If you dig beneath the surface, my guess is that most organizations have what my friend, Bryan Pearson (CEO of LoyaltyOne and author of The Loyalty Leap) calls "data ghettos." These clumps of data that sit in silos and rarely (if ever) speaks to one another. Is big data something important (and big)? Yes. Are businesses truly ready, willing and able to act on it? Unlikely (at this moment in time). Currently, data is hardly being amplified using real time technology, and even fewer brands are actually taking the time to create tests and iterations from these traditional data sets. Let alone doing that form of marketing optimization on a daily basis. Sadly.

What big data can look like.

Take this definition from Smart Data Collective: "A phenomenon defined by the rapid acceleration in the expanding volume of high velocity, complex and diverse types of data. Big Data is often defined along three dimensions- volume, velocity and variety." What is this telling us?

1. Big data is data that can be manipulated (slices and diced) with massive speed.
2. Big data is the not the standard fare that we use, but the more complex and intricate data sets.
3. Big data is the unification and integration of diverse data sets (kill the data ghettos).
4. Big data is based on much larger amount of data sets than what we’re used to and how they can be resolved with both speed and variety.
5. Big data extrapolates the information in a different (three dimensional) way.

What does this mean?

The net result is that big data delivers new and previously unavailable opportunities to drive business efficacy. That’s it. Plan and simple… and hard. A tonnage more of your traditional data mixed in with what big data really is and what you have is a world where marketers are drowning in their data. Bring in the analysts, because the only people who are going to turn big data from a shiny, bright object of jargon into a functional place that drives business solution will be these data analysts coupled with marketing strategists. Together, these people may be able to unlock the Da Vinci Code of big data to churn out actionable insights (as my good friend, Avinash Kaushik, calls them). Everything else is simple hype and hyperbole.

Watch this.

I recently appeared on The Agenda With Steve Paikin. The full thirty-minute conversation will air Friday, July 26th, 2013 on TVO. As a bonus, Steve asked me about the business of big data, and this is what I said: