Will Big Data And Analytics Deliver The Promised Land?

This post got published before I intended to publish it. Sorry for this oversight. I have now completed it as intended and am republishing it. I apologise for any inconvenience and thank you for your understanding.

What do B2B technology vendors sell?

No, it is not the technology.  Think again, what do B2B technology vendors sell?  They sell dreams that speak to a fundamental human need. What dreams? Dreams of control-mastery-domination over the ever flowing, every morphing, character of a process we turn into a noun: life.

What need do these dreams take root from and speak to?  The need for safety and security. At some fundamental level we get that nature is indifferent to our survival and wellbeing.  To deal with this anxiety we embrace anything that provides the illusion of safety-security. The Greeks embraced the Gods, we embrace technology and the latest technofix.

I notice that the big data and analytics space is hot right now.  It is the latest technofix being pushed by the B2B technology vendors.  It occurs to me that this technofix is designed to speak to those running large enterprises – especially those who are higher up and divorced from the lived experience of daily operational life at the coal face.

What I find astonishing is that so few actually ask the following two questions:

1. “What kind of a being is a human being?”

2. “What kind of a culture is human culture?”

What is the defining characteristic of human beings?

Allow me to illustrate by share a story I read many years ago:

Psychologist:  John, you have been referred to me by the authorities. They tell me that you think that are dead. Is that right? Are you dead?

John: Absolutely, I died a little while back.  I am dead. 

Psychologist: How interesting! You died a little back. Yet here you are talking with me. And I am not dead.  So how is it that you are dead and I am not dead, yet here we are talking? 

John: Beats me how this works or why it is happening. I know that I am dead. 

Psychologist: John, I have an idea. Do dead people bleed? 

John: Don’t be ridiculous! Everyone knows that dead people don’t bleed! 

The psychologist suddenly reaches over and cuts John’s hand with a knife. Both of them are looking at John’s hand. Blood, dark red blood, is seeping through the cut.  The psychologist looks at John with the look of satisfaction, of victory. Let’s rejoin the conversation.

Psychologist: John, do you see that blood on your hand? How do you make sense of it? You say that you are dead. And earlier you told me that dead people don’t bleed.

John: F**k me, dead people do bleed!

This is not simply an amusing story.  It is a story that captures the experience of a respected psychologist who has been dealing with many kinds of people, dealing with many kinds of problems, over a lifetime.  This story capture a fundamental truth of the human condition.

It appears that to survive in the world as it is and as we have made it, we need to be deluded. We need to distort reality: to make life more predictable, to make our current situation lighter-better than it is, to see a future brighter than is merited by the facts, to see ourselves stronger, more capable, more influential than we are. Studies suggest that those of us who lack this ability to distort reality and delude ourselves end up depressing ourselves.

What Kind Of A Culture Is Human Culture?

Symbolic and ideological.  Why?  Because human beings just don’t cope well with the world as it is. So we get together into tribes. And the glue that keeps the tribe together is a particular way of constructing the world, a particular way of giving meaning to the world, and a particular way of interacting with the world.  And when I speak world I include human being, and human beings; a human being is always a being-in-the-world as in always and forever an intrinsic thread in that which we call world.

The next question: which ideology do members of society espouse?  The dominant public ideology. In the world of business this is that of scientific management and in particular reasoning and making decisions objectively – irrespective of the past, of tradition, of our personal interests and opinions.

A more interesting question is that about the actual behaviour of the elites, the Tops. What is it that the Tops actually do?  They do that which protects and furthers their interests: their power, their status, their privileges, their wealth, their dominance.  So insight and recommendations (whether from big data and analytics or through conventional methods) that are in line with these interests are heartily accepted and actioned swiftly and vigorously.

Any insights and recommendations that challenge the vested interests of the elite (Tops) are repressed at the individual level, belittled-disputed-ignored at the societal level.  I invite you to read this article which can be summed up as the UK Government sacks the chair of the official Advisory Council on the Misuse of Drugs. Why? Because the chair was insisting on the reclassification of drugs. What happened?

  1. The Advisory Council looked at the data (of harm to the individual taking the drugs and others affected by his/her behaviour) on drugs at the request of the UK.

  2. On the basis of the data, the Advisory Council came up with the conclusion that “if drugs were classified on the basis of the harm they do, alcohol would be class A, alongside heroin and crack cocaine.”

  3. The drug rankings, associated findings and recommendations were ignored by the UK government. Why? Because they went against the government’s stance on drugs.

  4. The chair of the Advisory Council challenged the UK government’s refusal to act on the recommendations of the Advisory Council.  So the appropriate UK Government minister sacked him.

What Does The Future Hold for Big Data & Analytics?

If past behaviour is an adequate guide to the future then it is safe to say that technology vendors will get rich. And the business folks will have another layer of technology that they have to manage. One or two organisations may reap substantial benefits, the rest will be disappointed.  Yet, this disappointment will not last long. Why? By that time the technology folks will have come up with the latest technofix!

I leave you with the following thoughts:

1. There are no technofixes to the kinds of social issues-problems we continue to face;

2. Incremental improvements lie in the domain of big data and analytics;

3. Breakthroughs lie in our ability to see that which is with new eyes – a shift in dominant concepts, dominant paradigm, dominant ideology, dominant way of seeing that which is.

Put differently, big data & analytics is a red herring for those who aspire to lead: to cause-create that which does not exist today.  Managers, those whose horizon extends to daily operations and the next twelve months, may find big data and analytics useful – as long as it does not threaten the sacred cows of the Tops-Middles and the corporate culture.

Musings on Big Data, Customer Analytics, and Data Driven Business

On LinkedIn, Don Peppers is sharing his perspective on making better decisions with data.  This got me thinking and I want to share with you what showed up for me. Why listen to my speaking?  I do have a scientific background (BSc Applied Physics).  I qualified as a chartered accountant and was involved in producing all kinds of reports for managers and saw what they did or did not do with them. More recently, I was the head of a data mining and predictive analytics practice. Let’s start.

Data and data driven decision-making tools are not enough

Yes, there is a data deluge, and this deluge is becoming down faster and faster. Big enough and fast enough to be given the catchy name Big Data.  What is forgotten is the effort that it takes to get this data fit for the purpose of modelling.  This is no easy-cheap task. Yet, it can be done if you throw enough resources at it.

Yes, there are all kinds of tools for finding patterns in this data. And in the hands of the right people (statistically trained-minded, business savvy) these tools can be used to turn data into valuable (actionable) insight.  This is not as easy as it sounds. Why?  Because there is  shortage of these statistically trained and minded people: amateurs will not do, experts are necessary to distinguish between gold and fools gold – given enough data you can find just about any pattern.  It statistical savvy is not enough you have to couple it with business savvy. Nonetheless, let’s assume that we can overcome this constraint.

The real challenge in generating data driven decision-making in businesses is the cultural practices.  We do not have the cultural practices that create the space for data driven decision-making to show up and flourish.  A thinker much smarter-wiser than me has already shared his wisdom, I invite you to listen:

On the whole, scientific methods are at least as important as any other research: for it is upon the insight into the method that the scientific spirit depends: and if these methods are lost, then all the results of science could not prevent a renewed triumph of superstition and nonsense.

Clever people may learn as much as they wish of the results of science – still one will always notice in their conversation, and especially in their hypotheses, that they lack the scientific spirit; they do not have the distinctive mistrust of the aberrations of thought which through long training are deeply rooted in the soul of every scientific person.  They are content to find any hypothesis at all concerning some matter; then they are all fire and for it and think that is enough …….. If something is unexplained, the grow hot over the first notion that comes into their heads and looks like an explanation ….

– Nietzsche (Human, All Too Human)

It occurs to me that the scientific method never took route in organisational life. Put aside the rationalist ideology and take a good look at what goes in business including how decisions are made. I say you will find that Nietzsche penetrating insight into the human condition as true today as when he spoke it. The practice of making decisions in every organisation that I have ever come in contact with is not scientific: it does not follow the scientific method. On the contrary, managers make decisions that are in alignment with their intuition, their prejudices, and their self-interest.  It is so rare to come across a manager (and organisation) that makes decisions using the scientific method that when this does occur I am stopped in my tracks. It is the same kind of unexpectedness as seeing a female streaker running across the football pitch in a league match.

What are the challenges in putting data driven decision-making practices into place in organisations?

Technologists have a gift. What gift? The gift of not understanding, deeply enough, the being of human beings. Lacking this understanding they can and do (confidently) stand up and preach the virtues-benefits of technology.  If life were that simple.

Truth shows up as attractive to those of us who do not have to face the consequences of truth.  Data driven decision-making sounds great for those of us selling (making a living and hoping to get rich) data driven tools and services.

The challenge of putting in place data driven decision-making practices is that it disturbs the status quo. When you disturb the status quo you go up against the powerful who benefit from that status quo.  Remember Socrates:

The very nature of what Socrates did made him a disruptive and subversive influence. He was teaching people to question everything, and he was exposing the ignorance of individuals in power and authority. He became much loved but also much hated …. In the end the authorities arrested him for …., and not believing in the gods of the city. He was tried and condemned to die …

– Bryan Magee, Professor

Beware of being successful in putting in place a culture of data driven decision making!

With sufficient commitment and investment you can put in place a data driven decision making culture. Like the folks at Tesco did.  And by making decisions through harnessing the data on your customers, your stores, your products, you can outdo all of your competitors, grow like crazy and make bumper profits.  Again, again, and again.  Then the day of reckoning comes – when you come face to face with the flaws of making decisions solely on the basis of data.

Tesco is not doing so great.  It has not been doing so great for several years – including issuing its first ever profits alert in 2012.  What is the latest situation?  Tesco has reported a 23.5% drop in profits in the first half of this year.   What has Tesco been doing to deal with the situation? This is what the article says:

Last year, Tesco announced it would be spending £1bn on improving its stores in the UK, investing in shop upgrades, product ranges, more staff, as well as its online offering.

There are a number of flaws on data driven decision making. For one data driven decision making assumes that the future will be a continuation of the past.  Which is rather like saying all the swans that we have come across are white, so we should plan for white swans.  And then, one day you find that the black swan shows up!  The recession and the shift in consumer behaviour that resulted from this recession was the black swan for Tesco.

Furthermore, I hazard a guess that in their adoration at the pulpit of data driven decision making the folks at Tesco forgot the dimensions that matter but were not fed into the data and the predictive models. What dimensions? Like the customer’s experience of shopping at Tesco stores: not enough staff, unhappy staff, stores looking more and more dated by the day, the quality of their products ……

It looks like the folks at Tesco did not heed the sage words of one of my idols:

Not everything that counts can be counted, and not everything that can be counted counts.

– Einstein

IBM’s CEO 2012 study: is technology really the number1 priority of CEOs?

In the second half of 2012, IBM issued its fifth biennial Global CEO Study titled Leading Through Connection.  IBM says this study is based on face-to-face conversations with more than 1,700 CEOs in 64 countries.  I have been reading it and want to share with you what I make of it.

Social – concerned, fearful, skeptical, uncertain?

It appears that CEOs are increasingly accustomed to volatility and expect unpredictability.   The same cannot be said for social. 

My reading of the study suggests that CEOs are concerned and fearful about social, in particular, its role in enabling customers to exercise power over organisations.  Second, most of them are skeptical of the value of social media – I suspect in driving revenues and profits. Even those who do see value in getting into social, are uncertain as to how to go about it.  Given that so few of them have dived into social media this does not surprise me.  How does one get comfortable with riding a bike?  By riding a bike – you get on the bike, you ride, you fall off, you pick yourself up, you get back on the bike and soon you are riding the bike.  Is social any different?

Are CEO’s customer obsessed and what is their take on customer analytics?

According to the study, CEO’s told IBM that three leadership traits, in particular, are the most critical for navigating through this disruptive era:  ‘customer obsession’, ‘inspirational leadership’ and ‘leadership teaming across the C-suite’.  I’ll come back to the latter two, let’s take a look at ‘customer obsession’.

When the world is unpredictable and the power continues to shift to customers it makes sense that ‘customer obsession’ is seen as the critical leadership trait.  Yet I wonder what this means?   Does this mean that CEOs will get out of their offices and get deeply involved with customers?  Spending time with them face to face like Lou Gerstner did when he took charge of IBM.  Or undertaking the kind of experience showcased in Undercover Boss.

According to the study, CEOs are investing in analytical capabilities that promise to yield customer insights.  More than 70% of CEOs say that they are looking to get a better grasp of customer needs and generate improved organisational responsiveness to customer insights. So, I suspect that ‘customer obsession’ entails spending money on technology in the hope that this will yield insights into customers that enable the enterprise to stay one step ahead of customers.  This is great news for the organisations selling analytics technologies (SAS, IBM….).  Whilst I see the value of analytics I remain doubtful of the impact this will make until and unless the CEO gets out and actually walks in the shoes of the customer and experiences the experience of the front line workers.

What does IBM advise?  IBM recommends that CEOs orient their organisations to get insight into and engage customers as individuals rather than aggregates (segments, markets).  The earlier IBM CMO study suggested that marketers are stuck on traditional research methods that look at and provide insight into these aggregates and not individual customers.  Where can organisations get insight at the level of the individual customer?  Through harvesting and mining ‘Big Data’ according to IBM.

What are the roadblocks on this ‘data based, insight driven nirvana’?

In its study IBM refers to come organisations as ‘outperformers’ (they are doing better financially than their peers) and ‘underperformers’ (doing financially worse than their peers).  What are the three key differences between these ‘outperfomers’ and ‘underperformers’ according to IBM?

  • ‘translate insights into action’ – 84% better than industry peers;
  • ‘excel at managing change’ – 73% better than peers;
  • moving into adjacent industries – 48% better.

So here we see the two major obstacles in the way of ‘data enabled, insight driven nirvana of business performance’.   Ability to act on insight.  And, in particular, act in such a way as to effect organisational change with efficacy: to make changes in the business (based on the insight) and to get the people in the organisation to go along with and internalise these changes.  And to do so quickly before the window of opportunity closes.  Clearly some organisations are better at this than others – those that excel at this are in the minority.  Interestingly, this issue of taking action/effecting change based on insight has been surfaced by VoC vendors like Mindshare.

Are CEO’s aware of the scale of the challenge?

It appears that CEO’s are aware of the scale of the challenge that is facing them.  How do I come to this conclusion? Leadership traits that are of importance to CEOs and what traits CEO’s are looking for in employees.  Let’s consider the leadership traits first and what they can tell us.

When it comes to critical leadership traits, ‘customer obsession’, ‘inspirational leadership’ and ‘leadership teaming across C-suite’ were almost equal in importance.  Statistically speaking, I suspect there is no significant difference between the three of them.   It kind of suggests, to me, that these are inter-related.  Let’s take a look at the latter two and what they can tell us.

‘Leadership teaming across C-suite’ suggests that CEO’s get that the default state of organisation – the silo structure and silo metrics – is that of optimisation of the parts and suboptimisation of the whole.  It also indicates that CEOs are aware that genuine collaboration and teamwork starts at the very top – if the C-suite does not work well together then it is highly unlikely that the lower ranks will work well together.  I also read into this an implicit acknowledgement that many C-suites do not work well as one team – personal interests and functional agendas compete against the well being of the whole.  This explains why ‘inspirational leadership’ is seen as a critical leadership trait: CEOs get that they have to inspire (to bring forth) the best of their people (starting with the C-suite) including working as one team for the collective benefit.  Is it possible I am concocting a story that appeals to me and is not in the study?  I leave you to decide for yourself.

IBM claims that CEOs are creating more open and collaborative cultures.  That does not strike me as being an accurate description of what is so based on my travels.  And I get that I have only experienced a small number or organisations.  What is more interesting is the claim that collaboration is the primary trait that CEOs are looking for in employees: 75% of CEOs label this trait as critical.  Why?  According to IBM, CEO’s see technology as an enabler of collaboration and relationships and are focussed on changes in how people engage with the organisation and with one another in order to fuel responsiveness, creativity and innovation.

And finally

It is interesting to note that CEO’s put ‘human capital’ as the most important source of economic value closely followed by ‘customer relationships’, with ‘product/services innovation’ being in third place.  If ‘human capital’ is that important then the value placed on collaboration and open-cultures is understandable.  If ‘customer relationships’ and ‘product/services innovation’ are this important then the investments in analytics and collaboration technologies make sense.  What does not make sense, to me, is the attitude around social: customer can be a great source of innovation.

Only 33% of CEOs consider business model innovation as key source of economic value.  This puzzles me given that one of the key issues that organisations have to grapple with is that of coming up with fresh business models that make the most of the opportunities and deal effectively with the disruptions caused by ‘social’ technologies and customer behaviour.   Perhaps many CEOs have not fully awakened to the scale of the challenge/disruption/opportunity facing them.  What do you think?

I doubt that technology is the no1 priority of CEOs.  Why?  Because ‘technology’ got 71% of the votes, closely followed by ‘people skills’ at 69% and ‘market factor’ at 68%.  It occurs to me that there is nothing in it – that all three of these factors are as important as each other.

Want to get value out of your data and analytics investment? Then deal with this issue before you buy the software.

‘Rationals’, data and the wonders of analytics

I think I have said this before and I will say it again: my fascination is with us – human beings being human beings.  In particular I am fascinated by ‘Rationals’.  Whom I am speaking about when I speak ‘Rationals’?  I am pointing at/towards my fellow human beings who pride themselves in being ‘rational’, ‘objective’, ‘scientific’ – they usually love order, logic, reason and are attracted to / come from ‘engineering-science-accountancy-economics-mathemetics” type disciplines.

What is it that I find fascinating about my fellow ‘Rationals’?  Before I go further I should point out that I used to be a ‘Rational’ and do get sucked back into being a ‘Rational’ if I am not being mindful.  What do you expect?  I studied Mathematics, Physics, Chemistry.  I have a BSc in Applied Physics and then went on to study accountancy and qualified as a Chartered Accountant.  OK back to my question: what do I find fascinating about ‘Rationals’?  Bluntly speaking, ‘Rationals’ are the most irrational people and they totally do not get this paradox, this joke.   Put differently, ‘Rationals’ are blind they are to the way that human beings work, organisations work, society works.  They are suckers for universal principles and insist that the world act in accordance with these universal principles.  One of these fundamental “shoulds” is that “people should behave rationally”.  Why does this matter?

Right now there is a fad in progress.  The people behind this fad hail, loudly and frequently, the numerous wonders (and benefits) of what data and analytics can do.  I suspect that many of them are ‘Rationals’ with a good sprinkling of ‘Marketing’ types thrown in.  They proclaim that big data and state of the art analytics (social, content, text, predictive…..) will light up our world and lead to the promised land: mountains of revenue; costs trimmed to the bone – everything working so efficiently so as to render void the 2nd law of thermodynamics (order to disorder) and the messiness of the real world; and an ocean of profits.  And all you have to do is to mine the Big Data!

The ideal world: how ‘Rationals’ assume the world works

The ‘Rationals’ assume that every person and certain every influential person making decisions is John Maynard Keynes (the famous economist).  What do I mean?  He was once asked how he responds to new data that did not support his earlier decisions and judgements.  JMK replied “I change my opinion. What do you do sir?”

Yes, that is the ideal.  Every little one of us as a perfect computer: taking in data as input, crunching that data against any number of dependable algorithms, spitting out the answer and doing what is in line with that answer ignoring our ‘points of view’, our ‘prejudices and bias’.  Now lets take a look at reality.

The real world: human beings are strange, marvellous, creatures

Daniel Kahneman in his latest book (Thinking, fast and slow) spells out that human beings are essentially a meaning making organism that thinks/works in stories and jumps to instant conclusions as long as the story fits the preconceived schema.  He writes “The implication is clear: as the psychologist Jonathan Haidt said in another context, “The emotional tails wags the rational dog.” The affect heuristic simplifies our lives by creating a world that is much tidier than reality….” Mr Kahneman has titled one his chapters “Causes trump statistics“.

Lets just ignore the fact that the people interpreting data and presenting it to decision makers are human and they exhibit the same ‘marvels and failings’ as the ordinary person: in test after test conducted by Daniel Kahneman and Amos Tversky the professional statisticians made similar ‘errors’ to the ordinary person.  By ignoring this you would think I have just spelled out an excellent reason to deploy analytics to drive decision making – to take out these human failings.  If only the world was that simple my ‘Rational’ friend.  Let me share three stories with you to illuminate the nature of the real world.

Story 1.  On page 116, Mr Kahneman tells the story of how Tom Gilovich and Robert Vallone did a statistical analysis of thousands of sequences of shots in basketball and concluded that there is no such thing as a hot hand in basketball.  Thats right according to the data and the statistical interpretation of data, there is no such thing as a basketball player having a hot hand – it is a human invention.  Now here is the instructive part, how did the basketball public (coaches, media, fans…) react to this conclusion?  Disbelief – just in case you don’t get that it means they did not believe it!  This is what Red Auerback, the celebrated coach of the Boston Cetics said: “Who is this guy? So he makes a study. I couldn’t care less.”

Story 2.  There is an amusing and enlightening story about human beings and it goes like this.  Once upon a time an ordinary villager died.  His body was cleansed, taken to the local mosque, the villagers gathered together at the mosque, the prayers were said and all the necessary rituals performed with the family present.  Then the body was put into the coffin.  Some of the villagers lifted the coffin on their shoulders and headed for the cemetry.  On the way they heard a knocking coming from the inside of the coffin. Then they heard a voice say “Let me out of here. Let me out of here. I’m alive, why have you put me in this box?  Let me out of here!”  The folks carrying the coffin replied “No, this is a trick.  You are dead.  We have said prayers and carried out all the necessary rituals.  Now it is time to bury you.”  No matter how much the villager pleaded to be let out the folks carrying his coffin refused to listen.  They were adamant that he could not be alive after all hadn’t all the villager seen his dead body, said prayers….. they would not be made fools of by the ‘dead man’ and end up being laughed at by the villagers when they returned to the village, opened the coffin and found that the the voice coming from the coffin was a hoax!  No, he was dead, everyone knew that!

Story 3. During World War II there was a notable disaster – Operation Market Garden.  This is what Wikipedia says, “Montgomery was able to persuade Eisenhower to adopt his strategy of a single thrust to the Ruhr with Operation Market Garden in September 1944….. the operation failed with the destruction of the British 1st Airborne Division at the Battle of Arnhem and the loss of any hopes of invading Germany by the end of 1944.”  What makes this relevant is the fact that this disaster could have been avoided.  How?  Brian Urquhart.  This is what Wikipedia says, read it carefully:

“In the autumn, as the 1st Airborne Corps Intelligence Officer, he assisted with the planning for Operation Market Garden, an ambitious airborne operation designed to seize the Dutch bridges over the rivers barring the Allied advance into northern Germany. He became convinced that the plan was critically flawed, and attempted to persuade his superiors to modify or abort their plans in light of crucial information obtained from aerial reconnaissance and the Dutch resistance. The episode was described by Cornelius Ryan in his book on “Market Garden”, A Bridge Too Far. (In the film version, directed by Richard Attenborough, Urquhart’s character was renamed “Major Fuller”, to avoid confusion with a similarly named British General.) ……… but he became deeply depressed by his failure to persuade his superiors to halt the operation and requested a transfer out of the airborne forces.

General Browning ignored the intelligence supplied by Brian Urquhart, the intelligence officer.  Why?  He didn’t want to have to tell his boss Field Marshall Montgomery that he was cancelling another operation – many airborne operations had previously been cancelled.  How did the operation turn out?

So what do we have here?  We have an intelligence officer doing his job and providing the intelligence.  The intelligence goes against what the top brass is committed to and the intelligence is ignored, the operation goes ahead and it is a disaster.  Do you think this is a one-off event?  It happens all the time: think Iraq and WMDs; think of the financial crisis and the reckless lending that led to it and the warnings that were ignored….  Does the Sufi story (told earlier) sound far fetched now?

If you want to get value out of data and analytics then deal with human nature as it is

I love zen – it says see life as it is and as it is not, leave behind your mind full of theories, concepts, projections of how you want things to be.   You would do well to act on that advice before you spend a fortune on “big data and analytics”.   Why?  It is not easy to get the right data (information overload as much of an issue as data quality and data integration) and convert that into useful intelligence that can drive decision-making.  No it is not easy.  Yes, I do know that the smooth tongued marketers are promising you that it is so easy.  The reality is that it takes time (lots of it), effort (lots of it) and money (lots of it).  Yes, the technical aspect is doable if you put in the time, effort and money that is required: the experts will come in and do it for you.

The hard part, the really hard part, is the human part.  It is dealing with, working with, human nature as it is. Human beings do not get data and do not value data – they simply do not relate to it like they relate to, say, a dog.  Do you really think that the people in your organisation appreciate the value of data and put in the time and effort to enter the right data into the right fields?  If you think that then you have spent far too much of your time in the Ivory Tower of the executive office.   Yet even that issue – of getting your people to enter the right data into the right fields/systems – is insignificant to the real issue.  What is the real issue?  Getting managers to give up their pet theories, their ideological convictions, their vested interests, their intuition, their past experience and use data and analytics to make decisions.  That is the central issue that you have to and should deal with.

My advice

Before you go and spend a fortune on ‘big data and analytics’ do the following:

Find out the total cost of ownership.  The set-up cost (people, technology, other) and the on going operational cost.  Statisticians don’t come cheap and then their are annual software licensing fees to think of…

Hold a ‘Big Data and Analytics’ party and invite all the key people from you business to that party.  Spell out the wonders that ‘Big Data and Analytics’ will bring them and the company.  Then ask them to pay (out of their existing budgets as they are) to attend the party.  The price of admission to this party?  Just divide the total cost of ownership (say over three years) between each of the players in the organisation.  Then see who turns up.  That might give a true picture of how much passion there is for ‘Big Data and Analytics’ within your organisation.  Or you can try the “build it and they will come” approach – your party, your choice!

Why big data and analytics will not make you the next Apple

Are  ‘big data’ and ‘analytics’ the latest fads?

In my view there is a little too much hype around ‘big data’ and the power of analytics to drive business growth and profitability.  Can ‘big data’ and ‘analytics’ help you improve operations?  Yes – they can help you better staff and manage your call centre  or improve your supply chain or target your marketing better.   Will ‘big data’ and analytics make you the next Apple?  Highly unlikely.  Why does that matter?  Because whilst you are busy optimising operations someone else is inventing a future in which your optimised operations become redundant: think music, think publishing, think mobile phones.  Let’s explore this further.

What are the root causes?

Years ago when I worked in the business planning & analysis team of a brand name drinks company I noticed something interesting.   Managers could drill down and find out which particular markets had failed to make their numbers.  Great – we know which business unit is underperforming.  But why is this unit doing better or worse than expected?  This is where the fun started.  First, there were almost as many opinions as the people we asked.  Second, there was no easy to work out whether the answers given by local management held any resemblance to the ‘truth’.  When I dug further I found out that local management did not know why they had failed to make their numbers (revenues).  When they were asked, the local managers simply came up with the most plausible story.  So month end because a ritual where HQ asked questions and the local managers invented plausible stories.

Lesson 1:  For any complex event there are a multitude of plausible answers and working out the ‘real reason’ is notoriously difficult.

Lesson 2:  Without a sound grasp of the root causes it is difficult to formulate a sensible course of action to address the situation.

What do we do about it?

Lets assume that you have all the data and analysis has been done.  You know there is a problem.  What do you do about it?  Is it easy to get all the actors – who have to play ball – to agree on the course of action to take?  In my experience the answer is no: the more ‘strategic’ the issue at hand the more difficult it is to come to an agreement on a sensible course of action.  Let’s take a look at the recent banking crisis.  Did all the main actors (the western economies) come to a consensus on what course of action to take?  No.  Just take a look at the Euro crisis: why is it that the leaders of the EU cannot agree on the right course of action?  First, because different people have different ideas about what constitutes the right course of action.  Second, the right course of action from an objective perspective may simply not be viable from a political perspective.

Lesson 3: The more that is at stake the harder it is to get all the actors to agree to a single course of action and then act to play their part and execute that course of action

Lesson 4: If you are unable to act decisively and as a single unit then all the data, analysis and insight is worthless

Does it really tell you what you need to know to thrive in the future?

Larry Freed has written an article that resonates with me.  He points out that you need to be clear on what you know and what you do not know.  I’d say that you need to be clear about what data and analytics is telling you and what it is not telling you.  Larry, talking about a website, asks do you know:

  • Why visitors come to your site (to research, to buy, to complete a transaction, to get product support, to learn more about your company before interacting with you through another touch point, etc.)?
  • What influences visits to your site (a referral, a social media interaction, a failure to resolve an issue with a call centre, an advertisement, a news story, a previous affinity with your brand, etc.) and which customer acquisition sources result in traffic that is the mostly likely to convert?
  • What visitors need from your website? How needs differ by population segment or other segmentation that is useful to your business—perhaps first-time vs. repeat visitors, heavy users vs. light users, etc.?
  • What visitors expect from your website? Do men and women have the same expectations? Old and young? Do people who arrived as a result of a Google ad have the same expectations as those who arrived because of a TV ad?
  • What channel your visitors prefer, and are there ways you can influence that preference so they frequent less costly, more profitable channels?
  • How customers view your business, compared to the way non-customers view your business, relative to your competition?
  • How your customer profiles and expectations change in response to market and broader economic conditions? And what, if anything, you need to change as a result?”

Colin Shaw in a recent post makes the same point in a different way.  Here are some relevent extracts from his post:

“Google and Facebook, as Eli Pariser discusses as a part of TED, are engaged in the process of quantifying preferences from the timing and frequency of online clicks, and using this information to alter web content. The stated goal of this practice is to “personalize” the web experience.

A lesson gleaned from Dell’s 1990s laptop boom illustrates the point that preference and value are two different things. Dell let its customers customize all aspects of their computer’s hardware – from screen size to keyboards to RAM – everything but color. Nobody thought to customize color, because a laptop was supposed to be black or gray. However, when color laptops were introduced, sales skyrocketed and we all learned that color was indeed an important factor.

Imagine if Google and Facebook had monitored “clicks.” They would infer that because customers did not indicate a laptop color preference, it doesn’t drive value and is therefore irrelevant.”

Lesson 5: there is an assumption behind all predictive analytics and that is “all things being equal” – that is to say that predictive analytics assumes that the future will be a replay of the past.

Lesson 6: human behaviour is shaped by the ‘structures’ in which human beings are embedded, change the structure and you are likely to see human beings change their behaviour. Think about how the recession (e.g. job losses) have changed the shopping habits of consumers in the western economies.

Why won’t big data and analytics make you into the next Apple?

Apple was busy creating a new future (a break from the past) rather than exploiting the past.  If you take a look at the US automotive industry the big US automakers were busy building and selling gas guzzlers because the analytics showed that these were the cars that Americans were buying.  At the same time Toyota was busy living into a very different vision of the future: hybrid cars and electric cars.   Who was right?  According to the data and the analytics it was the US automakers.  What would you say now?  Toyota?

If you are not inventing the future you can still prosper by picking up the weak signals that point towards a new trend.  I once asked Bob Greenberg (R/GA) the secret of his success and he told me it was his ability to see these trends and act upon them before others.  You might imagine that analytics might help you to spot trends.  My experience of traditional analytics is that the modelers do all they can to strip out the outliers and create a normal distribution so that the maths works – in doing that they filter out the ‘weak signals’ that point towards these trends.

Perhaps it is best to end by remembering what Colin Shaw points out: how would analytics have disclosed that customers wanted to customise the colour of their laptops and that once this option was made available then Dell’s laptop sales would surge.

What do you think?  If I have it wrong then please do educate me.