Want to get value out of your data and analytics investment? Then deal with this issue before you buy the software.

‘Rationals’, data and the wonders of analytics

I think I have said this before and I will say it again: my fascination is with us – human beings being human beings.  In particular I am fascinated by ‘Rationals’.  Whom I am speaking about when I speak ‘Rationals’?  I am pointing at/towards my fellow human beings who pride themselves in being ‘rational’, ‘objective’, ‘scientific’ – they usually love order, logic, reason and are attracted to / come from ‘engineering-science-accountancy-economics-mathemetics” type disciplines.

What is it that I find fascinating about my fellow ‘Rationals’?  Before I go further I should point out that I used to be a ‘Rational’ and do get sucked back into being a ‘Rational’ if I am not being mindful.  What do you expect?  I studied Mathematics, Physics, Chemistry.  I have a BSc in Applied Physics and then went on to study accountancy and qualified as a Chartered Accountant.  OK back to my question: what do I find fascinating about ‘Rationals’?  Bluntly speaking, ‘Rationals’ are the most irrational people and they totally do not get this paradox, this joke.   Put differently, ‘Rationals’ are blind they are to the way that human beings work, organisations work, society works.  They are suckers for universal principles and insist that the world act in accordance with these universal principles.  One of these fundamental “shoulds” is that “people should behave rationally”.  Why does this matter?

Right now there is a fad in progress.  The people behind this fad hail, loudly and frequently, the numerous wonders (and benefits) of what data and analytics can do.  I suspect that many of them are ‘Rationals’ with a good sprinkling of ‘Marketing’ types thrown in.  They proclaim that big data and state of the art analytics (social, content, text, predictive…..) will light up our world and lead to the promised land: mountains of revenue; costs trimmed to the bone – everything working so efficiently so as to render void the 2nd law of thermodynamics (order to disorder) and the messiness of the real world; and an ocean of profits.  And all you have to do is to mine the Big Data!

The ideal world: how ‘Rationals’ assume the world works

The ‘Rationals’ assume that every person and certain every influential person making decisions is John Maynard Keynes (the famous economist).  What do I mean?  He was once asked how he responds to new data that did not support his earlier decisions and judgements.  JMK replied “I change my opinion. What do you do sir?”

Yes, that is the ideal.  Every little one of us as a perfect computer: taking in data as input, crunching that data against any number of dependable algorithms, spitting out the answer and doing what is in line with that answer ignoring our ‘points of view’, our ‘prejudices and bias’.  Now lets take a look at reality.

The real world: human beings are strange, marvellous, creatures

Daniel Kahneman in his latest book (Thinking, fast and slow) spells out that human beings are essentially a meaning making organism that thinks/works in stories and jumps to instant conclusions as long as the story fits the preconceived schema.  He writes “The implication is clear: as the psychologist Jonathan Haidt said in another context, “The emotional tails wags the rational dog.” The affect heuristic simplifies our lives by creating a world that is much tidier than reality….” Mr Kahneman has titled one his chapters “Causes trump statistics“.

Lets just ignore the fact that the people interpreting data and presenting it to decision makers are human and they exhibit the same ‘marvels and failings’ as the ordinary person: in test after test conducted by Daniel Kahneman and Amos Tversky the professional statisticians made similar ‘errors’ to the ordinary person.  By ignoring this you would think I have just spelled out an excellent reason to deploy analytics to drive decision making – to take out these human failings.  If only the world was that simple my ‘Rational’ friend.  Let me share three stories with you to illuminate the nature of the real world.

Story 1.  On page 116, Mr Kahneman tells the story of how Tom Gilovich and Robert Vallone did a statistical analysis of thousands of sequences of shots in basketball and concluded that there is no such thing as a hot hand in basketball.  Thats right according to the data and the statistical interpretation of data, there is no such thing as a basketball player having a hot hand – it is a human invention.  Now here is the instructive part, how did the basketball public (coaches, media, fans…) react to this conclusion?  Disbelief – just in case you don’t get that it means they did not believe it!  This is what Red Auerback, the celebrated coach of the Boston Cetics said: “Who is this guy? So he makes a study. I couldn’t care less.”

Story 2.  There is an amusing and enlightening story about human beings and it goes like this.  Once upon a time an ordinary villager died.  His body was cleansed, taken to the local mosque, the villagers gathered together at the mosque, the prayers were said and all the necessary rituals performed with the family present.  Then the body was put into the coffin.  Some of the villagers lifted the coffin on their shoulders and headed for the cemetry.  On the way they heard a knocking coming from the inside of the coffin. Then they heard a voice say “Let me out of here. Let me out of here. I’m alive, why have you put me in this box?  Let me out of here!”  The folks carrying the coffin replied “No, this is a trick.  You are dead.  We have said prayers and carried out all the necessary rituals.  Now it is time to bury you.”  No matter how much the villager pleaded to be let out the folks carrying his coffin refused to listen.  They were adamant that he could not be alive after all hadn’t all the villager seen his dead body, said prayers….. they would not be made fools of by the ‘dead man’ and end up being laughed at by the villagers when they returned to the village, opened the coffin and found that the the voice coming from the coffin was a hoax!  No, he was dead, everyone knew that!

Story 3. During World War II there was a notable disaster – Operation Market Garden.  This is what Wikipedia says, “Montgomery was able to persuade Eisenhower to adopt his strategy of a single thrust to the Ruhr with Operation Market Garden in September 1944….. the operation failed with the destruction of the British 1st Airborne Division at the Battle of Arnhem and the loss of any hopes of invading Germany by the end of 1944.”  What makes this relevant is the fact that this disaster could have been avoided.  How?  Brian Urquhart.  This is what Wikipedia says, read it carefully:

“In the autumn, as the 1st Airborne Corps Intelligence Officer, he assisted with the planning for Operation Market Garden, an ambitious airborne operation designed to seize the Dutch bridges over the rivers barring the Allied advance into northern Germany. He became convinced that the plan was critically flawed, and attempted to persuade his superiors to modify or abort their plans in light of crucial information obtained from aerial reconnaissance and the Dutch resistance. The episode was described by Cornelius Ryan in his book on “Market Garden”, A Bridge Too Far. (In the film version, directed by Richard Attenborough, Urquhart’s character was renamed “Major Fuller”, to avoid confusion with a similarly named British General.) ……… but he became deeply depressed by his failure to persuade his superiors to halt the operation and requested a transfer out of the airborne forces.

General Browning ignored the intelligence supplied by Brian Urquhart, the intelligence officer.  Why?  He didn’t want to have to tell his boss Field Marshall Montgomery that he was cancelling another operation – many airborne operations had previously been cancelled.  How did the operation turn out?

So what do we have here?  We have an intelligence officer doing his job and providing the intelligence.  The intelligence goes against what the top brass is committed to and the intelligence is ignored, the operation goes ahead and it is a disaster.  Do you think this is a one-off event?  It happens all the time: think Iraq and WMDs; think of the financial crisis and the reckless lending that led to it and the warnings that were ignored….  Does the Sufi story (told earlier) sound far fetched now?

If you want to get value out of data and analytics then deal with human nature as it is

I love zen – it says see life as it is and as it is not, leave behind your mind full of theories, concepts, projections of how you want things to be.   You would do well to act on that advice before you spend a fortune on “big data and analytics”.   Why?  It is not easy to get the right data (information overload as much of an issue as data quality and data integration) and convert that into useful intelligence that can drive decision-making.  No it is not easy.  Yes, I do know that the smooth tongued marketers are promising you that it is so easy.  The reality is that it takes time (lots of it), effort (lots of it) and money (lots of it).  Yes, the technical aspect is doable if you put in the time, effort and money that is required: the experts will come in and do it for you.

The hard part, the really hard part, is the human part.  It is dealing with, working with, human nature as it is. Human beings do not get data and do not value data – they simply do not relate to it like they relate to, say, a dog.  Do you really think that the people in your organisation appreciate the value of data and put in the time and effort to enter the right data into the right fields?  If you think that then you have spent far too much of your time in the Ivory Tower of the executive office.   Yet even that issue – of getting your people to enter the right data into the right fields/systems – is insignificant to the real issue.  What is the real issue?  Getting managers to give up their pet theories, their ideological convictions, their vested interests, their intuition, their past experience and use data and analytics to make decisions.  That is the central issue that you have to and should deal with.

My advice

Before you go and spend a fortune on ‘big data and analytics’ do the following:

Find out the total cost of ownership.  The set-up cost (people, technology, other) and the on going operational cost.  Statisticians don’t come cheap and then their are annual software licensing fees to think of…

Hold a ‘Big Data and Analytics’ party and invite all the key people from you business to that party.  Spell out the wonders that ‘Big Data and Analytics’ will bring them and the company.  Then ask them to pay (out of their existing budgets as they are) to attend the party.  The price of admission to this party?  Just divide the total cost of ownership (say over three years) between each of the players in the organisation.  Then see who turns up.  That might give a true picture of how much passion there is for ‘Big Data and Analytics’ within your organisation.  Or you can try the “build it and they will come” approach – your party, your choice!

Author: Maz Iqbal

Experienced management consultant working at the intersection of strategy, customer, and technology. Combine a tendency to think strategically with a penchant for getting my hands dirty at the coalface of implementation.

9 thoughts on “Want to get value out of your data and analytics investment? Then deal with this issue before you buy the software.”

  1. Very good post again, and true not just for the “Big Data” but for smaller tailored qualitative studies as well. How many times did we bring the hard facts to management only to see the associated recommendations thrown out the window because it wasn’t what they wanted to hear. There’s actually big business in writing research reports tailored to what the commissioning manager wants. Don’t even need to actually do fieldwork. But of course none of these research companies would admit to that (would be against the code of ethics). They just call it “massaging” the data so that it matches whatever narrative (story) the manager has already constructed in his mind, and the only one he or she will believe.

    Like

    1. Hello David

      My experience is in perfect alignment with yours. One of my biggest wake up calls was getting that customers were bringing us in (Big5 consultancies) to badge and sell their ideas to the Tops using our brand.

      As Kahneman says “Causes stump statistics.”

      If you spend any time in the psychotherapy / counselling world you will find that ‘client’ hunt around until a find a therapist that ‘buys into their story’ and sells the ‘client exactly what this clients as expressed in buying’. It is exactly the same in all domains of life and business is a domain of life. Human beings cannot help being human beings.

      All the best
      Maz

      Like

  2. It is interesting to note that numbers tell a story, amazingly, a different story according to how different people depending on how they read the numbers. Just like everything in life, the number story is interpreted differently depending on ones ‘life filters’. I find it interesting that management listens to information that supports their theory and throws out those that contradicts. The research report unfortunately, is also churned out according to the viewpoint and interpretation of the report writer. Management truly has to attend the ground research to ensure and thoroughly understand the accuracy of the report interpretation, as they bring in the understanding of the company versus the ‘rest of the world’. Indeed I was about to embark on one and am now mindful of the management’s receptiveness to analytics – your article was timely, thank you.

    Like

    1. Hello Anneliza

      Love your name!

      Management is simply a term we use to describe a bunch of human beings. As human beings we cannot help being human beings no more than a fish can help being a fish. If we use the that analogy (the fish) then human beings live in and from ‘storytelling’ and ‘ideologies- fixed patterns of thinking and interpreting the world’.

      Data/analytics when it comes up against ‘storytelling’ and ‘ideological commitments to certain values and courses of action’ will almost always fail. Yet there are some people who are more open than others and therein lies the opportunity. Furthermore, there are times when people are more uncertain about themselves, the world and thus more open to trying out new things out of desperation – that is also an opportunity.

      Thanks for dropping by and creating a conversation.

      Maz

      Like

  3. I’m so pleased that I came across THE CUSTOMER BLOG. It is really enjoyable to read your posts. I find them interesting, educational, informative, relevant, and thought provoking. I love the perspective you bring Maz. It is reassuring to see other people so passionate about customers.

    Like

    1. Hello Cathy
      Would it be fair to say we have not ‘met’ before?

      And you have lit up my world with your kind, supportive, encouraging and inspiring words. I thank YOU with all of my heart – your words make a HUGE difference to me. To know that I contribute to you matters to me. The whole point of this blog is for me to share my authentic voice AND contribute to my fellow human beings. Thank you for letting me know that I am living my stand in life!

      Would it be going to far to tell you that I love you? I’ll let you decide that. What I can tell is that in my heart there is love for you. I am grateful that you exist and our lives have touched even if it is online!

      Be great and do visit again. You are welcome!

      Maz

      Like

  4. Hi Maz,

    It strikes me that there are a number of different arguments flowing through your post.

    People aren’t rational, even the rational ones
    The irrational don’t listen to the rational
    I could take this to its logical conclusion (therefore…)

    I happen to agree with you that big data is a big expensive, and I won’t be signing a cheque any time soon, but I still have a problem:

    How can I help people see things that they do not see? Regardless of whether it is a rational perspective, or not?

    More to the point, how can I see those things myself?

    James

    Like

  5. Hello James
    Great points. From a zen perspective the answers to your questions are:

    a) practice being mindful – focus on the ‘here and now’;

    b) practice being mindful of being mindful, that is to say practice keeping mindfulness in existence;

    c) work from the stance that your ‘ego is playing you all the time’ and as such you will be utterly convinced of the truth of your position even when / especially when it is false;

    d) cultivate the practice of inviting in and listening, really listening, to the people who you find difficult to get along with – guess what they see the world very differently to you’

    e) use Poppers dictum – look to prove that your ‘point of view / hypothesis’ is false – and if you do not find facts on the ground to prove it is false then all that you can say is that ‘as yet I have not proved it to be false’.

    Now you know why so few of us do this. It is hard work and for some it occurs as impossible.

    All the best to you my friend. And thank you for your on going contribution to my life. I hope you forgive me for my tardiness in responding to you – been busy on various fronts!

    maz

    Like

Comments are closed.