On LinkedIn, Don Peppers is sharing his perspective on making better decisions with data. This got me thinking and I want to share with you what showed up for me. Why listen to my speaking? I do have a scientific background (BSc Applied Physics). I qualified as a chartered accountant and was involved in producing all kinds of reports for managers and saw what they did or did not do with them. More recently, I was the head of a data mining and predictive analytics practice. Let’s start.
Data and data driven decision-making tools are not enough
Yes, there is a data deluge, and this deluge is becoming down faster and faster. Big enough and fast enough to be given the catchy name Big Data. What is forgotten is the effort that it takes to get this data fit for the purpose of modelling. This is no easy-cheap task. Yet, it can be done if you throw enough resources at it.
Yes, there are all kinds of tools for finding patterns in this data. And in the hands of the right people (statistically trained-minded, business savvy) these tools can be used to turn data into valuable (actionable) insight. This is not as easy as it sounds. Why? Because there is shortage of these statistically trained and minded people: amateurs will not do, experts are necessary to distinguish between gold and fools gold – given enough data you can find just about any pattern. It statistical savvy is not enough you have to couple it with business savvy. Nonetheless, let’s assume that we can overcome this constraint.
The real challenge in generating data driven decision-making in businesses is the cultural practices. We do not have the cultural practices that create the space for data driven decision-making to show up and flourish. A thinker much smarter-wiser than me has already shared his wisdom, I invite you to listen:
On the whole, scientific methods are at least as important as any other research: for it is upon the insight into the method that the scientific spirit depends: and if these methods are lost, then all the results of science could not prevent a renewed triumph of superstition and nonsense.
Clever people may learn as much as they wish of the results of science – still one will always notice in their conversation, and especially in their hypotheses, that they lack the scientific spirit; they do not have the distinctive mistrust of the aberrations of thought which through long training are deeply rooted in the soul of every scientific person. They are content to find any hypothesis at all concerning some matter; then they are all fire and for it and think that is enough …….. If something is unexplained, the grow hot over the first notion that comes into their heads and looks like an explanation ….
– Nietzsche (Human, All Too Human)
It occurs to me that the scientific method never took route in organisational life. Put aside the rationalist ideology and take a good look at what goes in business including how decisions are made. I say you will find that Nietzsche penetrating insight into the human condition as true today as when he spoke it. The practice of making decisions in every organisation that I have ever come in contact with is not scientific: it does not follow the scientific method. On the contrary, managers make decisions that are in alignment with their intuition, their prejudices, and their self-interest. It is so rare to come across a manager (and organisation) that makes decisions using the scientific method that when this does occur I am stopped in my tracks. It is the same kind of unexpectedness as seeing a female streaker running across the football pitch in a league match.
What are the challenges in putting data driven decision-making practices into place in organisations?
Technologists have a gift. What gift? The gift of not understanding, deeply enough, the being of human beings. Lacking this understanding they can and do (confidently) stand up and preach the virtues-benefits of technology. If life were that simple.
Truth shows up as attractive to those of us who do not have to face the consequences of truth. Data driven decision-making sounds great for those of us selling (making a living and hoping to get rich) data driven tools and services.
The challenge of putting in place data driven decision-making practices is that it disturbs the status quo. When you disturb the status quo you go up against the powerful who benefit from that status quo. Remember Socrates:
The very nature of what Socrates did made him a disruptive and subversive influence. He was teaching people to question everything, and he was exposing the ignorance of individuals in power and authority. He became much loved but also much hated …. In the end the authorities arrested him for …., and not believing in the gods of the city. He was tried and condemned to die …
– Bryan Magee, Professor
Beware of being successful in putting in place a culture of data driven decision making!
With sufficient commitment and investment you can put in place a data driven decision making culture. Like the folks at Tesco did. And by making decisions through harnessing the data on your customers, your stores, your products, you can outdo all of your competitors, grow like crazy and make bumper profits. Again, again, and again. Then the day of reckoning comes – when you come face to face with the flaws of making decisions solely on the basis of data.
Tesco is not doing so great. It has not been doing so great for several years – including issuing its first ever profits alert in 2012. What is the latest situation? Tesco has reported a 23.5% drop in profits in the first half of this year. What has Tesco been doing to deal with the situation? This is what the article says:
Last year, Tesco announced it would be spending £1bn on improving its stores in the UK, investing in shop upgrades, product ranges, more staff, as well as its online offering.
There are a number of flaws on data driven decision making. For one data driven decision making assumes that the future will be a continuation of the past. Which is rather like saying all the swans that we have come across are white, so we should plan for white swans. And then, one day you find that the black swan shows up! The recession and the shift in consumer behaviour that resulted from this recession was the black swan for Tesco.
Furthermore, I hazard a guess that in their adoration at the pulpit of data driven decision making the folks at Tesco forgot the dimensions that matter but were not fed into the data and the predictive models. What dimensions? Like the customer’s experience of shopping at Tesco stores: not enough staff, unhappy staff, stores looking more and more dated by the day, the quality of their products ……
It looks like the folks at Tesco did not heed the sage words of one of my idols:
Not everything that counts can be counted, and not everything that can be counted counts.