The folly of getting customer feedback through automated surveys

I have been an Orange customer for many years and have several usable phones lying around.  This week I decided to let my wife have one of my newer phones so that she can access email on the go.  So I rang Orange to get the phone unlocked.  Call me naive but I did not expect any problems given that the phone is over 2 years old and Orange has made a significant amount of money from me.

The Orange customer services agent (CSA) told me that it would cost £20.00 to get the phone unlocked.  When I asked why given that Orange had more than recovered any phone subsidy.  She simply told me that it was company policy.  To which I simply said “thank you, I’ll see if I can get it unlocked elsewhere” and hung up.

Then I got an automated SMS based survey to complete.  Allow me to share that with you because it illustrates the folly of getting customer feedback through automated surveys. He goes:

Orange: “Thanks for calling Orange at……We’d love your feedback to help improve our service.  It’s only 4 questions and it’s free to take part.  Just reply YES”

Me: “YES”

Orange: “Firstly, on a scale of 0-10 where 0 is not at all and 10 is a great deal, how much did the customer service representative you spoke to care about your query?”

Me:  My first reaction was how the heck do I know!  Do you think I can read minds?  So I simply texted “5” to represent I do not know and this is a daft question.

Orange: “Thanks. Was your contact or query resolved? (text yes/no)”

Me: How do I answer that one.  Yes, I got an answer.  But no, I did not get the answer that I was expecting.  No, I did not feel that Orange treated me right as a loyal customer. So I ended up texting “No”

Orange: “Are you confident that the representative will resolve your query? (yes/no)”

Me: “No”

Orange: “On a scale of 0-10 where 0 is not at all likely and 10 is very likely, how likely are you to recommend Orange to a friend or a colleague?”

Me: “0”

Orange: “Finally, you could the representative you spoke to have improved the service you received when you contacted Orange?”

Me: “She can’t. Orange policy is preventing her from helping me out”

Orange: “As a treat for completing this survey you are now eligible for £100 off Orange Home Broadband! To redeem call 08000790035 – Orange”

Me:  at this point I simply laughed at the absurdity of my ‘treat’!

My take on this

If you want to learn about your customers and cultivate emotional bonds then it pays to have an empathic human being conduct that survey. Here is a great example.  If such a human being had contacted me it is possible that the gulf between me and Orange would be a non-existent and I would be an advocate.  Orange would have given me the ‘treat’ I rang up for (unlock my phone) rather than push a marketing message (Orange Broadband) at me and call it a ‘treat’.  In fact it is this  automated ‘treat’ that has left me more distant from Orange then the call centre representative – she was simply doing her job.

The purpose of the Orange survey (and many others like it) is to assess and control the performance of call centre agents.  But how exactly do you figure out whether the issue is with the policies and practices that the call centre agents have to follow or with the call centre agents themselves?  I am not a happy customer yet I am clear that is not the fault of the call centre agent.  My unhappiness is with the fact that I have been an Orange customer for nearly ten years and still got treated like a number.  Why should this matter to Orange?  Because I will be moving my entire family of Orange.  And I will do my best to get one of Orange’s VIP customers to move as well.A

Automated customer surveys really cannot capture the rich nuances that customers have to share. If you take a look at my interaction you will find that I struggled to answer some of the questions and in the end simply made things up.  The danger is that business decisions might be made from ambiguous answers like the ones that I gave.  The richer, deeper, insight is simply not understood and therefore cannot be acted upon.

5 guidelines for getting the most out of your mystery shopping programme

Some organisations are turning to mystery shopping to get an insight into the customer experience, the performance of their front line staff and what competitors are up to.  The idea is to use mystery shopping to find the ‘nuggets of gold’ that will enable the organisation to improve the competitive position by improving the customer experience.   Here are five guidelines to help you plan, execute and get the most out of your mystery shopping investment.

1. Begin with the end in mind

In order to shape the mystery shopping it is best to start with the end in mind.  What are you going to do with the mystery shopping?  What aspects of your business are you willing to change?  What aspects are simply out of scope?  Who will need to buy into any changes that you might make as a result of what you may find in the mystery shopping?

By answering these questions you can better shape the mystery shopping and involve the right people from the start.  Shaping the mystery shopping will allow you to focus the mystery shopping on the right areas and thus get the most value out of your budget.  Involving the right people is essential to getting the parts of the organisation to make changes in what they do and how they do it.

If you do not begin with a real clarity about the end that you have in mind then you end up with a situation like the US finds itself in as regards Afghanistan and Iraq.  I cannot stress enough that you should take the time to really think through the end game: what does success look like?; what constitues failure? ; what obstacles / hurdles you might come across and who you will deal with them? .   I have been involved in a diverse range of projects over the last 20+ years and when I look back I can clearly see that success or failure was built into projects and programmes right from the start.  It just took a little time for it to show up.

2. The most important part of mystery shopping is what you do with the results

The only organisations that get value out of mystery shopping are the organisation that act on the results – they change what they do and how they do it.    Interestingly there was a post on the 1to1 blog titled ‘You’re Not Listening Unless You Act on What You Hear’, I recommend that you read it.

This means that you have to plan how you are going to translate the results into improvement actions.  How exactly are you going to get the behavioural changes in the stores, in the call centres, on the website etc?  The key finding here is that the best way to effect change is to simply share the ‘real life’ mystery shopping experience with your front line staff and managers.  Let your people experience the actual shopping experience by seeing the video and hearing the audio recordings of the mystery shopping.  Why?  Because this has a much more potent emotional (experiential) impact than a researcher or consultants Powerpoint slides or research findings.

3. Ensure that your mystery shopping reflects your customer segments and how they shop

Sounds obvious yet it is a common mistake for you / your brand folks / executives to specify the mystery shopping programme that gives away the game.  What do I mean?  First, your brand folks tend to assume that what they consider important in the shopping experience is what the actual customers consider important.  Second, they assume that the way that they specify the shopping experience is the way that your customers actually shop.  And when the mystery shopper actual shops your front line people know that this person is a mystery shopper and not a real customer.  How?  Because real customers simply do not ask the kind of questions that your mystery shoppers ask nor do they behave the way that your mystery shoppers behave.

There is a great example of a premium automotive brand that instructed the mystery shoppers to ask about a specific feature that the brand folks are proud of and think should matter to customers.  Yet real customers simply do not ask about that feature.  So when the mystery shoppers asked about this feature they identified themselves as mystery shoppers to the sales folks in the dealerships.  And this clearly skewed the findings of the mystery shopping programme.

Next, it really helps to match the demographics of the people doing the mystery shopping with the demographics of your customer base.  If you want to get granular  insight then it really pays to segment your customer base into a small number of actionable customer segments and do mystery shopping at the level of these segments.  Why?  To work out the specific shopping experience of each customer segment.  The same process/behaviour of your front line people may have a signficantly different impact on customers in different segments.   Take a moment and think how you would like to be treated by a computer salesman if you are old, know nothing about computers, fearful and need lots of hand holding.  Now compare that with a university student who knows exactly what he wants because this is his umpteenth computer.  If the computer salesman followed the same process/script would it land the same – for the old man and the young man?  I think that you will agree that this is highly unlikely.

4.  Balance process with a focus on the customer experience

Too many mystery programmes focus on figuring out how well the customer facing staff did in following a specific process and/or script.   What if the process (and or script) actually gets in the way of delivering a good shopping experience and drives your prospects and customers into your competitors arms?  This fixation on defining quality as following the process is right for a manufacturing environment where you are dealing with inanimate objects.  It is inappropriate when you are dealing with human beings.  When you are dealing with human beings you have to be willing to improvise to deliver the right experience and that often means bending / talking out bits / re-sequencing and otherwise playing around with the process.

For example, a mystery shopping expert shared the story of a German company that had spent a considerable amount of money on training its front line staff in following a specific process.  The mystery shopping programme showed that the majority of front line staff were not following this process.  On the face of it this was bad news and upset some people – those that had paid for the training.  Yet, the mystery shopping also showed that customers were happy to buy and were buying from these front line staff.  Why?  Because the front line staff were doing the right things by the customers – they were doing what mattered to the customers!  And that left the customers with confidence to buy from these front line staff.

5.  Mystery shopping is a team game that involves learning and refinement 

To get the most out of a longitudinal (say a 1 year to 2 year) mystery shopping programme it is essential that all parties to the programme work as a team.  What does that mean?  It means that they all invest passion, time and effort into designing, executing, assessing the results and refining the mystery shopping programme.  The refinement of the programme should be carried out periodically – say every quarter.   So when you are planning the mystery shopping programme then all parties should be figuring out what needs to change based on what has happened in the market place (new entrants, new products, new moves by competitors), the changes that organisation has made (products, processes, touchpoints…) and what the last quarter’s mystery shopping produced.

So if you are the person that is commissioning the mystery shopping then you cannot simply specify it once and throw it over the fence to the mystery shopping team if you want to get the best value out of it.  You have to stay involved throughout the journey and you have to put your time and energy into.  You have to co-create and shape it on an ongoing basis.  And it really helps if you make sure that you take your colleagues – the ones that will have to make changes to their departments – along with you.

Credit:  I wish to thank fellow customer experience and mystery shopping professionals – Jeremy Braune and Chad Robbins – for being the source of this post. 

Why companies are wasting time and money on the Voice of the Customer

I have an issue with the VoC thing

Many large companies are busy tapping into the VoC.  In principle this is a great thing to do because the majority of companies do not have a good enough understanding of their customers.  In practice, I am left feeling that we will see a repeat of the technology centred CRM love fest:  these companies will collectively spend billions, the software companies will get fat and customer satisfaction will stay pretty much the same.   So what is my issue with the VoC thing?

A simplified look at the VoC process and issues

First, let’s take a look at the VoC process:

  • Determine listening posts;
  • Set up listening posts (platforms, tools, people);
  • Collect and consolidate the data;
  • Interpret (make sense of) the data;
  • Sell the interpretation of the data to the various Barons inside the enterprise;
  • Get the Barons to take action in their respective areas; and
  • Monitor/assess the impact on customers (and the business).

If you take a deep look into this you will notice an array of issues:  First, when it comes to surveys how do you know that you are asking the right questions and not ‘leading the witness’?  Second, how do you get access to all the customers that don’t want to complete surveys and make complaints?  Third, how can you be sure that the data you have collected is information and not noise?   Fourth, how do you know that your customer insight team is interpreting the data correctly?  And so on…..

The real issue: VoC can act as a barrier to connecting and empathising with the customers

These issues hide a much more important issue that VoC is a rational solution to an emotional issue.  What do I mean?  The challenge is to get the Baron’s out of their offices and shoes and experience the world  by walking in their customer’s shoes.

Put differently, the challenge is to get the Baron’s to emotionally connect with their customers by experiencing what these customer experience.  And if you accept this  then you will get that VoC programme gives these Baron’s the illusion that they can and do understand customers by reading the reports produced by the customer insight teams.

The problem with this intellectual understanding is that it is purely intellectual.   Intellectual understanding is dangerous because it leaves us thinking we have got it when we have not got it.  What do I mean?

I mean that we have not get it emotionally. That we are not touched, moved, inspired to take action because of having experienced our customer’s lives.  There is a whole body of neuroscience research that shows that the seat of all human action is the emotions and that we can feel/experience what other human beings feel/experience through mirror neurons.  To empathise with our fellow human beings we simply have to connect with them in the context of their day-to-day lives and then let the mirror neurons do the work.

What happens when an industry has no empathy for its customers

Why is that important? Frankly, if the Baron’s cannot or do not empathise with their customers then you end up treating your customers the way that the UK banks treat their customers.  Upon reading this article two paragraphs caught my attention:

“The fine reflects BOS’s serious failure to treat vulnerable customers fairly,” said Tracey McDermott, the acting director of enforcement at the FSA. “The firm’s failure to ensure it had a robust complaint-handling process in place led to a significant number of complaints being rejected when they should have been upheld.”

“We have fallen short of the high standards of service our customers should be able to expect of us and we apologize,” said Ray Milne, the risk director at Bank of Scotland. “We are in the process of contacting affected customers and will pay compensation where it is due.”

It is not hard to treat customers fairly.  The failure to do so by the banks and hide behind platitudes is simply a reflection of the gulf between the Baron’s who make policy and the customers who are impacted by the policy.

How do you cultivate empathy?

Empathy is the route to the human soul and any person who strives to get a meaningful insight into customers lives has to excel at empathy.  So how do you cultivate empathy?  I urge you to watch and listen attentively to the following TED video:   http://www.ted.com/talks/lang/eng/sam_richards_a_radical_experiment_in_empathy.html

Just in case you do not have the time here is a key extract from this presentation:

“Step outside of your tiny little world.

Step inside of the tiny little world of somebody else.

And then do it again, and do it again, and do it again.

And suddenly all of these tiny little worlds they come together in this complex web.

And they build a big complex world.

And suddenly without realizing it

you’re seeing the world differently.

Everything has changed.”

To sum it al up

To exaggerate I would say that an ounce of empathy is worth a mountain of VoC data.  Yet, I do not have fame to my name so I will let one of the worlds renowned business strategists (Kenichi Ohmae) say the final words:

“Personally, I would much rather talk with three homemakers for two hours each on their feelings about, say, washing machines than conduct a 1,000 person survey on the same topic.  I get much better insight and perspective on what customers are really looking for.”

The Coppid Beech Hotel: are you asking the wrong questions?

I wanted and needed a good nights sleep

On Saturday night at around 1:30am my wife and I ended up checking into The Coppid Beech Hotel  – a local hotel a few minutes drive from our home.  By this time we had been to a party, come home and spent some 45 minutes trying to get into our home and failing to do so because our children had accidentally locked us out.  We were both tired, had a big day the next day as friends were coming over for the day, and just wanted to get to our room and sleep soundly.

The hotel failed to deliver the core service: I did not sleep well

As soon as we got into our room I knew that I would not sleep well.  I looked at the bed and could clearly see that the mattress was sagging in the middle and so both my wife and I would end up being crunched together in the middle.  Now that is just fine for my wife as she is a heavy sleeper.  It is not fine by me; I am a light and some would say a fussy sleeper.

Being so tired I did eventually get to sleep only to find myself being woken up at least three times between the hours of 2am and 8am.  I could clearly hear the loud voices in the hallway and once I heard talking in the room next door.  The net result was that the hotel had failed to deliver the core service: a good nights sleep.  So it was with interest that I looked at the “Guest Comment Form” that was prominently displayed on the table in the room.

The hotel is asking for feedback and going about it the wrong way

The form asks about the all kinds of things: “Prior to arrival”; “Front of house”; “Bedroom”; “Breakfast in Rowans”; “Dining within the hotel”; “Lounge bar”; “Room service”; “Waves health & fitness”; and “Brasserie at the Keller”.  In total there are 9 sections and 55 questions: for most of the questions the guest is asked to rate the hotel as either “Excellent”, “Good”, “Average” or “Poor”; and for each of the 9 sections there are only two lines for comments and suggestions.

As I look at this guest feedback form I notice several things.  First and foremost, not one of the questions asks about the core service from the customer perspective.  What is the core service?  I’d argue for a hotel it is providing a good nights sleep.  There is no question about the quality of the bed or the pillows.  And there is no question about noise or the lack of it.

Second, given that there are 55 questions, how many guests are actually going to fill in the form?  I suspect only a small minority.  And if that is indeed the case then how representative is the feedback?

Third, how will this type of questionnaire (ticking the boxes) actually help the hotel to figure out what matters to customers?  And where things rank in terms of importance?  And what areas of the hotel to focus upon?  It is possible that an area is rated as “Average” and that is all that a customer expects in that area because that is an area that simply is not important to the customer.  It is also possible that another area is scored “Good” and yet that is insufficient because it is so essential to the customer that it needs to be “Excellent”.

Lessons for Customer Insight teams

I am sure that those that want to find lessons will be able to find several.  For my part I want to draw attention to how the hotel can get more useful feedback if they simply ask the following questions:

  1. Why did you choose to stay at our hotel?
  2. What were you looking for /expecting to get out of your stay in our hotel?
  3. How well did we deliver on that?
  4. In what areas can we improve?
  5. What do you suggest that we do differently?
  6. Are you willing to write a positive review and recommend us on TripAdvisor and/or our hotel website?

Finally, there is really big difference between designing Voice of the Customer feedback programs to look good (or simply go through the motions) and designing them to get a real insight into what matters to customers and how your organisation is doing in terms of delivering what matters.  I suspect the Coppid Beech Hotel is simply going through the motions of soliciting customer feedback like so many other organisations.  What do you think?