"Examples of our capacity for misplaced beliefs are not hard to find. If something seems plausible, impresses us, fits with what we'd like to think, or has been sold to us persuasively, we are willing to treat it as a truth," writes Philip Graves in Consumer.ology: The Market Research Myth, the Truth About Consumers, and the Psychology of Shopping. "So it is with market research. On the occasions when a research report's findings coincide with a positive outcome, it is taken as proof that the process was worthwhile and contributed positively to the course that was taken."
In this excerpt from Consumer.ology, Graves explains that it's not what consumers say, but what they do that really matters:
Understanding the Unconscious Mind: Why we buy what we do but can't explain it
The story of New Coke has gone down in marketing folklore. In the early 1980s Coca-Cola's main rival, Pepsi, was making significant inroads into Coke's market share. One strand of its attack was with the Pepsi Challenge, in which Pepsi conducted thousands of blind taste tests and publicized the fact that more people liked its product. Despite questioning the results, Coke's own research got the same result: 57% of people asked to taste both products preferred Pepsi. The Coca-Cola Company undertook extensive further research, which led to the creation of a new, sweeter formula for Coke. This recipe did the trick and turned around the taste test results: now Coke was beating Pepsi by around 7 percentage points. At that time, and given the value of the market the two were competing for, the $4 million spent to research and develop the new formula must have seemed like money well spent.
It's well known that the resulting launch of New Coke as a replacement to the original formula was something short of a complete success. It triggered a large public backlash and the company was inundated with complaints. Within just three months the product had been withdrawn from sale and the original formulation was back on the shelves.
Much has been written about why the market research was misleading and most of the arguments put forward have merit. There's a world of difference between sipping a drink and consuming an entire can of it: the initially sweet hit can become overpowering in much the same way that the first chocolate from the box is heavenly, but the tenth consumed in the same sitting can leave you feeling somewhat nauseous. Separating the product from the packaging also removes the brand from the equation, with the implication that marketing Coke is simply a way of reminding people that your brown fizzy drink exists and can be bought wherever you see the distinctive red-and-white logo.
However, amid all the analysis and explanations, no one to my know ledge has reached the ultimate conclusion to be drawn from the New Coke fiasco: it isn't just that Coke's extensive market research on the new recipe was wrong, it is that no such research can be right, other than by chance. Yes, there were technical flaws in the research process, but that doesn't mean that the theorized remedies would have produced a more accurate answer. Giving people a complete branded can to drink or a crate of them to consume over a month at home would probably have produced a different answer, but not necessarily one that would then have been borne out by reality.
Nevertheless, the belief remains: "Of course you can find out what people think by asking them, you just have to ask them the right questions in the right way." The market research industry has gone on unabashed; companies still believe that reassurance can be found in the exchange of corporate question for consumer answer and politicians that public opinion can be gauged from a poll or focus group. No verifiable alternative has emerged for product development, because the crux of the matter is far more challenging to a business world and research industry that rely heavily on the reassurance that market research provides: consumer behavior is a by-product of the unconscious mind, whereas research is inherently a conscious process.
New Coke highlights just how little companies understand about the role of the unconscious mind (little has changed in the intervening decades). Most organizations don't understand consumer behavior or how and why their marketing works (or doesn't work).
The unconscious mind is the real driver of consumer behavior. Understanding consumers is largely a matter of understanding how the unconscious mind operates; the first obstacle to this is recognizing how we frequently react without conscious awareness. As long as we protect the illusion that we ourselves are primarily conscious agents, we pander to the belief that we can ask people what they think and trust what we hear in response. After all, we like to tell ourselves we know why we do what we do, so everyone else must be capable of doing the same, mustn't they?
 Gladwell, M. (2006) Blink: The Power of Thinking without Thinking, Harmondsworth: Penguin.
 Pendergrast, M. (2000) For God, Country and Coca-Cola: The Unauthorized History of the World's Most Popular Soft Drink and the Company that Makes It, New York: Basic Books.
+ + + + + + + + +
About the Author: Philip Graves is a consumer behavior consultant, author, and speaker.
Reprinted with permission of the publisher, from Consumer.ology by Philip Graves. Nicholas Brealey Publishing, London/Boston. Copyright Philip Graves 2010.