Each year nearly $400 billion is spent on traditional market research, with the majority spent on interviews, focus groups, and on-line surveys. While technology has had a significant impact on the way we recruit and field research (imagine the scale, reach, and speed that SurveyMonkey and Qualtrics have provided to the world), it has not improved its effectiveness. Traditional market research has only a 24% link to in-market results. The limitations of research are all around us, from product failures (Quibi anyone?) to box-office busts (Mulan) and poor election predictions (Hillary Clinton).
Let’s step back from our Survey Monkey mind (no pun intended) and look at traditional research with a new lens:
1. It measures only conscious data when 90% of decision-making is subconscious.
David Ogilvy said consumers, “Don’t say what they think, and they don’t do what they say.” Why do you think that happens? Philosophers and psychologists in the west and spiritual leaders in the east have argued this for centuries. Now neuroscience has proven that most of our decisions are made in the subconscious – meaning we are blind to our own decision-making process.
But we like to feel that we are in control, so we try to rationally explain our thinking – especially when someone is paying us for an answer.
2. It relies on self-reported data.
Traditional research relies on consumers’ responses to questions, and that can be highly misleading. It assumes that the consumer understands the questions and interprets them the same way. A quick review of surveys suggests that many include questions that are leading, confusing, or unclear. Even questions that appear to be straightforward, such as rating a service on a scale of 1-5, often leave consumers debating if a 3 is a satisfactory score or a signal that improvement is needed. Traditional research assumes consumers answer with 100% honesty, which is sometimes not the case. People are influenced by those around them – they tend to agree with the greater group. This phenomenon is referred to as social desirability bias. There is also confirmation bias, where consumers anticipate the goal of the research and answer based on what they believe is the ‘correct’ response.
3. It is even less suited for breakthrough ideas.
It is amazing how many marketers quote Henry Ford (“If I had asked people what they wanted, they would have said faster horses”) and then go back to asking people questions. Disruptive innovation often fails in research because consumers are reacting to an idea in isolation – with one exposure. How would the iPhone have fared on IPSOS or Nielsen tests? These tools don’t take into account the magic of Steve Jobs’ launches, his excellent marketing, let alone the brilliance of the product. We discuss Steve as a true innovator – and then we go back to surveys. Great. RIP Steve.
4. It collects data post-exposure.
In traditional market research, data is collected after exposure to the stimuli. This approach disregards the hundreds of thoughts one might have throughout the viewing experience. By gathering data at the end, the best possible outcome is detailed recollection – which is short-term memory. Explicit memory, which is conscious long-term memory, accounts for only 10% of long-term memories. Thoughts and feelings throughout the exposure are not captured. The researchers’ understanding of an ad, for example, is based on consumers’ recall of which parts of the ad they found positive or negative. While this is interesting, it does not indicate if the ad is stored in long-term memory for future action. (some researchers will highlight the dials or other means of second-by-second data – we’ll address that in our next post.)
5. It is often conducted out of context.
Traditional research does not typically match the research vehicle and the context for the material being tested. Research is designed to be convenient and is often completed on phones and tablets. While this is appealing to participants, it is not ideal for testing stimulus intended to be seen on a big screen or in-store.
Why test a Superbowl commercial on a phone if it is designed to be viewed on a big screen? The research will yield different results.
6. It gives no optimization recommendations.
Since the research does not measure second-by-second response, there is not enough information to understand the needed improvements. This situation leaves few good options. Scrap millions of production dollars? Listen to a vague and random guess on how to improve the creative that has tested poorly (when it might be good) based on this self-reported, out-of-context data that at best correlates to in-market effectiveness at about 20%?
For too long we have made excuses: a) this is how we always do it, b) we have ten-year benchmarks, c) it gives us one number that our executives can understand d) no one got fired for trusting a large survey, e) they’re an approved vendor and are in the system, or f) the alternative looks impressive, but it’s too expensive.
We can do better.
Millions of dollars are at stake, the benchmarks are irrelevant, having one clear answer doesn’t make it right, and we have the right, power, and responsibility to contribute our best work to this industry.
Neuro-Insight has created unique, patented technology that helps marketers solve the most vexing problems by tapping into the deep and often overlooked reservoir of subconscious human motivations, emotions, and intuition that influence behavior. Our approach enables companies to understand consumers’ subconscious responses to stimuli quickly and easily. Our studies are designed to consider context and provide a second-by-second insight, which is critical for optimization. We have the most accurate measurement and the largest database of long-term memory, with an 86% correlation to in-market sales. We have addressed the shortcomings of traditional research.
Neuro-Insight is the world’s preeminent neuro-marketing firm. We would love to help you uncover the ultimate human truth.