As we learn more about how the mind actually works, we should update how we ask questions in surveys.
Because here’s the uncomfortable truth:
People can give you an answer… without it being the answer.
Some questions sound perfectly reasonable, but they reliably produce data that is misleading, incomplete, or flat-out wrong.
Here are three common ones you should avoid.
1. “Where did you first learn about this product?”
At first glance, this seems like a basic marketing question.
In reality, it’s a memory trap.
As reported in Customer Experience Management by Tavsan and Edrem:
“Previous studies show that individuals may remember specific information but may not remember where they learned it.”
That’s the key issue: people may remember the product, but not the source. If you’ve ever run this question for long, you’ve seen the weird results:
- “Advertising” gets checked even when no advertising was used
- People credit campaigns from years ago
- Brand familiarity gets mistaken for first exposure
In short:
Advertising is usually overstated.
Memory is messy, and surveys don’t clean it up.
2. “Did the advertising encourage you to purchase?”
This question runs into a different problem: pride.
Few people want to admit that an ad persuaded them. Most of us like to believe we are rational decision-makers, not billboard-following sheep. But test vs. control studies consistently show advertising works (assuming it’s decent creative and run with enough weight).
So what happens?
Advertising impact gets understated.
And neuroscience gives us a clue why.
Our subconscious brain processes 11 billion bits of information per second. Our conscious brain handles about 40.
So even when people think they’re ignoring ads… their brain is not.
As Melina Palmer explains:
“. . .much of what is happening and influencing your choices is below your level of conscious understanding.”
— What Your Customer Wants and Can’t Tell You
Erik DuPlessis adds:
“Consumers absorb something from the advertisement… perhaps without consciously thinking much… then at the time they are making purchase decisions they ‘use’ that impression to influence their choice.”
— The Advertised Mind, Erik Du Plessis
So yes, advertising affects choices.
People just don’t experience it that way.
3. “Why did you buy that?”
This is the classic survey question. And it might be the most dangerous. Because people don’t make decisions the way surveys assume.
We decide emotionally and subconsciously…
…and then explain it rationally afterward.
So when you ask “why,” you often get the story, not the cause.
Palmer puts it bluntly:
“Unfortunately, these two systems of the brain don’t speak the same language… People don’t know what they will do. And even worse, they can’t tell you after the fact why they did something…”
— What Your Customer Wants and Can’t Tell You
So respondents aren’t lying. Their brains are just improvising.
The survey answer is often the justification, not the driver.
It may be useful to know this, but don’t assume it is the primary motivation.
So, What Should You Do Instead?
The good news: you’re not stuck.
There are better ways to uncover true drivers of choice:
- Projective qualitative exercises
(“Why do you think your neighbor bought it?”) - Driver modeling and analytics
Quantitative methods that reveal what actually predicts behavior - Choice exercises if possible
Where respondents make choices as they do in the real world.
- Behavior-based questions
Ask what people did, not what they think caused it
The Real Risk of Bad Questions
There are two big dangers in asking these types of questions:
First, when questionable results come back
(e.g., “They saw it in advertising”… when there was no advertising),
you lose credibility internally.
But the bigger risk is worse:
Management may make major communication and investment decisions based on faulty data…
…and end up funding the wrong message, the wrong channel, or the wrong strategy.
Survey questions aren’t harmless.
Bad ones can be expensive.
