Misusing Marketing Research: There’s a Saying for That

How to Lie with Statistics by Darrell Huff (1954) — *How to Lie with Statistics*, Darrell Huff’s business classic, first published in 1954.

Personally, I have respect for marketing research as a discipline. I think most business decisions are better when they’re backed by the power of marketing research.

Still, I recognize that research can also be used in misleading or otherwise improper ways.

Even worse, research results can be contorted to justify business decisions that have been predetermined. All too often, “How can we produce results that justify our position?” is the impetus behind a research initiative.

It’s that “dirty little secret” of research that was brought to light decades ago in Darrell Huff’s business classic, How to Lie with Statistics. First published in 1954, this book been published in countless editions and remains in print even today, 60 years later.

Recently, Dan Quirk of Quirk’s Marketing Research Review, the American research industry’s leading practicum publication, asked subscribers to share their favorite research-related quotes — ones that point to the folly that can be part of the discipline at times.

Some of the reader contributions are great — and they certainly point to the downsides of the research field. Consider these bon mots:

“Science is built of facts the way a house is built of bricks … but an accumulation of facts is no more science than a pile of bricks is a house.” (attributed to Henri Poincaré)

“Don’t let the facts get in the way of the truth.”

“When research walks on the field, judgment does not walk off.” (attributed to Richard Kampe)

“Don’t theorize before one has data: One begins to twist facts to suit theories, instead of theories to suit facts.” (attributed to Sir Arthur Conan Doyle)

“Precise forecasts masquerade as accurate ones.” (attributed to Nate Silver)

“If you torture a data set long enough … it will confess.”

“There are three kinds of lies: lies, damned lies, and statistics.” (attributed to Mark Twain)

“Statistics can be misleading; the average human has one breast and one testicle.”

“A little nonsense now and then is relished by the wisest men.” (attributed to Roald Dahl)

And this one, which ties everything up in a neat little bow: “No research is better than bad research.”

If you have other memorable research quotes to add to the list, please share them with other readers here. It’ll be good for a chuckle at least!

One thought on “Misusing Marketing Research: There’s a Saying for That”

I’m not sure about any memorable research quotes I know. But a few years back, there was a VERY interesting “research book”: David Freedman’s Wrong: Why Experts Keep Failing Us—and How to Know When Not to Trust Them.

An excerpt from the introduction to an interview with the author in Time magazine: “As much as 90% of physicians’ medical knowledge has been found to be substantially or completely wrong. In fact, there is a 1 in 12 chance that a doctor’s diagnosis will be so wrong that it causes the patient significant harm. And it’s not just medicine. Economists have found that all studies published in economics journals are likely to be wrong. Professionally prepared tax returns are more likely to contain significant errors than self-prepared returns. Half of all newspaper articles contain at least one factual error. So why, then, do we blindly follow experts?”

Freedman goes on to speak about all the ways “experts” routinely get it wrong — and worse, intentionally cook numbers to support their own agendas. And so “yes,” research can be a powerful tool. It’s just knowing which numbers to believe.

If doctors — highly trained scientists — can be misinformed 90% of the time; and economists — “expert” statisticians — can be wrong almost all of the time, then the rest of us should approach marketing “research” with some circumspection, don’t you think?

Wes Green says:

November 20, 2014 at 4:41 pm

I’m not sure about any memorable research quotes I know. But a few years back, there was a VERY interesting “research book”: David Freedman’s Wrong: Why Experts Keep Failing Us—and How to Know When Not to Trust Them.

An excerpt from the introduction to an interview with the author in Time magazine: “As much as 90% of physicians’ medical knowledge has been found to be substantially or completely wrong. In fact, there is a 1 in 12 chance that a doctor’s diagnosis will be so wrong that it causes the patient significant harm. And it’s not just medicine. Economists have found that all studies published in economics journals are likely to be wrong. Professionally prepared tax returns are more likely to contain significant errors than self-prepared returns. Half of all newspaper articles contain at least one factual error. So why, then, do we blindly follow experts?”

Freedman goes on to speak about all the ways “experts” routinely get it wrong — and worse, intentionally cook numbers to support their own agendas. And so “yes,” research can be a powerful tool. It’s just knowing which numbers to believe.

If doctors — highly trained scientists — can be misinformed 90% of the time; and economists — “expert” statisticians — can be wrong almost all of the time, then the rest of us should approach marketing “research” with some circumspection, don’t you think?