Getting a handle on survey response rates.

It turns out, there are some predictive factors.

sgOne of the nice things about the proliferation on online surveys in recent years is that, over time, we’ve come to understand survey response dynamics much better.

Of course, predicting response rates with flawless precision is impossible due to the individual attributes of each individual survey, the sample composition and so forth.  But thanks to a 2015 compilation of “bottom-line” information by content marketing specialist Andrea Fryrear, the following points are good ones for marketing personnel undertaking market survey work.

Surveys aimed at “internal audiences” outperform external ones.

Targeting an internal audience such as a company’s own employee base is likely going to generate higher response rates (in the neighborhood of 35% to 40%, give or take). For surveys of an external audience, it’s more like 10% or perhaps even lower.

The reason is simple: Surveys aimed at internal audiences are likely very-well targeted, whereas with an external audience, often it’s difficult to reach only the right type of respondents.  At least some of them will turn out to be poor targets.

Additional motivating factors.

Other factors that can influence survey response rates include:

  • Customer loyalty – People who feel a connection with the brand conducting a survey tend to be more likely to participate.
  • Brand recognition – Surveys that focus on well-known brands will typically outperform ones from an unknown source or dealing with unfamiliar brands.
  • Perceived benefit – The “WIIFM” factor.  For example, response rates can soar even higher if the respondent population is motivated by serious incentives.  I recall getting more than a 60% response rate on a mail survey and an external sample because the monetary incentive was a $2 bill.
  • Demographics – The reality is that certain segments of the population are more likely to respond to surveys than others.  Think everything from age and gender to ethnicity and geographic location.
  • Survey distribution – Certain audiences are used to interacting on social media … others online … still others offline.  Chances are, you already know which type of research targets those are within your target markets, and it should influence your choice of survey delivery.

Survey length can make or break your response and completion rates.

To achieve the highest response rates, ideally surveys should take five minutes or less to complete. Ten minutes or less is probably OK, too.  But anything longer than that will likely have deleterious effect on your response rate.

How many questions does this mean? On average, respondents can complete five closed-ended questions in a minutes’ time … but only two open-ended ones.

Survey reminders? Yes.

Particularly with online surveys, it’s a good idea to send reminder notices to those who haven’t completed surveys as you get closer to the cut-off date. Sending two or three reminders is a good rule of thumb … and try sending them at different times of the day or different days of the week to that you can reach as many different prospective respondents as possible.

Learning from the experience of the thousands of surveys administered every month should make it easier for marketers to ensure their next survey will generate successful results instead of flame out.  There’s really no reason for failure considering the wealth of “experiential information” that’s out there.

Where Outside Suppliers of Business Services Fall Down on the Job …

Quirk's Corporate Research ReportQuirk’s Marketing Research Review is a periodical I’ve enjoyed reading for three decades or more.  Unlike the articles that appear in other research-related publications that are more “scholarly” and theoretical,  I find the articles in Quirk’s to be chockfull of insights, while at the same time being “efficiently practical” and easy to digest.

Recently, the magazine published findings from its second annual Quirk’s Corporate Research Report, designed to give corporate researchers an in-depth look into their world.

As part of the research-gathering process for the report, Quirk’s conducted a field survey covering budgets, outsourcing, research techniques in use and under consideration, how research findings are reported inside organizations and, last but not least, the experiences researchers have had when working with outside vendors.

When asked by Quirk’s to state what are the main problem areas when research vendors have come up short on a project, these eight factors were cited by respondents most often:

  • The vendor over-promised and under-delivered: ~56% of respondents mentioned
  • The project was handled by low-level staff: ~51%
  • Vendor failed to take time to understand the client’s business: ~50%
  • Vendor had poor communications: ~39%
  • Vendor failed to take time to understand the project’s needs: ~36%
  • Data integrity issues: ~35%
  • Vendor missed deadlines: ~35%
  • Tools/methodologies that the vendor suggested weren’t right for the project: ~14%

Notice how the most pervasive issues have less to do with the inherent quality of the research product being delivered, and more to do with how the vendor interfaces with and communicates with the companies they support.

The above behaviors represent challenges associated with conducting research projects. But I contend that they apply equally well to providers of other types of business and corporate services, whether they’re ERP or IT projects, website development projects, CRM implementation, SEM/SEO programs, media campaigns, PR initiatives … even IPOs, capital campaigns and the like.

Which of these shortcomings do you find to be most prevalent in your dealings with outside service providers — and what have you done about them? Please share any insights you may have with other readers here.

Misusing Marketing Research: There’s a Saying for That

How to Lie with Statistics by Darrell Huff (1954)
How to Lie with Statistics, Darrell Huff’s business classic, first published in 1954.

Personally, I have respect for marketing research as a discipline.  I think most business decisions are better when they’re backed by the power of marketing research.

Still, I recognize that research can also be used in misleading or otherwise improper ways.

Even worse, research results can be contorted to justify business decisions that have been predetermined.  All too often, “How can we produce results that justify our position?” is the impetus behind a research initiative.

It’s that “dirty little secret” of research that was brought to light decades ago in Darrell Huff’s business classic, How to Lie with Statistics.  First published in 1954, this book been published in countless editions and remains in print even today, 60 years later.

Quirk's Marketing Research ReviewRecently, Dan Quirk of Quirk’s Marketing Research Review, the American research industry’s leading practicum publication, asked subscribers to share their favorite research-related quotes — ones that point to the folly that can be part of the discipline at times.

Some of the reader contributions are great — and they certainly point to the downsides of the research field.  Consider these bon mots:

“Science is built of facts the way a house is built of bricks … but an accumulation of facts is no more science than a pile of bricks is a house.”  (attributed to Henri Poincaré)

“Don’t let the facts get in the way of the truth.”

“When research walks on the field, judgment does not walk off.”  (attributed to Richard Kampe)

“Don’t theorize before one has data:  One begins to twist facts to suit theories, instead of theories to suit facts.”  (attributed to Sir Arthur Conan Doyle)

“Precise forecasts masquerade as accurate ones.”  (attributed to Nate Silver)

“If you torture a data set long enough … it will confess.”

“There are three kinds of lies: lies, damned lies, and statistics.”  (attributed to Mark Twain)

“Statistics can be misleading; the average human has one breast and one testicle.”

“A little nonsense now and then is relished by the wisest men. (attributed to Roald Dahl)

And this one, which ties everything up in a neat little bow:  “No research is better than bad research.”

If you have other memorable research quotes to add to the list, please share them with other readers here.  It’ll be good for a chuckle at least!

What do consumers think of America’s corporations?

Corporate Trust ... Corporate ReputationWith the budget negotiations in full swing – and high dudgeon – on Capital Hill, naturally the public’s critical eye is trained on our political figures. And Congress is most assuredly taking a beating in the political polls, with approval ratings plunging astonishly below the 20% figure.

[Of course, is that really so surprising? After all, Congress is pretty evenly matched between the two parties … so partisans see much to criticize on both sides.]

The focus of attention on Washington has taken the spotlight off of corporate America – at least in terms of media attention. But that doesn’t mean that “John Q. Public” is giving companies much of a break.

I’ve blogged before about corporate reputations — most recently commenting on a field survey conducted early this year by Harris Interactive that measured the appeal of 60 of the “most visible” American corporate brands. That survey showed an uptick in positive opinions about those firms when compared to prior-year results.

But a May 2011 survey by GfK Custom Research North America shows otherwise. The findings from GfK’s online field survey of ~1,000 U.S. consumers include this doozy: Two-thirds of respondents believe that it’s harder today for American companies to be trusted than it was three years ago.

Furthermore, ~55% say it will be harder for companies to gain their trust in the years to come.

What’s bothering people about U.S. corporations? In order of significance, here are the key concerns:

 The perception that CEOs and other senior executives of corporations are overpaid.

 Corruption in senior management circles.

 Companies make up lost earnings at the expense of their customers.

 More products than ever are being manufactured overseas.

Interestingly, there’s less concern about declining product or service quality as a reason for lower levels of trust. And as has been found in other studies, the public’s view of technology companies is somewhat higher than its trust for companies in other industry segments.

But back to the rather grim overall findings … fewer than one in five survey respondents anticipate that corporate corruption will become better over time – a result that’s substantially lower than what was found in similar field research conducted by GfK a few years ago.

This survey underscores the fact that corporate America has a long way to go to change the sharply negative impressions consumers have of the world of business. Clearly, the financial crisis of 2008 continues to extend its long shadow more than two years later.

And it looms over everyone – public and private sector alike.

This helps explain the generally sour mood people are in these days.

Twitter: The “Next Big Thing” in Marketing Research?

By now, it’s obvious that Twitter has become the newest darling of the social marketing world. With somewhere around ten million users today and growing exponentially (there were fewer than one million just a year ago), it’s clear that Twitter has successfully made the leap from novel curiosity to mainstream communications vehicle.

Indeed, Twitter may have worthwhile applications beyond simply the ability for people to update their status information in real time from a mobile phone, computer or online portal. In fact, Silicon Alley Insider recently ran a contest inviting readers to submit their ideas for turning Twitter into a financially viable social network.

The winning entry? An idea from Chicago communications agency Denuo recommending that Twitter charge marketers for access to opted-in users willing to field an occasional research question from brands. Twitter would also charge for dashboard access to the research analytics.

I think this idea has a good deal of merit. Instead of incurring the cost to design and deploy custom research projects, simply tap into Twitter’s existing platform and huge user base to “anonymize” the data and open it up for mining.

Of course, some people voice concern that Twitter will soon be overrun by brand-related messages and advertising. That’s actually begun to happen as certain brands “follow” twitterers ad nauseum — so much it almost constitutes a form of cyber-stalking. But by offering operating an online research panel such as this, Twitter has the potential to deliver scads of valuable, actionable data at the speed of “now.”

Like YouTube, Twitter is actually going to have to figure out a way to make some money for its investors, and soon (imagine that?). So this idea bears watching.