In survey research, money talks … but to what degree?

For anyone who has attempted to survey consumers and businesses, it’s pretty universally understood that in order to boost the response rate, you need to give people a “WIIFM” reason to respond.

And that WIIFM incentive is often money. But what kind of monetary incentive works best these days, considering all of the different ways that people are being asked to participate in surveys?

One thing’s for sure: the trend data on response rates isn’t encouraging.  In 1997, the average response rate on telephone surveys was around 36%.  As of 2012, the percentage had nose-dived to just 9%.

It can’t have gotten any better in the five years since.

Recently, the Gallup organization set about to determine response rate dynamics in relationship to the types of monetary incentives offered. To do this, Gallup took the alumni listing from a major American university and deployed online surveys to three target groups of names drawn from it.

Each group was made up of randomly selected names, and each group received the exact same survey. The only difference was in in the incentive offered for recipients to respond to the survey:

  • Group A: 10,000 targeted people received no monetary incentive
  • Group B: 1,000 targeted people were promised a $5 gift card after completing the survey (post-paid incentive)
  • Group C: 1,000 targeted people received a gift card as part of the survey invitation (pre-paid incentive)

The Gallup test revealed that, as expected, offering a monetary incentive had a significant impact on the survey response rate:

  • Group A: 13% response rate
  • Group B: 20% response rate
  • Group C: 19% response rate

But perhaps more interestingly, the results suggest that a pre-paid incentive isn’t quite as strong as offering a monetary reward that comes after filling out the survey. Albeit, the results are very similar, so no definitive conclusions can be drawn.

What is clear, though, is that offering a monetary incentive of some kind does dramatically improve survey results – to the tune of ~50% higher.

Moreover, the Gallup research found no behavioral differences between income groups, suggesting that the “psychology” of being offered a token of appreciation for the survey-taker’s time is something universally appreciated, rather than it being tied to particular respondent characteristics like financial status.

Additional information about the Gallup research can be accessed here.

Is an online survey always the “slam-dunk” methodology for field research projects?

srIt’s been quite a long time since I’ve received an invitation to participate in a telephone research survey. And postal mail surveys?  I haven’t been asked to participate in one of those in eons.

It isn’t hard to understand why. Online surveys have become the “default” option for quantitative research.  Not only has digital shaped the way people interact, communicate and shop, online research has plenty of advantages over the more traditional survey methods.  It’s cheaper … it’s faster … it can be quite engaging … and it’s in line with modern behavior.

But being involved in market research in my business, I’m also finding that online surveys have their drawbacks — and it’s becoming more evident with each passing year.

The biggest problem? We’ve seen online survey participation rates crater in recent years as more “stuff” crams people’s inboxes.

When you’re forced to deploy survey invitations to thousands of e-contacts in order to obtain a few hundred usable responses, that’s a symptom of a pretty big problem.

And it leads to another potential concern: Will the respondent pool be representative of the required population?

We’ve known for a long time that certain groups tend to be underrepresented in terms of Internet or digital engagement. And now … to that we can add those people who suffer from “inbox overload.”

In the B-to-B world especially, it isn’t uncommon for a manager to receive 150+ e-mails in a single business day.  Not surprisingly, the great majority of them are trashed without any sort of recipient engagement whatsoever.

… And there goes your research invitation and online survey link.

Online spamming is also contributing to lower online survey participation rates as more people become concerned about the potential dangers of spam mail, thus hesitating to engage with unsolicited e-mails.

On the other side of the coin are the people who have become “professional respondents.” As the financial incentives to participate in surveys have become more lucrative, some people are in the business of survey-taking as revenue-generating proposition.

One wonders how “engaged” these people really are as survey takers — or if they care at all about the topics being studied.  “A mile wide and an inch deep” is more their style.

pst

Ironically, the sum total of these concerns seems to be making phone surveys (the CATI kind — computer-assisted telephone interviewing) the “new-old” alternative to online surveys.

CATI surveys are more expensive and more time-consuming. But in some cases, they may represent the difference between the success or failure of a research endeavor.

What’s more, employing a phone methodology can provide better control over the survey process, along with greater depth and “nuances” regarding the insights gleaned.

Am I going to be recommending CATI telephone survey methodologies to our clients going forward? In most cases, the overall economics — the price-to-value equation — still heavily favors online survey methodologies.

But the gap between them is narrowing … and I can easily envision at least some instances where a return to the classic methodologies of the past may be just what’s needed in the present.