Is an online survey always the “slam-dunk” methodology for field research projects?

srIt’s been quite a long time since I’ve received an invitation to participate in a telephone research survey. And postal mail surveys?  I haven’t been asked to participate in one of those in eons.

It isn’t hard to understand why. Online surveys have become the “default” option for quantitative research.  Not only has digital shaped the way people interact, communicate and shop, online research has plenty of advantages over the more traditional survey methods.  It’s cheaper … it’s faster … it can be quite engaging … and it’s in line with modern behavior.

But being involved in market research in my business, I’m also finding that online surveys have their drawbacks — and it’s becoming more evident with each passing year.

The biggest problem? We’ve seen online survey participation rates crater in recent years as more “stuff” crams people’s inboxes.

When you’re forced to deploy survey invitations to thousands of e-contacts in order to obtain a few hundred usable responses, that’s a symptom of a pretty big problem.

And it leads to another potential concern: Will the respondent pool be representative of the required population?

We’ve known for a long time that certain groups tend to be underrepresented in terms of Internet or digital engagement. And now … to that we can add those people who suffer from “inbox overload.”

In the B-to-B world especially, it isn’t uncommon for a manager to receive 150+ e-mails in a single business day.  Not surprisingly, the great majority of them are trashed without any sort of recipient engagement whatsoever.

… And there goes your research invitation and online survey link.

Online spamming is also contributing to lower online survey participation rates as more people become concerned about the potential dangers of spam mail, thus hesitating to engage with unsolicited e-mails.

On the other side of the coin are the people who have become “professional respondents.” As the financial incentives to participate in surveys have become more lucrative, some people are in the business of survey-taking as revenue-generating proposition.

One wonders how “engaged” these people really are as survey takers — or if they care at all about the topics being studied.  “A mile wide and an inch deep” is more their style.


Ironically, the sum total of these concerns seems to be making phone surveys (the CATI kind — computer-assisted telephone interviewing) the “new-old” alternative to online surveys.

CATI surveys are more expensive and more time-consuming. But in some cases, they may represent the difference between the success or failure of a research endeavor.

What’s more, employing a phone methodology can provide better control over the survey process, along with greater depth and “nuances” regarding the insights gleaned.

Am I going to be recommending CATI telephone survey methodologies to our clients going forward? In most cases, the overall economics — the price-to-value equation — still heavily favors online survey methodologies.

But the gap between them is narrowing … and I can easily envision at least some instances where a return to the classic methodologies of the past may be just what’s needed in the present.