It’s been quite a long time since I’ve received an invitation to participate in a telephone research survey. And postal mail surveys? I haven’t been asked to participate in one of those in eons.
It isn’t hard to understand why. Online surveys have become the “default” option for quantitative research. Not only has digital shaped the way people interact, communicate and shop, online research has plenty of advantages over the more traditional survey methods. It’s cheaper … it’s faster … it can be quite engaging … and it’s in line with modern behavior.
But being involved in market research in my business, I’m also finding that online surveys have their drawbacks — and it’s becoming more evident with each passing year.
The biggest problem? We’ve seen online survey participation rates crater in recent years as more “stuff” crams people’s inboxes.
When you’re forced to deploy survey invitations to thousands of e-contacts in order to obtain a few hundred usable responses, that’s a symptom of a pretty big problem.
And it leads to another potential concern: Will the respondent pool be representative of the required population?
We’ve known for a long time that certain groups tend to be underrepresented in terms of Internet or digital engagement. And now … to that we can add those people who suffer from “inbox overload.”
In the B-to-B world especially, it isn’t uncommon for a manager to receive 150+ e-mails in a single business day. Not surprisingly, the great majority of them are trashed without any sort of recipient engagement whatsoever.
… And there goes your research invitation and online survey link.
Online spamming is also contributing to lower online survey participation rates as more people become concerned about the potential dangers of spam mail, thus hesitating to engage with unsolicited e-mails.
On the other side of the coin are the people who have become “professional respondents.” As the financial incentives to participate in surveys have become more lucrative, some people are in the business of survey-taking as revenue-generating proposition.
One wonders how “engaged” these people really are as survey takers — or if they care at all about the topics being studied. “A mile wide and an inch deep” is more their style.
Ironically, the sum total of these concerns seems to be making phone surveys (the CATI kind — computer-assisted telephone interviewing) the “new-old” alternative to online surveys.
CATI surveys are more expensive and more time-consuming. But in some cases, they may represent the difference between the success or failure of a research endeavor.
What’s more, employing a phone methodology can provide better control over the survey process, along with greater depth and “nuances” regarding the insights gleaned.
Am I going to be recommending CATI telephone survey methodologies to our clients going forward? In most cases, the overall economics — the price-to-value equation — still heavily favors online survey methodologies.
But the gap between them is narrowing … and I can easily envision at least some instances where a return to the classic methodologies of the past may be just what’s needed in the present.
Nones Notes 
Online surveys cannot possibly produce anything approaching a true reflection of reality. This is for a number of reasons. Who wants to tell me their opinion? Most people don’t. They don’t think their opinion matters or don’t want to bother spending time that way or don’t want to cram their feelings into one of three categories, none of which they resonate with.
Honestly, it’s also a somewhat bloodsucking experience because I don’t know what “they” are doing with the time and energy I give.
Next, it is a little abusive as well, because surveys give the participant the illusion of being, yes, “participant”, i.e. part of something in a time where most people suffer from isolation and alienation and jump at any chance to not feel so alone on the inside.
“Somebody wants to know what I’m thinking!! Someone thinks I matter!!”
Dream on, my friend. Somebody wants to tap you for statistics. And they are rigged and we know it.
Take ebay, for example: no matter how dubious a transaction turned out, the user is two or three times discouraged from rating it even neutral. “Really? Are you sure?” and on and on you. This is clearly to prevent anyone from getting the impression dealing on ebay is anything but dizzyingly fantastic.
So, bottom line on that … why would anyone try to take surveys? Because they believe it’s good for their bottom line.
Actually, I sometimes take surveys because I don’t try to be “nice” and tossing off a few lines is no problem for me.
One thing, though, to get two “invitations” to take the same survey per day and then still get some more arm twisting even if I answered the damn thing, that is harrassing, plain and simple. Or when I’ve barely spent 30 seconds on a website and the popup window obscures my view to what I actually want to do, that, too, is harassing. It interferes with my time, my intention. And there are any number of variations of the game. They have become too demanding as well. Once you start it, you are treated like an applicant: “You must answer this question as well!” and “No, that is the wrong type of answer”.
The latest thing is government agencies asking me for my “customer experience”. Come on, I am no customer! I don’t have a choice to go elsewhere!
So, when online surveys “tank” or “flop” or “crater”, that is for many good, actually bad reasons.
So, how do I really feel?