Every day online, a pop-up or email from a major brand or frequently visited site implores me to answer a few questions about myself and them. I (almost) always decline. Not because I care about my privacy but because I don't care to waste my time.
Whenever I've responded, invariably and inevitably, I'm confronted by a boringly designed survey with at least ten to twelve questions that, frankly, are of much greater interest and benefit to them than to me. They want too much for too little. I haven't got the time. Sorry. (Just kidding; not sorry...).
I'm hardly atypical. You won't be surprised to learn that the response rates — and the statistical representativeness of actual responders — tends to be abysmal. People who value their time don't care to waste it on multipart questionnaires.
But there's a simple yet powerful technique for dramatically improving response. I've seen it work magic.
Ask less.
One Fortune 100 firm showed me its comprehensively designed internal "innovation culture" intranet survey containing 25 questions. I (strongly) suggested they ask no more than five. They thought that ridiculous. But they were persuaded to run an A/B experiment where a third of the 10,000+ recipients got the short form. The results were unambiguous. Response rates for the fewer questions were over 11X better than for the full questionnaire. When one factored in the declining quality of "tail end" answers — people clearly just "box ticking" the final five or six answers to be done with it — less proved to be more. (An inadvertently truncated short form with only four questions enjoyed an even higher response rate.)
Of course valuable information and feedback was missing. But design imperative was striking: Getting a robust response for the five most important questions was — on every dimension — far more valuable than puny responses for the top twenty. Yes, there's always another good question worth asking. But the discipline and constraint of honing in on the four or five that matter most helps avoid the intellectual sloth, cognitive laziness and marketing lassitude that longer surveys effectively invite.
Whenever I've worked with design teams collaborating with marketers, I've pushed hard for short vs. longer A/B questionnaire tests. Almost without exception, both the response rates — and responders — for the short form generate the most useful and usable insights. Needless to say, iterating another just-in-time follow-on survey to short forms proves much more practical and palatable than inflicting new questionnaires on long-form participants.
The simplicity of Facebook's "Like" button is no accident. What the smartest innovators and smartest marketers I know have in common is not asking a lot of questions but making sure the most important questions get asked and answered. By definition, the more questions asked, the less important any single question seems. Customers and clients have no interest in viewing your market research surveys as final exam questions. Indeed, what marketers rationalize away as customer-centric curiosity, customers frequently interpret as opportunistic exploitation. Ironic how "proactively eliciting customer insights" can hurt your reputation, isn't it?
I regret that I was never able to persuade one of my credit card clients to test the idea of surprising customers who answered the four question e-survey with the promise of 1,000 reward points if they answered another five questions. I always thought the promise of incentives and recognition would go a long way to improving both the response rates and reputation of virtual market research. But if you can't make your questions sweeter, at least make them shorter. Much shorter. You'll learn a lot more by asking a lot less.
Michael Schrage
No comments:
Post a Comment