In addition, it’s important to think about the LOI for people who do not qualify for your study. It is tempting to include all sorts of sizing questions in the screener, but because the LOI of your screened-out participants is also monitored, a long screener (i.e. 5+ minutes) will negatively impact the overall performance of your survey.
If you suspect your LOI is discouraging participants, you can incrementally iterate on your design: consider hiding low-scoring attributes or removing those ‘on-the-fence’questions to quickly shorten the survey.
3) Lower the drop off rate
Aside from a short LOI and making a survey mobile-friendly, survey engagement is key to preventing drop-off. If your participants are giving you feedback that leads you to believe that a survey is too bland, consider “story-fying” your survey. Story-fying means giving your survey more of a storybook feel with an enticing landing page, a pleasant user interface, illustrations, a conversational tone, and a creative and engaging sense of progression.
After the soft launch, review the questions that result in the highest drop-off rate, then make small adjustments to improve the experience. Can you adjust the language to be more friendly and engaging? Can you change the question format to make it more digestible?
4) Improve the incidence rate
After the soft launch, you should have a sense of where the incidence rate stands. Knowing that a low IR could potentially slow traffic to your survey, it’s wise to look at where people are screening out and consider relaxing some of the less-important screener criteria, if you can.
Furthermore, examine the IR by quota group. This will give you an idea of how difficult fieldwork will be near the end, when the most difficult quotas are left open to fill. Will you be looking for a needle in a haystack?
5) Lower the reconciliation rate
Everyone in the industry agrees that we absolutely cannot compromise when it comes to data quality. Yet, what qualifies as “good” data remains subjective, a theoretical gray area. By extension, reconciliation is also rather a subjective exercise.
Generally, to prevent the completes you toss out from impacting your conversion, consider using a “quality control redirect”, which lets your sample supplier know in real-time that it was the participant’s fault for termination.
Incrementally adapt, or perish
Survey metrics are more transparent than ever to both sample suppliers and panelists. Let’s pay attention! These metrics are communicating important details about how participants feel about your survey. We ought to listen to the data, listen to the participants, and course-correct our surveys to optimize EPC.
The opportunity is before us to make adjustments during fieldwork to establish and maintain healthy survey metrics. Designing surveys that are optimized for modern sampling technologies not only benefits researchers, but also improves the participant experience and the participation rate. It’s the little things: small, incremental iterations that can result in huge, positive impacts.