To uphold the quality of your CAPI and Online research, you need to ensure survey results aren’t contaminated by ‘bad’ interviews. This calls for identifying and disqualifying responses which show evidence of being falsified.
Keeping it real
Simple pointers to unreal answers and/or respondents include short interview duration, short time spent on each question, short or meaningless open-ended answers and lack of cohesion between answers to related questions. Another giveaway in CAPI fieldwork could be when a particular interviewer’s work is completed via the shortest-possible route.
Methods for identifying these pointers in Nfield CAPI interviews are explained in CAPI Quality Control: Audit Trail helps you eliminate falsified fieldwork and Nfield’s Quality Control Options for Rock-Solid CAPI Data.
In addition to identifying suspicious interviews from interview completion data, you can also adopt a practice of contacting certain respondents to verify their input.
Setting your threshold
Every company or survey project has its own criteria for whether an interview should be classed as ‘bad’ and therefore disqualified from results.
Samples from panel providers are generally trustworthy, as panel providers deploy their own mechanisms for maintaining an honest and active respondent pool. But if your survey is conducted outside the confines of a panel, there is a higher chance of contribution from respondents who are just making mischief, possibly to get promised rewards, whose answers are not genuine.
To determine where to set your tolerance, you need to get to know your own typical response patterns for the criteria described above (under “Keeping it Real). When answers fall outside of these, it’s time to raise a red flag.
Throwing out bad apples to purify results
Just as bad apples need to be disposed of to avoid the whole crop becoming unusable, ‘bad’ interviews need to be disqualified from survey results in order to come away with genuinely useful insights.
This is easily done in via Nfield Manager for Nfield Online and Nfield CAPI.
Once you reject an interview, it is automatically removed from all counts related to successful completes and corresponding quota cells. So disqualification of ‘bad’ interviews may mean continuing your survey a bit longer to reach all the required targets.
Disqualification process
In Nfield Manager, go to the Quality Control tab. Here you’ll find a Qualification Control overview, as well as each individual interview’s validation status.
To disqualify an untrusted interview from survey results, simply select the interview ID and click Reject.
As an alternative to rejection, you can carry out positive selection by approving individual interviews. And if you aren’t sure whether to approve or reject an interview, you can un-verify it, indicating a requirement to conduct further checking.
Clicking “Reset” will restore all validation states to “Not checked”. This will include any interviews which had been set to “Rejected”, thereby re-including them in corresponding quota counts as successful completes.
Automated disqualification
If you want to automate identification and disqualification of untrusted interviews, this can be achieved by integrating appropriate tools with Nfield via an API. See API – What Researchers Need to Know to learn more about Nfield API.
Downloading Rejected Answers
You can download a record of all rejected interviews in the same way as you do for other interview records.
If you have any questions or comments, please do not hesitate to contact us.
