Privacy concerns in responses to sensitive questions: a survey experiment on the influence of numeric codes on unit nonresponse, item nonresponse, and misreporting

Abstract

"Paper-and-pencil surveys are a widely used method for gaining data. Numeric codes printed on the questionnaire are often a prerequisite for the use of scan software, which, in turn, permits a fast and efficient entering of the data from such surveys. However, printed numbers used for optical mark recognition on a questionnaire can provoke concerns about anonymity that may lead to unit nonresponse, item nonresponse, and misreporting. To test this, we conducted an experiment in a mail survey on group-focused enmity, printing a scanner code on half of the questionnaires. Our results show no significant deviation concerning unit nonresponse. We find a higher item nonresponse and misreporting bias towards socially desirable answers in sensitive questions if the questionnaire is marked with a code. The influence of biased responses on regression results is minor. If the numeric code is brought to the respondents’ attention in the cover letter, regression coefficients might be affected. Therefore we conclude that researchers should trade off these small biases against the usefulness of the code. From a methodological perspective, we recommend not to make a statement concerning the numeric code in the cover letter. Our results are of relevance for researchers conducting paper-and-pencil surveys as well as for those analyzing data sets from these surveys. While this article analyzes biases caused by scanner codes, the results are potentially transferable to printed identification numbers used in panel studies, in survey experiments, or to match paradata or context data." (author's abstract)