Notice
There is a newer version of this document available.
It can be found at https://www.ssoar.info/ssoar/handle/document/83635.2
Download full text
(983.2Kb)
Citation Suggestion
Please use the following Persistent Identifier (PID) to cite this document:
https://nbn-resolving.org/urn:nbn:de:0168-ssoar-83635-7
Exports for your reference manager
An experimental test of the effectiveness of cognitive interviewing in pretesting questionnaires
[journal article]
Abstract Pretesting survey questions via cognitive interviewing is based on the assumptions that the problems identified by the method truly exist in a later survey and that question revisions based on cognitive interviewing findings produce higher-quality data than the original questions. In this study, we ... view more
Pretesting survey questions via cognitive interviewing is based on the assumptions that the problems identified by the method truly exist in a later survey and that question revisions based on cognitive interviewing findings produce higher-quality data than the original questions. In this study, we empirically tested these assumptions in a web survey experiment (n = 2,200). Respondents received one of two versions of a question on self-reported financial knowledge: either the original draft version, which was pretested in ten cognitive interviews, or a revised version, which was modified based on the results of the cognitive interviews. We examined whether the cognitive interviewing findings predicted problems encountered in the web survey and whether the revised question version was associated with higher content-related and criterion-related validity than the draft version. The results show that cognitive interviewing is effective in identifying real question problems, but not necessarily in fixing survey questions and improving data quality. Overall, our findings point to the importance of using iterative pretesting designs, that is, carrying out multiple rounds of cognitive interviews and also testing the revisions to ensure that they are indeed of higher quality than the draft questions.... view less
Keywords
online survey; survey; interview; pretest; questionnaire; data quality; validity; survey research; experiment
Classification
Methods and Techniques of Data Collection and Data Analysis, Statistical Methods, Computer Methods
Free Keywords
Cognitive interviewing; Criterion-related validity; Pretesting
Document language
English
Publication Year
2022
Page/Pages
19 p.
Journal
Quality & Quantity (2022)
DOI
https://doi.org/10.1007/s11135-022-01489-4
ISSN
1573-7845
Status
Published Version; peer reviewed
Licence
Creative Commons - Attribution 4.0
FundingGefördert durch die Deutsche Forschungsgemeinschaft (DFG) - Projektnummer 491156185 / Funded by the German Research Foundation (DFG) - Project number 491156185