Inferences based on Probability Sampling or Nonprobability Sampling: Are They Nothing but a Question of Models?

Abstract

The inferential quality of an available data set, be it from a probability sample or a nonprobability sample, is discussed under the standard of the representativeness of a sample with regard to interesting characteristics, which implicitly includes the consideration of the total survey error. The paper focuses on the assumptions that are made when calculating an estimator of a certain population characteristic using a specific sampling method, and on the model-based repair methods, which can be applied in the case of deviations from these assumptions. The different implicit assumptions regarding operationalization, frame, selection method, nonresponse, measurement, and data processing are considered exemplarily for the Horvitz-Thompson estimator of a population total. In particular, the remarkable effect of a deviation from the assumption concerning the selection method is discussed. It is shown that there are far more unverifiable, disputable models addressing the different implicit assumptions needed in the nonprobability approach to sampling, including big data. Moreover, the definition of the informative samples with respect to the expressed survey purpose is presented, which complements the definition of the representativeness of samples in the practice of survey sampling. Finally, an answer to the question in the title of this study is given and detailed reports regarding the applied survey design are recommended.