Assessing Alternative Precision Measures when Adjusting for Conditional Bias at the Subnational Level through Calibration Weighting

Abstract

Calibration weighting improves inference by adjusting for observed differences between the realized sample and the population. Unfortunately, a commonly-used linearization-based variance estimator often does not account for the increased efficiency provided by the calibration process. As a result, precision estimates based on calibrated weights can be artificially high. Using a relatively new alternative linearization-based variance estimator allows analysts to utilize calibration-weighting techniques while producing more accurate precision estimates. We use calibration weighting to produce more reliable subnational estimates and assess the differences in point estimates resulting from these weight adjustments in the National Crime Victimization Survey, a nationally representative survey designed to calculate victimization rates solely at the national level. We then assess the estimated precision of these point estimates using a conventional linearization-based variance estimator and the alternative estimator. We find that the calibration adjustments mostly reduced the standard errors in subnational estimates but to successfully measure the reduction required using the alternative variance estimator.