Too good to be true? Machine learning in survey validation

Many statistics agencies employ surveys to gather data, and since survey data is prone to measurement error, the organizations also work to assure the quality of these data. Processes of quality assurance involve a validation process that is usually manual, rendering the process time consuming, unclear and dependent on individuals. At the same time, learning from previous data and other data sources to predict the likelihood of a correct survey answer is a task that is well suited for machine learning. In this paper, I use the survey from Statistics Norway on enterprises’ research and development efforts to test how machine learning algorithms can improve efficiency, transparency and transferability in the validation process, thus also improving the overall quality of the data.