The potential of ‘Big Data’ to enable faster, cheaper production of official statistics has for years been a hot topic among statistical offices.
UNECE has been at the forefront in guiding countries through the many challenges—data quality, processing techniques, analytical capacity, access and reliability of sources. Back in 2013 the Conference of European Statisticians conducted an in-depth review of the role and potential of Big Data for official statistics. A position paper asking ‘what does big data mean for official statistics?’, prepared by the UNECE-led High-Level Group for the Modernization of Official Statistics, is widely used by NSOs to navigate through the enormous but risky possibilities.
Until now, however, national statistical offices have confined their use of Big Data sources to modelling, benchmarking, providing sampling frames and comparing findings. There has been no shortage of pilot studies and experiments, but actually using Big Data as a source to produce official statistics remains a futuristic prospect for many.
The United States Census Bureau broke new ground at the 2019 UNECE-Eurostat Work Session on Migration Statistics, which took place in Geneva, presenting results of their work to combine Big Data and survey-based estimates to produce official migration statistics. Used to producing migration statistics from information gathered in surveys, the Census Bureau’s hand was forced by the chaotic aftermath of hurricane Maria in 2018, which led to a massive drop in data quality from the surveys, obliging them to look elsewhere for the data they needed. Information on passenger flights between the mainland United States and Puerto Rico, produced by the US Bureau of Transportation Statistics, is combined with data from household surveys to produce the official national estimates of migration between these two areas.
The 2019 UNECE Guidance on Data Integration for Measuring Migration, whose recommendations are taken into account in the USA’s methodology, helps countries not only to work with Big Data but to combine multiple sources such as population registers, immigration and naturalization records and survey data to improve statistics on immigration and emigration. The UNECE-Eurostat Work Session saw examples from Turkey, the United Kingdom, Colombia, Chile and Hungary, among others.
In Colombia, integration of data from two different statistical registers—one on the resident population and one on international migrants—is allowing the statistical office for the first time to map the geographical distribution of emigration abroad from different parts of Colombia. In Hungary, work to reconcile administrative and survey data is permitting measurement of circular migration, where people repeatedly cross a national border for a medium or long-term stay. Information from social insurance records, address registers and a national microcensus are combined to produce detailed estimates of numbers of circular migrants and the trends in these numbers, with information on their destination, educational level, family situation and other characteristics important for informed policy-making on migration.