Results of an international survey conducted by the UNECE Conference of European Statisticians (CES) to explore the use of generative AI within statistical organizations reveal the profound impacts that AI is having on the field of official statistics.
The survey, carried out in June 2024, gathered insights from 41 national and international statistical bodies, shedding light on the current applications, policies, and challenges related to generative AI in statistics.
The survey revealed that generative AI is making a significant impact in specific areas of statistical operations. Notably, half of the participating organizations believe that AI will have a "highly impactful" role in coding and IT development, as well as in dissemination and communication efforts. Currently, 71 per cent of organizations are using generative AI for coding, while 46 per cent employ it for data processing and generating textual communication materials.
Generative AI is widely expected to have a high impact on statistical organizations, particularly on efficiency and productivity. Responses indicate that over the next two to three years, IT-related areas, including coding, will be the most impacted, closely followed by dissemination and communication. Other areas, such as data collection, processing, analysis, and administrative functions, are also expected to be moderately impacted. Statistical organizations anticipate that efficiency and productivity will be most affected, followed by enhanced service delivery to users and fostering creativity.
Despite the lack of explicit policies on AI use in more than half of the organizations, many are proactively moving forward. Approximately two-thirds of respondents are either developing or planning to develop in-house AI solutions, highlighting a strong trend towards integrating AI into their workflows. However, the majority of organizations do not yet have formal policies in place for governing the use of generative AI. Only 30 percent of organizations have established guidelines, while the rest are still in the process of developing them. Common themes in these guidelines include data security, transparency, and ethical use, focusing on minimizing risks and promoting accountability.
Policy Development and key concerns
When it comes to policy and governance, the survey found that 30 per cent of organizations have already established guidelines for the use of generative AI, with another 17 per cent actively developing such policies. The primary concerns surrounding AI adoption are security and accuracy, with 66 per cent and 61 per cent of organizations, respectively, stating that these issues are of major concern. Further, many organizations express concerns about risks related to data security, privacy, and accuracy. Ethical considerations, copyright, and legal issues are also of substantial importance to the majority of respondents. Additionally, the potential for misuse and the lack of reproducibility are significant challenges.
To mitigate these risks, statistical offices are taking various approaches, including developing internal policies, training staff, and using self-hosted solutions to minimize the sharing of confidential information. Yet, investments in infrastructure, such as hardware and data centers, remain a key limitation for certain countries. Some organizations also prefer using open-source software to reduce dependency on external providers. There is a clear need for internal capacity building to raise awareness about the risks of using generative AI with sensitive information. Most respondents consider the availability of staff with appropriate skills and the capability to use generative AI at the organizational level a limiting factor.
Addressing challenges
The survey results paint a comprehensive picture of the challenges and opportunities associated with generative AI in statistical organizations. Key concerns such as security, skills availability, and infrastructure are being tackled through a combination of training, policy development, and the creation of in-house AI solutions.
The findings also emphasize the importance of international cooperation, including for the development of common standards and guidelines. Sharing knowledge, best practices, training resources and experiences were also highlighted as key.
UNECE supports statistical organizations to advance the use of generative AI
Recognizing the transformative potential of generative AI, particularly Large Language Models (LLMs), the UNECE High-Level Group for the Modernisation of Official Statistics (HLG-MOS) has been at the forefront of exploring the use of these technologies. A 2023 white paper highlighted the need for continuous exploration and knowledge sharing, detailing the transformative potential, capabilities, challenges, and impacts of LLMs on statistical operations. Earlier this year, the Group launched a dedicated project on generative AI to explore governance and implementation aspects, informed by experiences from statistical organizations worldwide.
Full survey results can be found here: https://unece.org/sites/default/files/2024-08/AI%20Survey%20results.pdf