UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE - PROJECT E264
ENHANCING COHERENCE AND INTEGRATION OF ECONOMIC AND SOCIAL STATISTICS IN SUPPORT OF
THE IMPLEMENTATION OF 2008 SYSTEM OF NATIONAL ACCOUNTS (SNA)
2017-2020 (EXTENSION FOR 2021-2023)
EVALUATION REPORT
Period of evaluation: July – October 2023
Evaluator: Constance Graziadei
Evaluation commissioned by United Nations Economic Commission for Europe
3
EVALUATION REPORT
The Evaluation report compiled by Constance Graziadei evaluates the UNECE project “Enhancing
Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008
System of National Accounts” implemented from 2017-2023.
The views and interpretations are those of the author and do not necessarily represent those of the
United Nations Economic Commission for Europe (UNECE).
ACKNOWLEDGEMENTS
Ms. Graziadei would like to thank all who contributed to successful completion of this evaluation: to the
UNECE Statistical Division who provided access to all the project documentation and support to the
process, to all who responded to the two online surveys and to those who generously gave their time to
be interviewed. Your contributions are truly appreciated.
4
TABLE OF CONTENTS
ABBREVIATIONS AND ACRONYMS ........................................................................................................................ 5
EXECUTIVE SUMMARY .......................................................................................................................................... 6
1.0 INTRODUCTION ......................................................................................................................................... 9
1.1 PURPOSE OF THE EVALUATION .................................................................................................................. 10
1.2 OBJECTIVES OF THE EVALUATION .............................................................................................................. 10
1.3 SCOPE OF THE EVALUATION ....................................................................................................................... 10
1.4 EVALUATION METHODOLOGY ................................................................................................................... 11
2.0 FINDINGS ................................................................................................................................................ 12
2.1 RELEVANCE ................................................................................................................................................. 12
Figure 1: Evaluation of the UNECE Project on Economic and Social Statistics ................................................. 13
2.2 EFFECTIVENESS ........................................................................................................................................... 15
2.3 EFFICIENCY .................................................................................................................................................. 17
Table 1: Budget Allocation ................................................................................................................................. 17
Table 2: Initial Project Allocation of Funds and expenditures per module (2017-2020) .................................. 18
Table 3: Budget Allocation for 2021-2023 ......................................................................................................... 18
2.4 SUSTAINABILITY ......................................................................................................................................... 19
Figure 2: Evaluation Survey – Evaluation of the UNECE Project on Economic and Social Statistics ................ 20
2.5 GENDER AND HUMAN RIGHTS CONSIDERATIONS ..................................................................................... 21
3.0 CONCLUSIONS AND RECOMMENDATIONS .............................................................................................. 22
3.1 CONCLUSIONS ............................................................................................................................................ 22
3.2 RECOMMENDATIONS ................................................................................................................................. 23
4.0 ANNEXES ................................................................................................................................................. 24
4.1 TERMS OF REFERENCE FOR THE EVALUATION ........................................................................................... 24
4.2 ONLINE SURVEY QUESTIONNAIRE – BENEFICIARY COUNTRIES ................................................................. 28
4.3 ONLINE SURVEY RESULTS – BENEFICIARY COUNTRIES .............................................................................. 30
4.4 ONLINE SURVEY QUESTIONNAIRE – STAKEHOLDERS/PARTNERS ............................................................. 36
4.5 INTERVIEW GUIDE – KEY STAKEHOLDERS .................................................................................................. 38
4.6 LIST OF PROJECT ACTIVITIES BY MODULE .................................................................................................. 39
4.7 STAKEHOLDERS INVITED FOR IN-DEPTH INTERVIEW ................................................................................. 41
4.8 LIST OF DOCUMENTS REVIEWED ................................................................................................................ 42
5
ABBREVIATIONS AND ACRONYMS
CES Conference of European Statisticians
CIS Commonwealth of Independent States
ECASTAT European and Central Asian Statistics
EECCA Eastern Europe, Caucasus, and Central Asia
EFTA European Free Trade Association
ILO International Labour Association
IMF International Monetary Fund
NSO National Statistical Office
OECD Organization for Economic Cooperation and Development
ROSSTAT Russian Federal State Statistical System
SBR Statistical Business Registers
SNA System of National Accounts
SEE South East Europe
SDGs Sustainable Development Goals
UNECE United Nations Economic Commission for Europe
UNESCAP United Nations Economic and Social Commission for Asia and the Pacific
UNSC United Nations Statistical Commission
UNSD United Nations Statistical Division
WB World Bank
6
EXECUTIVE SUMMARY
The purpose of this evaluation is to assess the extent to which the objectives of the UNECE Project “Enhancing
coherence and integration of economic and social statistics in support of the implementation of 2008 SNA” were
achieved. This evaluation assessed the relevance, effectiveness, efficiency, and sustainability of the project
considering its stated goals and objectives.
The evaluation also attempted to assess how the project’s activities contributed to gender equality and women’s
empowerment as well as the realization of human rights. The evaluation considered good practices and lessons
learned from the project to formulate recommendations aimed at improving future projects.
The evaluation was carried out through a desk review of all relevant project documentation, analysis of the results
of two on-line surveys designed for the purpose of this evaluation and in-depth interviews with key project
stakeholders/partners.
The evaluation determined that the project was fully relevant to the objective of the UNECE to increase the
capacity of target countries 1 in compiling national accounts and supporting statistics according to the
recommendations of the 2008 SNA and other relevant international standards by focusing on the outstanding
priority areas identified by the countries. These priority areas were derived from the Global Assessments of
countries in the EECCA region, which was jointly carried out by the UNECE, Eurostat and EFTA as well as the
regional and national plans for implementation of the 2008 SNA. The project work was guided by the UNECE
Steering Group on National Accounts. The project built on the activities and progress achieved in the work to
support the implementation of the 2008 SNA and supporting economic statistics in the EECCA region, prior to its
inception. Throughout the project, the needs and priorities of the EECCA countries formed the basis of regional
workshops and special sessions designed to address their specific issues or challenges.
The most significant challenge to achieving the expected results on time was the onset of the global pandemic in
early 2020. All planned activities for 2020 were delayed and had to realigned. The UNECE quickly adapted to the
new global reality and implemented the use of online/virtual meetings in order to complete the planned activities.
Face-to-face meetings were replaced with online sessions and adjustments were made to accommodate the new
way of conducting the project (i.e. shorter sessions over many days). A dedicated wiki page was created to share
the results of the activities.
The evaluation determined that the project was effective deploying both traditional and innovative methods to
ensure the objectives and results were achieved. The project consisted of four inter-related modules with the goal
to enhance coherence and integration of economic and social statistics in support of the implementation of the
2008 SNA. Each module had specific objectives with activities designed to achieve the objectives. All the activities
were completed, and additional activities were added with the project extension.
The evaluation determined that the project efficiently achieved its objectives within the allocated budget and
adjusted timeframe. As mentioned above, the UNECE introduced virtual meetings to continue the momentum of
the project during unprecedented challenges. The savings associated with the new way of doing business and its
ability to draw on established partnerships to obtain additional resources through in-kind or financial contributions
extended the project to the benefit of the targeted countries. The no-cost extension expanded the scope of the
project to address outstanding issues, identified by EECCA countries, resulting from the COVID-19 pandemic. The
use of virtual meeting platforms granted a broader access allowing additional stakeholders and partners to
participate. This included not only experts from national statistical offices but from their central banks and
ministries of finance.
1 The target countries for the project were the 12 EECCA countries of Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova,
Russian Federation, Tajikistan, Turkmenistan, Ukraine, and Uzbekistan
7
There were limitations to the evaluation. Although all UNECE work is designed to be as inclusive as possible, the
human rights dimension is a difficult aspect to cover. The human rights dimension was not incorporated in the
project proposal and the project outcomes make no explicit reference to the human rights. No gender
consideration was recorded in the project’s proposal and no indicator established.
In addition, to properly assess whether the knowledge and expertise of the staff in the national statistical offices of
EECCA countries improved and/or their capacity to implement standards and recommendations in practice could
only be achieved through an in-depth country peer review or update to the Global Assessments. This was not
done as part of this evaluation.
The benefits of the project will continue after its completion although there will be some challenges. The UNECE
has a mandate to support the implementation of 2008 SNA and supporting economics statistics in the EECA region.
The Steering Group on National Accounts, established by the Bureau of the Conference of European Statisticians
guides the work of the UNECE Statistical Division in the area of national accounts and supporting economic
statistics, including activities to support and monitor the implementation of the 2008 SNA in the EECCA and to
advise on the provision of methodological recommendations, training and other capacity building activities on
national accounts. Country representatives from the EECCA are members of the Steering Group. By nature of
their mandates these partners/stakeholders are committed to maintaining and progressing the work undertaken
during the project.
Continued sharing of experiences and lessons learned is critical to the ongoing improvement of the statistical
system in the beneficiary countries. EECCA participation in the Expert Group meetings is encouraged and where
possible special regional workshops would continue to offer continuous growth and learning opportunities. For
the EECCA countries, lack of funding, insufficient experienced staff, staff turnover, and limited technical capabilities
are likely to be significant challenges in sustaining the work.
In the spirit of continuous learning and based on the findings and good practices identified throughout the
evaluation, a series of recommendations are intended to help improve future UNECE activities.
1. The global pandemic forced the UNECE to rethink the way it conducted business to ensure continuity of
the project. These innovative solutions introduced good practices that should be continued. The UNECE
is encouraged to use a hybrid model when organizing its workshops and regional sessions in the future.
The introduction of short webinars and the video capture of in person sessions provided additional
learning opportunities. A combination of face-to-face activities and virtual offerings reaches a broader
group of people. Using appropriate and tested technologies that accommodates everyone’s participation
is encouraged.
2. To properly assess the degree to which the knowledge and expertise of the staff in the national statistical
offices of EECCA countries improved and/or their capacity to implement standards and recommendations
increased, an in-depth country peer review or update to the Global Assessments should be conducted.
3. Continued sharing of experience and lessons learned is critical to the ongoing improvement of the
statistical system in the beneficiary countries. EECCA participation in the Expert Group meetings is
encouraged and where possible regional workshops or special sessions would continue to offer
continuous growth and learning.
4. To ensure sustainability, future activities should be linked to the work programme of the UNECE Statistical
Division. Although the budget of the UNECE Statistical Division is limited, it is encouraged to use its
exceptional collaboration and outreach skills to secure additional funding from key project stakeholders,
be it in-kind or financial resources to expand the scope of its work.
8
5. For gender and human rights perspectives to be better included in future projects, there should be an
explicit objective built into the project design that can be monitored and evaluated during and at the end
of the project. Expertise in gender quality, disability inclusion, environment and or human rights should
be sought in the development of these objectives and with the evaluation process.
9
1.0 INTRODUCTION2
In 2009 at the 40th session of the United Nations Statistical Commission (UNSC), it supported the Global
Implementation Strategy for the 2008 System of National Accounts (SNA) and requested to pay particular
attention to improving basic economic statistics and to the need for consistency and harmonization across sectoral
statistics from other macroeconomic frameworks. The UNSC recognized the role of the UN regional commissions
in facilitating coordination and regional cooperation towards the implementation of 2008 SNA. 3 Following the
decision, the UNSC, the United Nations Statistics Division (UNSD) requested that regional commissions develop
regional implementation plans and establish Steering Groups to guide and coordinate activities.
Since 2010, the UNECE has actively been working to support the implementation of 2008 SNA and supporting
statistics in countries of Eastern Europe, Caucasus and Central Asia (EECCA)4. At the request of the UNSC, UNECE
developed a regional plan for implementation of 2008 SNA in close cooperation with EECCA and South-East
European (SEE) countries.
The regional implementation plan was developed in consultation with the Steering Group on National Accounts,
established by the Bureau of the Conference of European Statisticians (CES). The Steering Group on National
Accounts guides the work of the UNECE Statistical Division in the area of national accounts and supporting
economics statistics, including activities to support and monitor the implementation of the 2008 SNA in the EECCA
region and to advise on the provision of methodological recommendations, training and other capacity building
activities on national accounts. The Steering Group is composed of representatives of all major regional partners
such as CIS-Stat, Eurostat, International Monetary Fund (IMF), Organization for Economic Cooperation and
Development (OECD), and UNSD, as well as country representatives from each UNECE sub-region, including from
the EECCA.
The ECASTAT project was intended to build on the activities and progress achieved in the work to support the
implementation of 2008 SNA and supporting economic statistics in the EECCA region. The project was intended to
enable the UNECE in carrying out its mandate to support the implementation of 2008 SNA and supporting statistics
in the region.
The project was part of the ECASTAT regional trust fund established by the World Bank for assisting countries to
improve their statistical systems.
The overall objective of the Project was to increase the capacity of countries in Eastern Europe, Caucasus and
Central Asia in compiling national accounts and supporting statistics according to the recommendations of
international statistical standards.5 Another objective was to include overall improvement of the knowledge and
2 ECASTAT Project for 2017-2020. Implementation of 2008 SNA. Enhancing Coherence and Integration of Economic and Social Statistics
Support of the Implementation of 2008 SNA. Project Proposal – Concept note 2016
3 Statistical Commission. Report on the fortieth session (24-27 February 2009). https://unstats.un.org/unsd/statcom/doc09/Report-English.pdf
4 The target countries for the project were the 12 EECCA countries of Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova,
Russian Federation, Tajikistan, Turkmenistan, Ukraine, and Uzbekistan.
5Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 SNA
https://www.statswiki.unece.org/display/EC/ECASTAT+Home
10
expertise of the staff in the NSOs of EECCA countries regarding relevant international standards and
recommendations in practice.
The project was intended to build on existing UNECE programmes to support the implementation of 2008 SNA and
supporting statistics in the region, in addition to the global assessments of national statistical systems. It intended
to draw on established partnerships with national and international statistical agencies involved in statistical
capacity building activities in the region, and in particular, the WB, UNESCAP, IMF, Eurostat, OECD, European Free
Trade Association (EFTA), International Labour Organization (ILO), Russian Federal State Statistical System
(ROSSTAT), CIS-STAT and other relevant multinational and bilateral partner organizations.
1.1 PURPOSE OF THE EVALUATION
The purpose of this evaluation is to assess the extent to which the objectives of the UNECE Project “Enhancing
coherence and integration of economic and social statistics in support of the implementation of 2008 SNA” were
achieved.
The evaluation assessed the relevance, effectiveness, efficiency, and sustainability of the aforementioned project.
The evaluation also attempted to assess any impacts the project may have had on progressing human rights, gender
equality, disability inclusion, climate change and disaster risk reduction in the context of this engagement. The
evaluation considered the activities repurposed to address the impact of the COVID-19 crisis and assessed UNECE’s
COVID-19 early response.
1.2 OBJECTIVES OF THE EVALUATION
The evaluation of the project has the following specific objectives:
1. Determine, as systematically and objectively as possible, the relevance, effectiveness, efficiency, and
sustainability of the project results considering its goals and objectives.
2. Assess how the project activities contributed to gender equality and women’s empowerment, as well as
the realization of human rights, with an emphasis on “leaving no one behind” and, if needed, make
recommendations on how these considerations can be better addressed in future activities of the sub
programme.
3. Identify good practices and lessons learned from the project and formulate action-oriented, forward-
looking recommendations addressed to the sub programme for improving future interventions.
1.3 SCOPE OF THE EVALUATION
The evaluation covered the full project implementation period 2017-2023 (initially 2017-2020), including the no-
cost extension that was granted to the project to cover the period of 2021-2023.
The evaluation was guided by the objectives, indicators of achievement and means of verification established in
the initial project proposal and the no-cost extension.
The scope was limited to the activities described in project documents and the evaluation criteria established by the
UNECE in its Terms of Reference (see Annex 4.1) for the project evaluation.
The evaluation was conducted ensuring accessible data collection and inclusive consultation, adhering to ethical
standards and triangulation of data for inclusion.
11
1.4 EVALUATION METHODOLOGY
The evaluation was conducted in accordance with: the ECE Evaluation Policy6; the Administrative instruction guiding
Evaluation in the UN Secretariat7; and the United Nations Evaluation Group (UNEG) Norms and Standards for
Evaluation8. Human rights and gender equality considerations were integrated at all stages of the evaluation.
The evaluation was guided by the evaluation matrix articulated in the Inception Report and assessed the degree of
alignment between the project’s stated objective and outcomes with the results of the activities carried out by the
UNECE in the implementation of the project.
The evaluation process included a desk review of all available project documentation, analysis of the results from
the two on-line surveys designed for the purpose of this evaluation and interviews with key stakeholders and
partners to gather more in-depth information and assessment of results.
1.4.1 Desk Review
The desk review included all project documentation provided by the UNECE Statistical Division, relevant websites
and a review of the global assessments conducted jointly by the UNECE, Eurostat and EFTA. The documents
reviewed included: the Project proposals - 2017-2020 and 2021-2023; the 11 semi-annual Project progress reports;
reports from the meetings, workshops and webinars (including a review of the material presented); the meeting
evaluations; the draft World Bank’s Implementation Completion and Results Report, and the 8 available unaudited
interim financial reports (June to December 2017 to 30 June 2022). Unfortunately, the final interim financial
report covering the final year of the project and the final program financial report were not completed in time for
inclusion in the evaluation process. A complete list of documents reviewed are provided in Annex 4.8.
1.4.2 Surveys
Two surveys were conducted in English; one with all beneficiary countries and the second with a select group of
stakeholders and partners. The survey questions are included in Annex 4.2 and 4.4 of this report. The surveys
were conducted online and available to respondents from 4 - 29 September 2023.
The first survey was sent to beneficiaries of the target countries for the project. 36 individuals (20 women; 16
men) representing 12 countries were invited to participate in the survey. At the close of the collection period, 15
individuals from 8 countries responded (41.6% response rate). Of these responses, 12 respondents represented
national statistical offices; 3 represented other domestic agencies; 11 were women and 4 were men. Full results
are included in Annex 6.4 to this report.
The second survey was sent to 18 key stakeholders and partners (6 women; 12 men). Even with an extension of
the collection period, only 6 responses were received (33% response rate) from 4 men and 2 women. The low
response rate limited the type of analysis that could be conducted. The results were used to confirm/challenge
the findings identified through the desk review and the in-depth interviews.
6 UNECE Evaluation Policy. https://unece.org/sites/default/files/2021-12/Item10_ECE_EX_2021_35_Evaluation%20Policy.pdf
7 ST/AI/2021/3
8 UNEG 2016 Norms and Standards for Evaluation. https://www.unevaluation.org/document/detail/1914
12
1.4.3 Interviews
A series of interviews with the stakeholders/partners were conducted to further gather both qualitative and
quantitative information. The interviews were conducted by video conference during the month of September
2023. The interview guide is included in Annex 4.5 to this report.
The invitation was sent to 10 key stakeholders/partners (3 women; 7 men) who represented the various
components of the project (SNA, CPI, SBR, Informal Sector and Remittances). Nine individuals (2 women; 7 men)
graciously gave their time and feedback regarding the various activities and outcomes. Their insights and views
were valuable in forming the findings, recommendations and the lessons learned of this evaluation. A list of the
key stakeholders is included in Annex 4.7 of this report.
2.0 FINDINGS
2.1 RELEVANCE
2.1.1 To what extent was the project design appropriate for meeting the needs of beneficiary countries?
The project design was appropriate for meeting the needs of the beneficiary countries.
The overall aim of the project was to increase the capacity of EECCA countries in compiling national accounts and
supporting statistics according to the recommendations of the 2008 SNA and other relevant international
standards, with focus on outstanding priority areas identified by the countries. These priority areas were derived
from the Global Assessments of countries in the EECCA region, which was jointly carried out by the UNECE,
Eurostat and EFTA as well as the regional and national plans for implementation of the 2008 SNA. The project
work was guided by the UNECE Steering Group on National Accounts. The project built on the activities and
progress achieved in the work to support the implementation of the 2008 SNA and supporting economic statistics
in the EECCA region, prior to its inception.
The project activities included participation of experts from beneficiary countries in UNECE Group meetings,
regional workshops and special sessions to ensure capacity building and shared learning within the EECCA
countries. The design also included simultaneous interpretation in meetings and workshops and translation of
project documentation, training materials and handbooks to Russian to facilitate participation and ongoing work.
At the end of each workshop or meeting, participants were encouraged to share suggestions for improvement and
ideas for future agenda items. The feedback and input were used to formulate the agenda and organization of
subsequent meetings.
13
2.1.2 To what extent did the project respond to the priorities and needs of national statistical offices in beneficiary
countries? How relevant were the project activities to the countries’ needs and priorities?
The project responded “to a high extent” to the priorities and needs of national statistical offices in the beneficiary
countries.
FIGURE 1: EVALUATION OF THE UNECE PROJECT ON ECONOMIC AND SOCIAL STATISTICS
• Source: Project Evaluation Online survey – Question 5
The results of the online survey of beneficiary countries showed 73% of respondents indicated the project’s
activities met their country’s needs to a high extent and 27% to a moderate extent. This finding was also
supported in the review of the evaluations of the various activities that were conducted throughout the project.
Overall the participants were very satisfied with the workshops and meetings as well as the quality of the
discussions and presentations.
During the span of the project, the meetings of the Group of Experts generally included special sessions for the
beneficiary countries with topics identified by the countries. For example, the 2017 and 2019 meetings of the
Group of Experts on Business Registers included Special Sessions for the EECCA and SSEE countries to encourage
exchanges and presentations aimed at improving the coverage and accuracy of Statistical Business Registers and
short-term business statistics. In 2017 the UNECE Guidelines on SBR – activities in the CIS was presented and
produced in both English and Russian. In 2019 the special session focussed on different CPI data collection
methods; the treatment of seasonal items and missing observation; index calculation and weighting; owner-
occupied housing and rentals in the CPI; and how to meet user needs in terms of documentation, dissemination
and communication of the CPI. All of these topics were developed to achieve the project’s objectives and respond
to the beneficiary countries’ needs.
Due to the global pandemic, several activities were delayed, postponed or moved to a virtual platform rather than
face-to-face meetings. There were mixed reviews regarding the online forum. The online forum increased the
number of experts who could participate in the meetings and workshops as travel was no longer required.
However, there were some technical difficulties raised with some of the platforms that were used. Several people
consulted during the evaluation process spoke about the value in having face-to-face meetings as many extended
discussions and personal learning happen during the breaks and outside the formal meeting agenda.
2.1.3 To what extent was the project aligned with the SDGs?
Partnerships (SDG 17) between stakeholders including governments, international and regional organizations,
businesses and academia and civil society are a driving force of UNECE’s work. This includes work to improve
connectivity with the regions (SDGs 7, 8, 9, 11,13), and contributing to creating more dynamic and resilient
economies (SDGs 7, 8, 9,11, 13). The UNECE works for member countries by supporting evidence-based decision
making through reliable statistics and improved monitoring, analysis and policy advice; fostering cooperation
To what extent did the event respond to your country's
statistical needs and priorities?
To a high extent To a moderate extent To a low extent Not at all
14
among all stakeholders at the country and regional level; building countries’ capacities to implement its norms,
conventions and standards.9
Basic sound economics, including key figures of the national accounts are fundamental in relation to the SDGs.
Several proposed SDG indicators on income, consumption, and production depend on the availability of
comprehensive and high-quality national statistics that are internationally comparable.
The project was designed to help EECCA countries improve the coherence, timeliness and internationally
comparability of socio-economic statistics that are crucial for the monitoring of a country’s social and economic
progress. The project design encouraged the collaboration of experts from the statistical offices as well as central
agency representatives from national banks and finance ministries which improved the coherence of the statistical
outputs.
2.1.4 What takeaways are there for ensuring relevance of future projects of UNECE Statistical Division?
The project activities were linked to the work programme of the UNECE Statistical Division to ensure continuity
and coordination with international work on national accounts and supporting statistics. This is a critical element
for continued success of future projects.
The global pandemic forced the UNECE to examine and implement innovative ways to continue with the activities
essential in achieving a successful outcome to the project.
UNECE’s ability to draw on established partnerships and to obtain additional resources through in-kind or financial
contributions extended the project to the benefit of the targeted countries and expanded the scope of the project
to address outstanding issues, identified by EECCA countries, resulting from the COVID-19 pandemic.
2.1.5 To what extent were gender, human rights and disability perspectives integrated into the design and
implementation of the project? What results can be identified from these actions? How can gender and human
rights perspectives be better included in future projects design and implementation?
There is no evidence from document review or information gathered throughout this review that gender, human
rights and disability perspectives were integrated into the design and implementation of the project. However, it
is evident from the evaluation that women and youth were involved in all aspects of the project and persons with
disabilities were accommodated throughout the various meetings, workshops and special sessions.
To ensure gender and human rights perspectives be better included in future projects, there should be an explicit
objective built into the project design that can be monitored and evaluated during and at the end of the project.
Expertise in gender quality, disability inclusion, environment and or human rights should be sought in the
development of these objectives and with the evaluation process.
9 SDG Priorities. Supporting countries to achieve the SDGs https://unece.org/sdg-priorities
15
2.2 EFFECTIVENESS
2.2.1 To what extent were the project objectives and results achieved?
The overall aim of the project and the expected outcomes were achieved.
The project consisted of four inter-related modules with the overall goal to enhance coherence and integration of
economic and social statistics in support of the implementation of the 2008 SNA. The four modules were:
Methods and compilation techniques for the implementation of the 2008 SNA and supporting economic statistics;
Price Statistics – development of price indices and deflators for national accounts aggregates; Statistical business
registers – improving coverage and accuracy of national statistical business registers; and Expanding the use of
household surveys for the estimates of the informal sector and remittances.
Each module had specific objectives with activities designed to achieve the objectives (see Annex 6.6 for full
details). All the activities within each module were completed and additional activities were added with the
project extension. The project activities included participation of experts from EECCA countries in international
statistical Expert Group meetings, regional workshops and special sessions designed specifically to address their
needs. The project activities also included the development of statistical methods/tools for integrating migration
in household surveys and development of training material. The desk review, supported by the online survey
results and interviews with key stakeholders, provided evidence that the capacity of countries in compiling
national accounts and supporting statistics increased.
However, there were limitations in the evaluation. For example to properly assess whether the knowledge and
expertise of the staff in the national statistical offices of EECCA countries improved and/or their capacity to
implement standards and recommendations in practice could only be achieved through an in-depth country peer
review or update to the Global Assessments.
2.2.2 To what extent did the project improve the competencies of national statistical offices in the beneficiary
countries to produce and use macroeconomic statistics in their countries?
The activities of the project were designed and implemented to improve the competencies of national statistical
offices in the beneficiary countries.
An explicit example is the Special Session that was organized to support the EECCA and SEE countries on the use of
Supply and Use tables (SUTs) as well as Input Output tables (IOTs). SUTs are at the core of National Accounts and
represent a framework to ensure better quality, coherency, and consistency of macroeconomic aggregates. A
survey among the target countries carried out before the session revealed that 15 countries were developing SUTs,
some of them being at initial or experimental estimates. The main challenges identified by countries were lack of
suitable data sources, human resources and time constrains. The focus of the special session was to help the
EECCA & SEE countries address their challenges. During this session, different IT tools for the compilation of SUTs
and IOTs were presented which including a short training session on two tools developed by the IMF to support
balancing of SUTs and conversion into IOTs. The Special Session also helped EECCA and SEE countries share
experience on data sources and compilation techniques and informed them on new developments in SUTs and
IOTs. The evaluation survey at the end of the session showed that 95 percent of the participants considered the
session to be very helpful or useful for their future work.
The second concrete example is the development of a harmonized survey module on migration and remittances
for countries in EECCA. The survey module, developed within the context of the project, was designed to enable a
regular sustainable collection of internationally comparable data on migration and remittances. Improvements in
the coverage, exhaustiveness, and comparability of statistics on migration & remittances support SDG monitoring
needs, in line with international standards. The module was tested in Tajikistan in late 2019 with a technical
assistance mission supported by the project fund. Findings from the pilot test were shared with the EECCA
countries in 2020 and the harmonized module was released in 2021 as a standard tool that has been produced in
Russian and English.
16
2.2.3 To what extent are the project activities coherent and harmonized with those of other partners operating
with the same context, particularly those of other UN system entities?
This UNECE-managed project is coherent and harmonized with those of partner organizations. Representatives
from EFTA, Eurostat, ILO, OECD, UNSD and World Bank were all consulted during the evaluation and were all very
complimentary of the UNECE’s collaboration and coordination expertise. Apart from the World Bank, who was the
sponsor of the project, these partners are members of the UNECE Steering Group on National Accounts who guided
and advised on the activities undertaken throughout the project.
During this evaluation the UNECE was recognized for its exceptional organization skills and the collaboration with
other partners in the international statistical system to provide high quality learning activities. The UNECE was
effective at bringing together expertise and ensuring no duplication with other international activities. The team
members were also recognized for constantly seeking feedback and input and acting on the information gathered
to improve subsequent events/activities. Consultation regarding agenda items and effective advanced planning
ensured a positive experience for participants.
In the development of the various activities carried out throughout the project, the UNECE considered the CIS-Stat
activities financed by the Trust Fund for Statistical Capacity Building and ensured there were no overlaps between
the two projects.
2.2.4 What were the challenges/obstacles (Including COVID-19 and sub-regional instability) to achieving the
expected results? How successfully did the project overcome these?
The most significant challenge to achieving the expected results on time was the onset of the global pandemic in
early 2020. All planned activities for 2020 had to be adjusted. Face-to-face meetings were replaced with online
sessions and adjustments were made to accommodate the new way of conducting the project (ie shorter sessions
over many days). A dedicated wiki page was created to share the results of the activities.
In October 2020 a three-day special online session was conducted with a focus on the EECCA countries to provide
an overview of the migration and remittances situation in these countries during the pandemic. The session
explored the impact of the pandemic on the measurement of migration and remittances and review the use of
administrative data in producing migration statistics. A consultant was also hired to carry out a survey on
migration and remittances measurement in the pandemic which included data sources, methods and challenges
faced and solutions applied. The results of the survey were presented and discussed during the meeting.
The June 2020 meeting of the Group of Experts on Consumer Price Indices had to be postponed to 2021 due to the
pandemic. During the meeting in 2021 a draft Guide on producing CPI under lockdown was presented. The Guide
provided recommendations and good practices for data collection, imputation methods and communication
approaches that countries can refer to in periods of lockdown or extenuating circumstances. The Guide was
published in August 2021 and is available in English on the UNECE website.
The second challenge that the project faced was the political instability in the region in 2022. Few mitigation
measures are possible as events organized under ECASTAT are technical and not political events.
2.2.5 What (if anything) has prevented the project from achieving the desired results?
All the challenges and obstacles the project faced were addressed. The global pandemic and the forced change in
the way the activities were conducted resulted in a no-cost project extension and additional workshops and
sessions being organized and conducted. The use of online platforms for workshops and sessions increased the
number of participants from targeted countries including statisticians as well as representatives of their central
agencies.
17
2.3 EFFICIENCY
2.3.1 Were the resources allocated adequate for achieving results?
The project was sponsored by the ECASTAT regional trust fund established by the World Bank for assisting
countries to improve their statistical system. According to the Project proposal a budget of $912,00010 (Table 1)
was established to support the project.
TABLE 1: BUDGET ALLOCATION
Activity Planned events Budget Amount - $912,000
Seminars and workshops
- Expert Group Meetings
- Regional Workshops &
Seminars
- Interpretation & logistics
- 5
- 6
$508,600
Translation of training material $29,400
Consultants & contractual services $178,000
Consultant – project administration $96,000
Staff Travel - 6 regional workshops $24,000
Miscellaneous costs $58,000
Independent project evaluation $18,000
During the initial project timeframe (2017-2020) $399,000 was used to support the project module activities,
project administration and UN Programme support (Table 2). The unused $513,000 was the result of delayed or
postponed activities due to the global pandemic as well as an increase in resources to support the project provided
by EFTA and the UN Development Account Programme on Statistics and Data. The $513,000 was re-allocated to
the no cost project extension that was approved for 2021-2023.
10 All financial figures are in US Dollars
18
TABLE 2: INITIAL PROJECT ALLOCATION OF FUNDS AND EXPENDITURES PER MODULE (2017-2020)
Modules Budget Expenditures Unused Funds
Methodology & compilation techniques of 2008
SNA
$234,000 $38,000 $196,000
Price Statistics & compilation of national accounts
aggregates at constant prices
$142,000 $58,000 $84,000
Improving the coverage and accuracy of statistical
business registers & short-term statistics
$116,000 $47,000 $69,000
Expanding the use of household surveys for the
estimates of the informal sector and remittances
$248,000 $183,000 $65,000
Project Administration, UN Programme Support
and Project Evaluation
$172,000 $73,000 $99,000
TOTAL $912,000 $399,000 $513,000
The following activities were designed and implemented within the project extension: 3 meetings of Group of
Experts on National Accounts; 2 meetings of the Group of Expert on Consumer Prices Indices; 1 meeting of the
Group of Experts on Business Registers; a workshop on migration statistics and two regional workshops on
National Accounts. Table 3 presents the budget allocation for the 2021-2023 project extension.
TABLE 3: BUDGET ALLOCATION FOR 2021-2023
Year Activity Budget
2021 Meeting of the Group of Experts on National Accounts
Meeting of the Group of Experts on CPI
Workshop on Migration Statistics
$126,000
2022 Meeting of the Group of Experts on National Accounts
Meeting of the Group of Experts on Business Registers
Regional workshop on National Accounts
$144,000
2023 Meeting of the Group of Experts on National Accounts
Meeting of the Group of Experts on CPI
Regional workshop on National Accounts
$144,000
Project Administration $49,000
UN Programme support costs $32,410
Independent project evaluation $17,880
TOTAL $513,290
19
The final financial reporting information was not completed at the time of this evaluation. However, the
evaluation concluded that all the planned activities, including those added as a result of the no-cost extension,
were completed. There were 24 Expert Group meetings, webinars, regional workshops, special sessions
successfully conducted from 2017-2023 (see Annex 4.6). In addition, training materials were developed, a Guide
on Producing CPI under Lockdown11 was developed and a harmonized survey module on migration and remittances
for countries in EECCA was created (see 2.2.2 for details).
2.3.2 Were the results achieved on time and were all activities organized efficiently?
The project was completed within the allocated timeframe. Some activities were postponed due to COVID-19 and
political unrest in the region, but all were eventually completed albeit in formats different from the original plan.
The UNECE quickly adapted to the new global reality and implemented the use of online/virtual meetings in order
to complete the planned activities. Efficient use of funds and inter-agency collaboration supported a no-cost
extension to the project. The use of virtual meeting platforms increased the number of participants expanding the
exchange and experiential learning encouraged by the project.
2.3.3. To what extent were the resources used economically and how could the use of resources be improved?
The financial resources were effectively and efficiently managed. In addition to the funding providing by the
ECASTAT Trust Fund, the UNECE collaborated with EFTA which provided funding and in-kind contributions for
many of the planned activities UNECE was also able to augment the available resources by using funds from the
UN Development Account Programme on Statistics and Data to cover costs such as the participation of national
experts in meetings and workshops as well as the critical translation of training materials. The no-cost extension
for 2021-2023 was possible due to these additional sources of funding that occurred during the initial project
timeframe (2017-2020) as well as the cost savings during COVID-19 due to the delay, postponement or
replacement of face-to-face meetings with virtual sessions.
2.4 SUSTAINABILITY
2.4.1 What measures were adopted to ensure that project outcomes would continue after the project ended and
to what extent have these measures addressed the existing risks for sustainability?
The following measures have been adopted to ensure the project outcomes continue: all project documentation is
available through online channels and all activities encouraged the exchange of good practices and participation of
EECCA experts in workshops, meetings and regional sessions. The project stimulated the creation of a network of
experts from the EECCA region. This network which includes representatives from the national statistical offices as
well as representatives from central banks and ministries of finance aims to improve the institutional cooperation
and consistency between the national accounts and balance of payments.
In addition, the guidance notes of the SNA and Balance of Payments Manual, 6th Edition (BPM6) developed by the
joint UN Task Team on Informal Economy has been translated into Russian to ensure that the EECCA region is fully
involved in the consultation process.
The development of the harmonized survey module for a standard measurement of migration and remittances for
countries in EECCA establishes the way in which these countries can arrive at estimates of migration and
remittances that support SDG monitoring needs, are in line with international standards and are harmonized
among CIS countries. The module is available as a standard tool and produced in Russian and English.
11 United Nations Geneva, 2021 Guide on producing CPI under lockdown https://www.unece.org/statistics/publications/guide-producing-cpi-
under-lockdown
20
The results of the online survey of beneficiary countries rated the quality of support provided by the UNECE as
“excellent” (60%) and “good” (40%). It also showed the extent of the work of the activities still relevant to their
country as “to a high extent” (73%) and “to a moderate extent” (20%). When asked whether the outputs of the
activities are sustainable, 47% said “to a high extent”; 40% said to a “moderate extent”, and 13% said “to a low
extent”. The major risks identified for sustainability included lack of expert staff (67%), staff turnover (47%),
limited technical capabilities (53%) and lack of funding (40%).
FIGURE 2: EVALUATION SURVEY – EVALUATION OF THE UNECE PROJECT ON ECONOMIC AND SOCIAL STATISTICS
* source – Project Evaluation Survey – Questions 8 and 6
2.4.2 To what extent do the partners and beneficiaries “own” the outcomes of the work? How is the stakeholders’
engagement likely to continue, be scaled up, replicated or institutionalized?
The UNECE has a mandate to support the implementation of 2008 SNA and supporting economics statistics in the
EECA region. The Steering Group on National Accounts, established by the Bureau of the Conference of European
Statisticians guides the work of the UNECE Statistical Division in the area of national accounts and supporting
economic statistics, including activities to support and monitor the implementation of the 2008 SNA in the EECCA
and to advise on the provision of methodological recommendations, training and other capacity building activities
on national accounts. Country representatives from the EECCA are members of the Steering Group.
By nature of their mandates these partners/stakeholders are committed to maintaining and progressing the work
undertaken during the project.
Continued sharing of experience and lessons learned is critical to the ongoing improvement of the statistical
system in the beneficiary countries. EECCA participation in the Expert Group meetings is encouraged and where
possible regional workshops would continue to offer continuous growth and learning. Lack of resources and
funding are likely to be a major challenge in sustaining the work.
How would you rate the quality of the
support provided by the UNECE
Excellent Good Average Poor
To what extent is the work still relevant
to your country?
To a high extent To a moderate extent
To a low extent Not at all
21
2.5 GENDER AND HUMAN RIGHTS CONSIDERATIONS
Throughout the review, it was evident that women and men were involved in the various project activities, and in
the evaluation process. Youth were also involved in the various activities. Consideration was given to those with
physical disabilities during the workshops conducted. Training materials were translated to ensure beneficiary
countries could learn in their preferred language. However, no gender consideration was recorded in the project’s
proposal and no indicator established.
The evaluation process was inclusive by ensuring that a diverse group of stakeholders, partners and beneficiary
countries were consulted and encouraged to provide information and input. This included women, men and youth.
It is unknown if anyone who was consulted during the process was a person with disabilities. The two online surveys
included a question on gender and the survey intended for stakeholders asked a question regarding the participation
of women, persons with disabilities, minority groups and youths in the project’s activities and events. The interviews
conducted with international and regional partners explored cross cutting questions regarding the inclusion of
women, persons with disabilities, minority groups and youth as well as gender and human rights.
Although all UNECE projects are as inclusive as possible, this one included, the human rights dimension is a difficult
and complex aspect to cover. The human rights dimension was not incorporated in the project proposal and the
project outcomes make no explicit reference to the human rights. It should be considered however, that the Generic
Law on Official Statistics (GLOS)12 requires “equal and simultaneous access” to official statistics for all users and is
aligned with the United Nations Fundamental Principles of Official Statistics-Principle 1 of Relevance, Impartiality
and Equal Access.13 The GLOS was developed within the UNECE Project “Strengthening national capacity in the most
vulnerable UNECE countries for the sustainable development of statistics”. It was developed primarily for the
beneficiary countries of the project and EECCA countries were consulted throughout its development.
12 https://unece.org/DAM/stats/publications/2016/ECECESSTAT20163_E.pdf
13 https://unstats.un.org/unsd/dnss/hb/E-fundamental%20principles_A4-WEB.pdf
22
3.0 CONCLUSIONS AND RECOMMENDATIONS
3.1 CONCLUSIONS
Based on the evaluation findings, the evaluator has the following conclusions.
The overall aim of the project was to increase the capacity of EECCA countries in compiling national accounts and
supporting statistics according to the recommendations of the 2008 SNA and other relevant international
standards. This was accomplished through a series of four inter-related modules with specific activities designed
to achieve the goal to enhance coherence and integration of economic and social statistics in support of the
implementation of the 2008 SNA. The project design was appropriate for meeting the needs and priorities of the
beneficiary countries. All the activities designed to support the project’s objectives were completed on time and
on budget.
Basic sound economics, including key figures of the national accounts are fundamental in relation to the SDGs.
Several proposed SDG indicators on income, consumption, and production depend on the availability of
comprehensive and high-quality national statistics that are internationally comparable. The project was designed
to help EECCA countries improve the coherence, timeliness and internationally comparability of socio-economic
statistics that are crucial for the monitoring of a country’s social and economic progress. The project design
encouraged the collaboration of experts from the statistical offices as well as central agency representatives from
national banks and finance ministries which improved the coherence of the statistical outputs.
The project activities were linked to the work programme of the UNECE Statistical Division to ensure continuity
and coordination with international work on national accounts and supporting statistics. This is a critical element
for continued success of future projects.
The most significant challenge that the project faced was the unprecedented global pandemic in early 2020 that
forced the UNECE to examine and implement innovative ways to continue with the activities essential in achieving
a successful outcome to the project. The use of virtual platforms saved costs and increased participation in the
workshops and session organized within the project.
Although the UNECE Statistical Division has limited resources, its ability to draw on established partnerships and to
obtain additional resources through in-kind or financial contributions extended the project to the benefit of the
targeted countries and expanded the scope of the project to address outstanding issues, identified by EECCA
countries, resulting from the COVID-19 pandemic.
During this evaluation the UNECE was recognized for its exceptional organization skills and the collaboration with
other partners in the international statistical system to provide high quality learning activities. The UNECE was
effective at bringing together expertise and ensuring no duplication with other international activities. The team
members were also recognized for constantly seeking feedback and input and acting on the information gathered
to improve subsequent events/activities. Consultation regarding agenda items and effective advanced planning
ensured a positive experience for participants.
All UNECE work is designed to be as inclusive as possible, however, the human rights dimension is a difficult aspect
to cover. The human rights dimension was not incorporated in the project proposal and the project outcomes
make no explicit reference to the human rights. No gender consideration was recorded in the project’s proposal
and no indicator established.
There were limitations in the evaluation. For example to properly assess whether the knowledge and expertise of
the staff in the national statistical offices of EECCA countries improved and/or their capacity to implement
standards and recommendations in practice could only be achieved through an in-depth country peer review or
update to the Global Assessments. This was not done as part of this evaluation.
23
3.2 RECOMMENDATIONS
Based on the findings and the good practices identified throughout the evaluation of the project, the evaluator offers
the following recommendations for consideration.
1. The global pandemic forced the UNECE to rethink the way it conducted business to ensure continuity of
the project. These innovative solutions introduced good practices that should be continued. The UNECE
is encouraged to use a hybrid model when organizing its workshops and regional sessions in the future.
The introduction of short webinars and the video capture of in person sessions provided additional
learning opportunities. A combination of face-to-face activities and virtual offerings reaches a broader
group of people. Using appropriate and tested technologies that accommodates everyone’s participation
is encouraged.
2. To properly assess the degree to which the knowledge and expertise of the staff in the national statistical
offices of EECCA countries improved and/or their capacity to implement standards and recommendations
increased, an in-depth country peer review or update to the Global Assessments should be conducted.
3. Continued sharing of experience and lessons learned is critical to the ongoing improvement of the
statistical system in the beneficiary countries. EECCA participation in the Expert Group meetings is
encouraged and where possible regional workshops or special sessions would continue to offer
continuous growth and learning.
4. To ensure sustainability, future activities should be linked to the work programme of the UNECE Statistical
Division. Although the budget of the UNECE Statistical Division is limited, it is encouraged to use its
exceptional collaboration and outreach skills to secure additional funding from key project stakeholders,
be it in-kind or financial resources.
5. For gender and human rights perspectives to be better included in future projects, there should be an
explicit objective built into the project design that can be monitored and evaluated during and at the end
of the project. Expertise in gender quality, disability inclusion, environment and or human rights should
be sought in the development of these objectives and with the evaluation process.
24
4.0 ANNEXES
4.1 TERMS OF REFERENCE FOR THE EVALUATION
I. Purpose
The purpose of this evaluation is to assess the extent to which the objectives of the UNECE project E264
“Enhancing coherence and integration of economic and social statistics in support of the implementation of 2008
System of National Accounts” were achieved.
The evaluation will assess the relevance, effectiveness, efficiency and sustainability of the project.
The evaluation will also assess any impacts the project may have had on progressing human rights, gender
equality, disability inclusion, climate change and disaster risk reduction in the context of this engagement. The
evaluation will finally look at the activities repurposed to address the impact of the COVID-19 crisis, and assess, to
the extent possible, UNECE’s COVID-19 early response through this project.
II. Background
The project aimed at increasing the capacity of countries in Eastern Europe, Caucasus and Central Asia in compiling
national accounts and supporting statistics according to the recommendations of international statistical
standards. The activities of the project included the following four modules:
1) Methods and compilation techniques for implementation of the 2008 SNA and supporting economic statistics.
2) Price statistics - development of price indices and deflators for national accounts aggregates.
3) Statistical business registers - improving coverage and accuracy of national statistical business registers.
4) Expanding the use of household surveys for the estimates of the informal sector and remittances.
The activities were carried out in cooperation with other international and regional partners, including CIS-STAT,
EFTA, Eurostat, ILO, IMF, OECD and World Bank.
III. Evaluation objectives, scope and questions
The evaluation will be guided by the objectives, indicators of achievement and means of verification established in
the logical framework of the project document. The evaluation will be conducted in Q3 of 2023. It will cover the
full implementation of the project, from June 2017 to June 2023 in Eastern Europe, Caucasus and Central Asia
(Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Russian Federation, Tajikistan,
Turkmenistan, Ukraine and Uzbekistan)
The final evaluation of the project has the following specific objectives:
· Determine as systematically and objectively as possible the relevance, effectiveness, efficiency and
sustainability of the project results in light of its goals and objectives.
· Assess how the project activities contributed to gender equality and women’ s empowerment, as well as
the realization of human rights, with an emphasis on ‘leaving no one behind’ and, if needed, it will make
recommendations on how these considerations can be better addressed in future activities of the sub-programme.
· Identify good practices and lessons learned from the project and formulate action-oriented, forward-
looking recommendations addressed to the sub-programme for improving future interventions.
25
The evaluation criteria are relevance, effectiveness, efficiency and sustainability.
Relevance
1. To what extent was the project design appropriate for meeting the needs of beneficiary countries?
2. To what extent did the project respond to the priorities and needs of national statistical offices in beneficiary
countries? How relevant were the project activities to the countries’ needs and priorities?
3. To what extent was the project aligned with the SDGs?
4. What takeaways are there for ensuring relevance of future projects of UNEE Statistical Division?
5. To what extent were gender, human rights and disability perspectives integrated into the design and
implementation of the project? What results can be identified from these actions? How can gender and human
rights perspectives be better included in future projects design and implementation?
Effectiveness
6. To what extent were the project objectives and expected results achieved?
7. To what extent did the project improve the competencies of national statistical offices in the beneficiary
countries to produce and use macroeconomic statistics in their countries?
8. To what extent are the project activities coherent and harmonized with those of other partners operating within
the same context, particularly those of other UN system entities?
9. What were the challenges/obstacles (including COVID-19 and sub-regional instability) to achieving the expected
results? How successfully did the project overcome these?
10. What (if anything) has prevented the project from achieving the desired results?
Efficiency
11. Were the resources adequate for achieving the results?
12. Were the results achieved on time and were all activities organized efficiently?
13. To what extent were the resources used economically and how could the use of resources be improved?
Sustainability
14. What measures were adopted to ensure that project outcomes would continue after the project ended and to
what extent have these measures addressed the existing risks for sustainability?
15. To what extent do the partners and beneficiaries ‘own’ the outcomes of the work? How is the stakeholders’
engagement likely to continue, be scaled up, replicated, or institutionalized?
IV. Evaluation approach and methodology
The evaluation will be conducted in accordance with: the ECE Evaluation Policy; the Administrative instruction
guiding Evaluation in the UN Secretariat; and the United Nations Evaluation Group (UNEG) Norms and Standards
for Evaluation. Human rights and gender equality considerations will be integrated at all stages of the evaluation:
(i) in the evaluation scope and questions; (ii) in the methods, tools and data analysis techniques; (iii) in the findings,
conclusions and recommendations of the final report. The evaluator will explicitly explain how human rights,
gender, disability, SDGs, and climate change considerations will be taken into account during the evaluation.
The evaluator is required to use a mixed-method approach, including qualitative as well as quantitative data
gathering and analysis as the basis for a triangulation exercise of all available data to draw conclusions and
findings.
The evaluation should be conducted based on the following mixed methods to triangulate information:
1. A desk review of all relevant documents, including the project document and information on project activities
(monitoring data); materials developed in support of the activities (agendas, plans, participant lists, background
documents, donor reports and publications); Proposed programme budgets covering the evaluation period;
project reports to the donor.
26
2. Online survey of key stakeholders and beneficiaries: the survey will be developed by the consultant on her/his
preferred platform.
3. Interviews (in-person and/or by telephone/video): the evaluator shall interview a wide range of diverse
stakeholders and beneficiaries.
4. Remote observation of virtual workshops and meetings, including recordings of meetings.
The evaluator will further elaborate on the evaluation methodology in the Inception Report that will among others
include the survey questions and whether any of the 12 countries will be selected for an in-depth assessment. The
evaluation report will be written in English, will consist of approximately 30 pages and will include an executive
summary (max. 2 pages) describing the evaluation methodology, key findings, conclusions and recommendations.
The evaluator will also produce an Evaluation Brief summarizing key evaluation findings, lessons learned and
recommendations, including through images and infographics.
V. Evaluation schedule
May 2023 ToR finalized
June 2023 Evaluator selected
June 2023 Contract signed. Evaluator starts the desk review
July 2023 Evaluator submits inception report including survey design
August 2023 Launch of data gathering, including survey and interviews
September 2023 Evaluator submits draft evaluation report and evaluation brief
October 2023 Evaluator submits final evaluation report and evaluation brief
VI. Resources and Management of the evaluation
An independent consultant will be engaged to conduct the evaluation under the management of the PMU.
Payment will be made upon satisfactory delivery of work.
The Programme Management Unit (PMU) will manage the evaluation and will be involved in the following steps:
Selection of the evaluator; Preparation and clearance of the Terms of Reference; Provision of guidance to the
Project Manager and evaluator as needed on the evaluation design and methodology; Clearance of the final report
after quality assurance of the draft report.
The Project Manager, in consultation with the Division Director, will be involved in the following steps: Provide all
documentation needed for desk review, contact details, support and guidance to the evaluation consultant as
needed throughout the timeline of the evaluation; Advise the evaluator on the recipients for the questionnaire and
for follow-up interviews; Process and manage the consultancy contract of the evaluator, along the key milestones
agreed with PMU.
VII. Intended use / Next steps
The results of the evaluation will be used in the planning and implementation of future activities of the UNECE
Economic Cooperation and Integration Subprogramme.
Findings of this evaluation will be used when possible to:
· improve direct project’s follow up actions, implementation of products by project beneficiaries and dissemination
of the knowledge created through the project;
· assess the gaps and further needs of countries in the area of this project;
· formulate tailored capacity building projects to strengthen the national capacity in enhancing innovation.
The results of the evaluation will be reported to the Conference of European Statisticians
27
Following the issuance of the final report, the Project Manager will develop a Management Response for
addressing the recommendations made by the evaluator. The final evaluation report, the management response
and the progress on implementation of recommendations will be publicly available on the UNECE website.
VIII. Criteria for evaluators
The evaluator should have:
1. An advanced university degree or equivalent background in relevant disciplines.
2. Knowledge of and experience in working on data and statistics; knowledge of and experience in economic
statistics desirable.
3. Relevant professional experience in design and management of evaluation processes with multiple stakeholders,
survey design and implementation, project planning, monitoring and management, gender mainstreaming and
human-rights due diligence.
4. Demonstrated methodological knowledge of evaluations, including quantitative and qualitative data collection
and analysis for end-of-cycle project evaluations, including demonstrated experience in conducting questionnaires
and interviews.
5. Fluency in written and spoken English.
Evaluators should declare any conflict of interest to UNECE before embarking on an evaluation project and at any
point where such conflict occurs.
28
4.2 ONLINE SURVEY QUESTIONNAIRE – BENEFICIARY COUNTRIES
4.2.1 Introductory text
This online survey is conducted in the context of the evaluation of the UNECE Project: Enhancing Coherence and
Integration of Economic and Social Statistics in Support of the Implementation of 2008 System of National Accounts.
The purpose of this survey is to gather the views of target countries and beneficiaries on the events and activities
organized by the UNECE associated with this project.
The survey, which contains 15 questions, will be conducted in English and take approximately 15 minutes to
complete. All responses are confidential, and no individual can be identified from their responses. UNECE will not
have access to the raw survey results but only to aggregated information. Should you have any question or concerns,
please contact the evaluator, Constance Graziadei at [email protected].
The survey will be available until Friday 22 September 2023.
Thank you in advance for your participation.
4.2.2 Questionnaire
1. Country
• (drop down menu)
2. Which institution do you represent?
• National Statistical Office
• Other Domestic Agencies (Ministry of Finance, Central Bank)
• Other: _______________________
3. Gender of respondent
• Male
• Female
• Other: _________________
• Prefer not to respond
4. What event (meeting or workshop) organised by UNECE have you participate in?
• National Accounts event
• Price statistics event
• Statistical business registers event
• Events on expanding the use of household surveys for the estimates of the informal sector and
remittances
5. To what extent did the event (s) respond to your country’s statistical needs and priorities?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
6. To what extent is the work of the event (s) still relevant to your county?
• To a high extent
• To a moderate extent
29
• To a low extent
• Not at all
• Do not know
7. To what extent did the activities and outputs from the event help to improve your statistics?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
8. How would you rate the quality of the support provided by the UNECE?
• Excellent
• Good
• Average
• Poor
• Do not know
9. How effective were the regional workshops or the UNECE Expert Group meetings?
• Excellent
• Good
• Average
• Poor
• Do not know
10. How effective was the sharing of best practices throughout the event (s)?
• Excellent
• Good
• Average
• Poor
• Do not know
11. To what extent did the project respond to the impact of the COVID-19 pandemic on planned activities?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
• Not applicable
12. To what extent are the outputs from the event sustainable?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
13. What are the main risks for the results to be maintained? Please select all that apply
• Lack of expert staff
• Staff turnover
• Limited technical capacities
30
• Lack of funding
• Government change
• None
• Other: ___________________________
14. Are there any lessons learned or good practices from the project that should be considered for future
projects?
Text box
15. Do you have any comments or recommendations?
Text box
4.3 ONLINE SURVEY RESULTS – BENEFICIARY COUNTRIES
15 responses (11 women; 4 men) were received and results analyzed. On average it took respondents 15:27 minutes
to complete the questionnaire.
Q1. Country
4
1
4
1 0 0 1 0 1
2
0 0 1
Arm
en
ia
Azer
baij
an
Belar
us
Geo
rgi
a
Kaza
kh
sta
n
Kyrg
yzs
tan
Moldova
Mongo
lia
Russi
an
Fe
dera
tio
n
Ta
jik
ist
an
Tu
rkm
enist
an
Ukra
ine
Uzb
ek
ist
an
Countries
Responses
31
Q2. Which institution do you represent?
Q3. Gender
Q4. What events (meeting or workshop) organized by the UNECE did you participate in?
National Statistical Office Other domestic agencies
Gender
Woman Man Prefer not to say Other
11
4
0 0
GENDER
Gender
53%
27%
7%
13%
Event participants
System of National Accounts
Price Statistics
Statistical Business Register
Measuring Migration and
Remittances
32
Q5. To what extent did the event (s) respond to your country’s statistical needs and priorities?
Q6. To what extent is the work of the event (s) still relevant to your country?
Q7. To what extent did the activities and outputs from the event (s) help to improve your statistics?
11
4
0
0
0 2 4 6 8 10 12
TO A HIGH EXTENT
TO A MODERATE EXTENT
TO A LOW EXTENT
NOT AT ALL
Statistical needs & priorities
Statistical needs & priorities
To a high
extent 7
To a
moderate
extent
8
To a low
extent 0
Not at all 0
Improve Statistics
To a high extent To a moderate extent
To a low extent Not at all
33
Q8. How would you rate the quality of the support provided by the UNECE?
Q9. How effective were the regional workshops or the UNECE Expert Group meetings?
Very effective 12 Somewhat ineffective 0
Somewhat effective 3 Very ineffective 0
Neither effective or ineffective 0 Do not know 0
Q10. How effective was the sharing of best practices throughout the event (s)?
Very effective 12 Somewhat ineffective 0
Somewhat effective 3 Very ineffective 0
Neither effective or ineffective 0 Do not know 0
Q11. To what extent did the project respond to the impact of the COVID-19 pandemic on planned activities?
60%40%
QUALITY OF SUPPORT
Excellent Good
0
1
2
3
4
5
6
7
8
COVID-19 response
To a high extent
To a moderate extent
To a low extent
Not at all
Do not know
34
Q12. To what extent are the outputs from the various events sustainable?
Q13. What are the main risks for the results to be maintained? Please select all that apply
Q14. Are there any lessons learned or good practices from the project that should be considered for future projects?
- Measuring and consistent recording of non-observed economy in balance of payments the recording of e-commerce
cross boarder transactions cryptocurrencies SPE MNE Groups data and others.
- A lot of theoretical information.
- From the learned lessons we have planned the improvement of the statistical data production and tools.
- Presentation of the experience of countries with good practices.
- Good practices from the project helped us to establish Business Demography statistics and introduce geographical
coordinates in the Statistical Business Register.
- Try to give the floor for all participation.
- We made a research of card transactions, for 2019, by MCC and reflecting the results by EBOPS, after a presentation of
similar experience during NA Group of expert’s meetings and had a close collaboration with some colleagues to share
our experience after 2023 meeting and presentation of our model of estimations of refugee’s transactions in BOP.
Q.15 Do you have any comments or recommendations?
- All UNECE events on price statistics are always very useful and helpful and informative.
- We would like to see more workshops on price statistics, with participation from both regional statistical offices and
from countries with best practices in index calculations, which may be given the opportunity to consult themselves on
Sustainability
To a high extent To a moderate extent To a low extent Not at all
10
7
8
6
0 0 0
0
2
4
6
8
10
12
Lack of expert
staff
Staff turover Limited
technical
capabilities
Lack of funding Government
change
None Other
Risk Factors
Risk Factors
35
the compilation of indices in practice, using the best practices for the collection of information they possess. Organize
the exchange of experience between statistical offices in a particular country, to study practical skills in calculating
indices.
- Participation in statistical business register events was highly informative for us.
- To keep organizing regional meetings, additional to global events.
36
4.4 ONLINE SURVEY QUESTIONNAIRE – STAKEHOLDERS/PARTNERS
4.4.1 Introductory Text
This online survey is conducted in the context of the evaluation of the UNECE Project: Enhancing Coherence and
Integration of Economic and Social Statistics in Support of the Implementation of 2008 System of National
Accounts.
The purpose of this survey is to gather the views of the stakeholders and partners who have invested time and
resources to help the UNECE carry out its mandate to support the implementation of 2008 SNA and supporting
statistics in the Eastern Europe, Caucasus and Central Asia region.
The survey, which contains 12 questions, will be conducted in English and take approximately 10 minutes to
complete. All responses are confidential, and no individual can be identified from their responses. UNECE will not
have access to the raw survey results, only to aggregated information. Should you have any questions or concerns,
please contact the evaluator, Constance Graziadei at [email protected].
The survey will be available until 29 September 2023. Thank you in advance for your participation.
6.4.2 Questionnaire
1. Which organization do you represent? Please select all that apply.
• Drop down menu
2. Gender of respondent
• Male
• Female
• Other: _________________
• Prefer not to respond
3.What is your relationship with the project?
• Donor
• Expert
• Oversight
• Regional Partner
• Sponsor
• Other: _________________
4. To what extent did the project respond to the priorities and needs of the beneficiary countries?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
5. How effective was the UNECE in drawing on established partnerships to achieve the results of the project?
• Very effective
• Somewhat effective
37
• Neither effective nor ineffective
• Somewhat ineffective
• Very ineffective
• Do not know
6. To what extent were the project's activities harmonized with activities, events or projects within your
organization?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
7. To what extent were alternative methods used to achieve the results more efficiently?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
8. To what extent are the results of the project sustainable?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
9. To what extent did the results of the activities and events organized by the UNECE align with the SDGs?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
10. To what extent did the UNECE encourage the participation of women, persons with disabilities, minority groups
and youths in the events and activities?
• To a high extent
• To a moderate extent
• To a low extent
• Not at all
• Do not know
11. Are there any lessons learned or good practices from the project that should be considered for future projects?
Text box
12. Do you have any comments or recommendations?
Text box
38
4.5 INTERVIEW GUIDE – KEY STAKEHOLDERS
These questions are based on the evaluation matrix and will be used as a guideline only for interviews with
international and regional partners. The questions will be tailored to each interview conducted. Follow up
questions may be asked to explore emerging themes or discussions.
General
1. What is your relationship with the project and for how long have you been involved?
Relevance
2. To what extent did the project respond to the priorities and needs of the beneficiary countries?
3. To what extent did the activities and outputs from the project align with the project’s stated objectives?
a. Methodology and compilation techniques of 2008 System of National Accounts
b. Price statistics
c. Improving the coverage and accuracy of statistical business registers
d. Expanding the use of household surveys for the estimates of the informal sector and remittances
Effectiveness
4. To what extend did the UNECE contribute to achieving the stated objectives?
5. How effective were the various activities used to achieve the project’s outcomes?
a. Regional workshops
b. Participation in the UNECE Expert Group meetings
c. Sharing of best practices
6. What were the challenges or obstacles to achieving the expected results?
7. What was the impact of the COVID-19 pandemic on planned activities? How was the project realigned to
help participant countries address these issues?
8. How effective was the UNECE in drawing on established partnerships to achieve the results of the project?
9. To what extent were the project activities coherent and harmonized with activities within your
organization?
Efficiency
10. Were the resources sufficient to achieve the desired results?
11. Were alternative methods used to achieve results more efficiently? Please describe.
Sustainability
12. To what extent are the outputs from the various modules sustainable?
a. Methodology and compilation techniques for 2008 System of National Accounts
b. Price statistics
c. Improving the coverage and accuracy of statistical business registers
d. Expanding the use of household surveys for the estimates of the informal sector and remittances
13. To what extent do the results of the project align with the SDGs?
Cross Cutting Questions
14. Was the project designed to ensure inclusion of women, persons with disabilities, minority groups and
youth?
15. Did the UNECE include women, persons with disabilities, minority groups or youth?
16. How can gender and human rights perspectives be better included in the design and implementation of
future projects?
39
4.6 LIST OF PROJECT ACTIVITIES BY MODULE
4.6.1 Module 1 – National Accounts
YEAR ACTIVITY LOCATION
2017 Workshop on Consistency between National Accounts &
Balance of Payments Statistics
Minsk, Belarus
2018 Workshop in compilation, dissemination and use of Supply
and Use tables
Meeting on the Group of Experts on National Accounts
Chisinau, Moldova
Geneva, Switzerland
2020 Webinars of the Group of Experts on National Accounts Online
2021 Workshop on measuring informal economy
Meeting on the Group of Experts on National Accounts
Online
Online
2022 Workshop on Financial Accounts
Meeting of the Group of Experts on National Accounts
Brussels, Belgium
Online
2023 Meeting of the Group of Experts on National Accounts Geneva, Switzerland
4.6.2 Module 2 – Price Statistics
YEAR ACTIVITY LOCATION
2018 Meeting of the Group of Experts on Consumer Price Indices Geneva, Switzerland
2019 Regional Workshop on Consumer Price Indices for EECCA
countries
Minsk, Belarus
2020 Webinars on producing the CPI under lockdown Online
2021 Meeting of the Group of Experts on Consumer Prices Indices Online
2023 Meeting of the Group of Experts on Consumer Prices Indices Geneva, Switzerland
40
4.6.3 Module 3 – Business Registers
YEAR ACTIVITY LOCATION
2017 Meeting of the Group of Experts on Business Registers Paris, France
2019 Meeting of the Group of Experts on Business Registers Geneva, Switzerland
2022 Workshop on Business Registers
Meeting of the Group Experts on Business Registers
Online
Online
4.6.4 Module 4 – Migration Statistics
YEAR ACTIVITY LOCATION
2017 Workshop on Migration Statistics Geneva, Switzerland
2018 Workshop on Migration Statistics
Harmonized survey module on international migration &
remittances
Geneva, Switzerland
2019 Workshop on Migration Statistics
Testing of harmonized survey module on migration &
remittances in countries
Geneva, Switzerland
2020 Analysis of the test results on the survey module
Workshop on Migration Statistics
Online
2021 Harmonized module for measuring & collecting information
on migration & remittances after 2020
Workshop on Migration Statistics
Online
2022 Group of Experts on Migration Statistics Geneva, Switzerland
41
4.7 STAKEHOLDERS INVITED FOR IN-DEPTH INTERVIEW
ORGANIZATION FULL NAME TITLE RELATIONSHIP TO THE
PROJECT
World Bank Mr. Maurice Nsabimana ECASTAT Task Team Leader Donor
ILO Ms. Valentina Stoevska Senior Statistician Group of Experts on CPI
EFTA Mr. Marius Andersen Deputy, EFTA Statistical Office Group of Experts on NA
European Union,
Eurostat
Mr. John Verrinder Head, National Accounts,
Methodology, Standards &
Indicators
Group of Experts on NA
UNSD Mr. Herman Smith Chief of National Accounts Group of Experts on NA
Consultant Ms. Anna Prokhorova Consultant - Developer Harmonized module for
measuring migration and
remittances
UNECE Steering
Group on National
Accounts
Mr. Gerard Eding Director of National Accounts –
Statistics Netherlands
Chair of Steering
Committee
UNECE Steering
Committee on
Consumer Price
Indices
Mr. Christopher Jenkins Assistant Deputy Director,
Prices Division, Office for
National Statistics, UK
Chair of Steering
Committee
Armenia Ms. Lusya Khachartryan Head of Macroeconomics
Indicators and National
Accounts Division, Statistics
Armenia
Group of Experts on
National Accounts
United Kingdom Mr. Sanjiv Mahajan Head of Methods and
Research Engagement, Office
for National Statistics, UK
Steering Group on
National Accounts
42
4.8 LIST OF DOCUMENTS REVIEWED
1. ECASTAT Project for 2017-2020 Implementation of 2008 SNA. Enhancing Coherence and Integration of Economic
and Social Statistics in Support of the Implementation of 2008 SNA – Project Proposal – Concept note 2016
2. ECASTAT Project no-cost extension for 2021-2023. Enhancing Coherence and Integration of Economic and Social
Statistics in Support of the Implementation of 2008 SNA – Project Proposal
3. Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of
2008 SNA https://www.statswiki.unece.org/display/EC/ECSTAT+Home
4. World Bank’s Implementation Completion and Results Report on a Small Grant to the United Nations Economic
Commission for Europe (UNECE) for Enhancing Coherence and Integration of Economic and Social Statistics in
Support of the Implementation of 2008 SNA (Draft-v1)
5. Statistical Commission. Report on the fortieth session (24-27 February 2009).
https://unstats.un.org/unsd/statcom/doc09/Report-English.pdf
6. SDG Priorities. Supporting countries to achieve the SDGs https://unece.org.sdg-priorities
7. United Nations Geneva, 2021 Guide on producing CPI under lockdown
https://unece.org/statistics/publications/buide-producing-cpi-under-lockdown
8. Generic Law on Official Statistics https://unece.org/DAM/stats/publications/2016/ECECESSTAT20163_E.pdf
9. United Nations Fundamental Principles of Official Statistics https://unsd/dnss/hb/E-
fundamentatl%principles_A4-WEB.pdg
10. Interim Unaudited Financial Reports
- June - December 2017
- January - June 2018
- July – December 2018
- January – May 2019
- June - December 2020
- January – June 2021
- July – December 2021
- January – June 2022
11. Reports from Meetings and Workshops
- Group of Experts on Business Registers (2017; 2019; and 2022)
- Group of Experts on Consumer Price Indices (2018; 2021; and 2023)
- Consumer Price Indices webinars (2020)
- Group of Experts on Migration Statistics (2020; 2021; and 2022)
- Group of Experts on National Accounts (2018; 2020; 2021; 2022; and 2023)
- Workshop on Migration Statistics (2019)
- Workshop on Consumer Price Indices (2019)
- Workshop on Financial Accounts (2022)
- Workshop on Measuring Informal Economy (2021)
- Workshop on Migration Statistics (2017; 2018)
- Workshop on Supply and Use Tables for EECCA and SEE countries (2018)
- Workshop on the Implementation 2008 SNA: Consistency between national accounts and balance of
payments (2017)
12. Meeting Evaluation Results
- Workshops of Migration Statistics (2017; 2018; and 2019)
43
- Meeting on Migration Statistics (2021)
- Meetings of the Group of Experts on National Accounts & webinars (2020; 2021; 2022; and 2023)
- Regional Workshop on Consumer Price Indices 2019
- Meetings of the Group of Experts on Consumer Price Indices (2018; 2021 and 2023)
- Meeting of the Group of Experts on Business Registers 2019
13. Semi-annual progress reports
- June – December 2017
- January – June 2018
- July – December 2018
- January – May 2019
- June – November 2019
- December 2019 – May 2020
- June 2020 – December 2020
- January 2021 – June 2021
- July 2021 – December 2021
- January 2022 – June 2022
- July – December 2022
14. Global Assessment of the National Statistical Systems of EECCA (beneficiary) countries, where available:
Armenia (2009), Azerbaijan (2017), Belarus (2020), Georgia (2019), Kazakhstan (2017), Kyrgyzstan (2011), Moldova
(2013), Mongolia (2014), Tajikistan (2013), Turkmenistan (2014), Ukraine (2017), Uzbekistan (Joint review - 2018)
