Skip to main content

Russian Federation

EVAL_Trade_UNDA2023W_MR_2024

Global Initiative towards post-Covid-19 resurgence of the MSME sector (UNDA project 2023W) Period of Review: 2020 - 2023   Date of Evaluation Report: 30/04/2024 

Languages and translations
English

MANAGEMENT RESPONSE

Evaluation Title: Global Initiative towards post-Covid-19 resurgence of the MSME sector (UNDA project 2023W)

Period of Review: 2020 - 2023

Date of Evaluation Report:

30/04/2024

Approved by:

Chief, PMU Nicolas Dath-Baron

Date Signature 05/06/2024

Cleared by:

Director of Divisions: Elisabeth Türk, Economic Cooperation and Trade Division Dario Liguti, Sustainable Energy Division

18/07/2024

Prepared by:

UNECE Project Managers: Hana Daoudi Oleg Dzioubinski

05/06/2024

1

2

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

Recommendation 1:

1. (i) UNCTAD should further leverage the experience gained through the project to map out how components of the Entrepreneurship Policy Framework relate to and can be best positioned to support MSMEs in their recovery from different types of crises and (ii) DESA and the Regional Commissions should add their analysis of how their work can contribute towards the objective.

2. UNCTAD, DESA and the Regional Commissions could build on the project to map their areas of intervention in support of the MSME sector, as well as capacities and knowledge on the implementation of the EPF components, including in a crisis context, and opportunities for broadening the uptake of EPF components at the regional level based on national needs. UNCTAD, DESA and the Regional

Accepted Through existing and future UNDA projects and subject to the availability of funds and the potential for synergies withs UNCTAD, DESA and the remaining regional commissions, UNECE will focus its actions on three key areas:

Integration of evidence-based sectoral insights, focused on the impact of trade disruptions and fluctuations in national, regional and global sustainable development for guiding the development of resilience strategies for MSMEs within the EPF.

Capacity-building: UNECE can help MSMEs implement the strategies outlined in the EPF through training on new technologies, sustainable trade practices to mitigate the effects of crises and capitalize on new market opportunities as they arise.

Strengthening broad-based networks and partnerships, which bring together national and local governments; UNECE sustainable trade expert communities, enterprise

UNECE Economic Cooperation and Trade Division and Sustainable Energy Division

2 years Yes

3

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

Commissions could further consider identifying areas of joint interventions that would trigger complementarities and synergies between the agencies. This could involve collaborating on the development of capacities of all national and sub-national actors, i.e. Governments, MSMEs, other partners (e.g. Chamber of Commerce, Business Incubators, etc.) in line with United Nations Sustainable Development Cooperation Frameworks (SDCF) in respective countries; strengthening or contributing to facilitate access to finance for the MSMEs (including seed money, grants, access to credit, etc.); improving the coordination of MSMEs related policies across ministries; increasing interventions at the local level, such as by supporting NGOs or MSMEs outside of the main cities. The application of a human rights-based approach, gender responsiveness and

support organizations working with UNECE and MSMEs to support the practical application of the EPF.

4

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

inclusion of other vulnerable groups (LNOB) should be ensured.

Recommendation 2:

UNCTAD should continue building on the momentum generated by the project to continue fostering knowledge exchanges and promoting the Entrepreneurship Policy Framework.

UNCTAD, DESA and the Regional Commissions should identify means to more meaningfully continue to share good practices and lessons learned on the demand-driven support they provide to the MSME sector, including on areas such as green/circular economy; innovation policies and ecosystems; MSME formalization and strengthening MSME capacities for the empowerment and leadership of women, youth and groups in vulnerable situations. UNCTAD and the Regional Commissions could also

Accepted Through existing and future UNDA projects and subject to the availability of funds and the potential for synergies with UNCTAD, DESA and the remaining regional commissions, UNECE’s actions can be framed around the following activities:

Facilitating knowledge exchange by participating in, and possibly co-hosting national and regional events for discussing emerging issues and sharing national and regional experiences.

Promoting best practices by leveraging its extensive network and expertise to compile and disseminate case studies and success stories from its member states.

Supporting national, regional and sector- specific initiatives that align with the EPF, including contributing to the development of policies and programs that foster innovation, the transition to a circular

UNECE Economic Cooperation and Trade Division and Sustainable Energy Division

2 years Yes

5

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

propose organizing regional events and/or a global conference to promote entrepreneurship policy and further advance the goals and impact of the Surge project. Furthermore, learnings from the experiences with the Surge about external partnerships should encourage UNCTAD to promote the EPF and entrepreneurship development to other UN agencies also engaged in this area (e.g. ILO, FAO, UN Women, etc.), to development banks, or to related initiatives such as the recent Global Accelerator on Jobs and Social Protection. UNCTAD could also consider joining and supporting events that promote the achievement of SDG 8.3 as an avenue to further promote the EPF.

economy, and enable MSMEs to integrate sustainability practices throughout their supply chains.

Recommendation 3:

The DA-PMT should develop a clear framework for assessing the costs and benefits of implementing a global or

6

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

inter-regional project versus regional projects.

As a global crisis can affect regions and countries differently, global or joint projects should be developed only when there are clear benefits of joint implementation. Thus, a global or inter-regional vs. regional response would not be a priori decision, but a result of a clear assessment. A specific guideline or framework should be developed for this. Some of the assessment criteria could include the need or demand from member States for an integrated response; the range of common versus entity-specific activities and added value of complementary interventions (i.e. in terms of geographical coverage, reach of the target population, capacity, and/or coverage of multiple reinforcing technical areas, or networks and partnerships); capability to execute joint interventions (e.g. in terms of time, resources, logistics, and flexibility to

7

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

pivot interventions without bureaucratic procedures); coordination costs (which increase with the number of participating UN entities); project inception modalities and governance and knowledge management requirements to facilitate collaboration and knowledge sharing; scaling and sustainability plans; expected results of spreading resources versus concentrating on fewer countries; and so forth. The time taken to conduct such assessments will increase project coherence and effectiveness and facilitate the identification of the financial and human resources needed for project implementation. The network of DA Focal Points is a key existing asset for this assessment.

Recommendation 4:

Implementing UN entities should ensure that they have a comprehensive Results Framework for the entire project as well as an

Accepted This guidance already exists for UNECE projects, whether funded from XB or UNDA. UNECE divisions will continue to apply and enrich in future projects.

UNECE Economic Cooperation and Trade Division and Sustainable Energy Division

8

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

adequate monitoring plan, with indicators that are designed to support the ongoing monitoring.

The results framework should have one objective and ideally have one outcome per cluster/workstream. The objective should state the intended goal of the project, describe the overall achievement targeted by the project, involving a process of change aimed at meeting the needs of identified beneficiaries, and reflect the overall funding available to the project. Each objective should include reference to the project’s beneficiaries and its substantive focus. The objective should not attempt to explain the ways in which the project intends to achieve the objective (i.e. it should not include the word ‘through’ or describe the internal work of the UN using verbs such as ‘support’, ‘facilitate’ or ‘contribute’). The outcomes (OCs) should describe the changes that are expected to occur as a result of the completion of outputs.

9

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

The OCs should be achievable within the project’s timeframe and budget, and should be specific enough to be measured by the associated indicators of achievement. The indicators of achievement (IAs) should provide measures for monitoring progress towards achieving the OCs and reporting on them after completion of the project. Every indicator needs to provide clearly defined baselines, units of measurement and targets, detailing the quantity, quality and timing of expected results. The monitoring, evaluation, accountability, and learning (MEAL) system should also be designed to capture HRBA, gender and LNOB aspects. In phased interventions or during project execution, any changes in the logframe if/when pivoting activities should be clearly explained to the wider team.

Recommendation 5: Accepted This guidance already exists for UNECE projects, whether funded from XB or

UNECE Economic Cooperation and

10

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

Implementing UN entities should ensure that sufficient resources are allocated to project coordination, technical collaboration, and partnership building.

The absorption capacity of implementing entities can be challenged by crisis response projects which add to the planned programme of work. This is further compounded by projects that come with an extensive UN partnership and a global scope. Sufficient resources should be dedicated to global coordination and to building global partnerships with strategic stakeholders (e.g., UN organisations engaged in supporting the MSME sector; development banks). Capacities should also be directed to supporting technical collaboration and the staff implementing interventions, including towards synergy or liaison with the UNCTs. When designing the project, implementing UN entities should consider featuring

UNDA. UNECE divisions will continue to apply and enrich in future projects.

Trade Division and Sustainable Energy Division

11

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

coordination and partnership- building in the Theory of Change or logframe of the project. Tools to support continuous connections and knowledge exchange, and to ensure institutional memory should be part of the response package, such as a project website, SharePoint space for all team members, and a Yammer network or Teams channel. The integration of cross-cutting aspects (HRBA, gender responsiveness, LNOB) also requires expertise with sufficient and dedicated time and resources. Guidance could be development to project managers on how to do this.

EVAL_Trade_UNDA2023W_ToR_2024

Evaluation Terms of Reference  Evaluation of the United Nations Development Account 12th tranche “Global Initiative towards post-Covid-19 resurgence of the MSME sector” (2023W)  

Languages and translations
English

Annex 7: Evaluation Terms of Reference Evaluation of the United Nations Development Account 12th tranche “Global Initiative towards post-Covid-19 resurgence of the MSME sector” (2023W)

TERMS OF REFERENCE

A. BACKGROUND A1. ABOUT THE DEVELOPMENT ACCOUNT

The Development Account (DA) is a mechanism to fund capacity development projects of the 10 economic and social entities of the United Nations Secretariat, namely: the Department of Economic and Social Affairs (DESA), the Economic Commission for Africa (ECA), the Economic Commission for Europe (ECE), the Economic and Social Commission for Asia and the Pacific (ESCAP), the Economic and Social Commission for Western Asia (ESCWA), the Economic Commission for Latin America and the Caribbean (ECLAC), the United Nations Conference on Trade and Development (UNCTAD), the United Nations Environment Project (UNEP), the United Nations Human Settlements Project (UN-Habitat) and the United Nations Office on Drugs and Crime (UNODC). The DA provides capacity development support to developing countries in their implementation of the 2030 Agenda for Sustainable Development, as well as recommendations and decisions made in intergovernmental processes and relevant governing bodies. The DA-funded projects build on the mandates, individual technical capacities and comparative advantages of the respective implementing entities, while providing those mostly non-resident entities with the ability to operationalize their knowledge and know-how to deliver capacity development support at regional, sub-regional and country levels. The Under-Secretary-General (USG) for Economic and Social Affairs is designated as the Project Manager of the Development Account with responsibility for overall coordination, programming, monitoring and evaluation, as well as for reporting to the intergovernmental bodies. The Project Manager is supported by the DA Steering Committee, who advises him/her on strategic policy and project-support matters.166 The Project Manager is also supported by the DA Project Management Team (DA-PMT) located within the Capacity Development Programme Management Office (CDPMO) of DESA, which assists with all aspects of the management of the DA, in particular with regard to programming, monitoring, evaluation and reporting. DA-PMT also liaises with the DA Focal Points in the implementing entities, who are most often the head of the entity’s unit responsible for project planning, project management, capacity development or technical cooperation, on all aspects of the management of DA-funded projects. In response to the COVID-19 crisis, the Development Account has funded five short-term joint projects to help developing countries alleviate the socio-economic impact of the pandemic, including the project on Global initiative towards post-COVID-19 resurgence of the MSME sector (2023W).

A2. ABOUT THE PROJECT The COVID-19 crisis is plunging the global economy into a deep recession and micro, small and medium enterprises – which play a major role in emerging economies – are amongst the hardest hit. Trapped in economic stagnation due to large-scale lockdowns, millions of MSMEs have become the most vulnerable to COVID-19 within the private sector. Compared with large firms, small businesses have fewer resources and lower capacities to cope with the abrupt economic shocks economies are currently facing. With more than two-thirds of the global population employed by MSMEs, the unprecedented outbreak of the pandemic has vividly shown how tightly their activities are woven into the economic and social fabric of the world, as well as their critical role in social and economic resurgence. The objective of the project is to develop and implement capacity-building tools for governments and micro, small and medium enterprises (MSMEs) to facilitate the resurgence and strengthen the resilience of MSMEs in developing countries and economies in transition. These capacity-building tools will seek to mitigate the

economic and social impact of the global COVID-19 crisis and to facilitate the contribution of MSMEs to the SDGs implementation. The project was designed based on the request for assistance for MSMEs from more than 50 Member States, including countries in Latin America and the Caribbean, Asia and the Pacific, Africa, Europe and the Arab regions, as well as intergovernmental demands and resolutions on COVID-19. The project is jointly implemented by UNCTAD, DESA, ECA, ECE, ECLAC, ESCAP and ESCWA, and builds upon the comparative advantages of the participating agencies to provide immediate advice, capacity-building and support to governments and MSMEs during the ongoing global pandemic. The project is structured in five clusters that address the most critical areas of the MSME recovery. Broadly, the roles and lead entities for each of the clusters and workstreams are as presented in the following table:

Table 1. Project clusters and leads Project cluster/workstream Lead agency

Overall coordination UNCTAD

Project cluster/workstream 1 Entrepreneurship and business skills promotion

UNCTAD

Project cluster/workstream 2 Business facilitation/formalization

UNCTAD/DESA

Project cluster/workstream 3 Access to finance/financial literacy

ESCAP

Project cluster/workstream 4 Access to technology and innovation

UNECA

Project cluster/workstream 5 Access to markets

UNECE

The beneficiary countries cover different geographical regions, as shown in Annex 2. The expected outcomes, indicators of achievement, and outputs are presented in the project results framework (Annex 3). The project was developed and implemented under three phases. A new set of outputs was designed or added at each of the three phases of the project. Under the three-phase approach, the project budget was approved by phase. In 2021, when the phase 3 budget was discussed, the Development Account faced a funding gap. To bridge the gap, in November of the same year, the five joint projects were requested to reduce their proposed phase 3 budget by 1 million USD, which led to the curtailment of certain planned activities. For this project, the budget was reduced by $310,000. Overall, a total of $4,490,500 was allocated under this project. Concretely, UNCTAD received $2,671,000, ECLAC received $134,000, ESCAP received $240,000, ECA received $467,000, ECE received $448,500, ESCWA received $370,000 and DESA received $160,000. The project started its implementation in May 2020 and was scheduled to conclude on 31 March 2022, but received approval in February 2022 for an extension until 30 June 2022.

B. EVALUATION OBJECTIVES AND SCOPE B1. EVALUATION PURPOSE, OBJECTIVES AND SCOPE

The present evaluation will constitute a terminal evaluation of the Project. Terminal evaluations are mandatory for all DA-funded projects with a value above $1 million. The evaluation will be largely guided by the UN Development Account Project Evaluation Guidelines, issued in October 2019 and the evaluation policies of the implementing entities, in particular, UNCTAD, which leads the evaluation. The main purpose of the evaluation will be to support accountability for results, and to enable learning. This terminal evaluation of the project has the following specific objectives:

• Assess the results and establish the link between achievements and activities of the intervention;

• Assess the response delivery and external coordination167, including the extent of gender, human rights and disability mainstreaming; and • Identify good practices and lessons learned from the project that could feed into and enhance the implementation of related interventions.

The primary intended users of the assessment are the management of the implementing entities. The evaluation will also provide accountability to project beneficiaries and member States. Furthermore, the evaluation will form a key input to the programme-level evaluation of the DA’s response to COVID-19 to be initiated by the CDPMO/DESA. The programme-level evaluation will entail: a synthesis of the terminal evaluations of five COVID-19 joint DA projects, including this project; a review of relevant 10th and 11th tranche DA projects; and a programme-level assessment. The primary audiences of the programme-level evaluation will include the DA Steering Committee, the DA-Programme Management Team (DA-PMT), and the management of the implementing entities. The results of the programme-level evaluation will also be presented to the General Assembly, through the biennial progress report on the implementation of the DA. The evaluation will cover the duration of the project from May 2020 to 30 June 2022, covering all phases, clusters and activities.

B2. EVALUATION CRITERIA AND QUESTIONS The evaluation will assess the Project’s performance against the main criteria of relevance, efficiency, effectiveness, coherence, sustainability, gender, human rights and disability. In particular, the evaluation is expected to address a number of questions under the following criteria168:

Table 2: Evaluation criteria and tentative questions Relevance 1. To what extent was the project designed to target the new

needs and priorities of participating countries as a result of COVID- 19?

Relevance 2. To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g. COVID-19 Socio-Economic Response Plan)?

Efficiency 3. How well coordinated was the response among the entities implementing the joint project?

Efficiency 4. How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

Effectiveness 5. To what extent did the programme (Development Account) and project governance and management structures and processes enabled, or hindered, the effective implementation of the joint project and the achievement of its results?

Effectiveness 6. To what extent has the project contributed to the expected outcomes as enunciated in the project document?

Effectiveness 7. How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

Effectiveness 8. What innovative approach or tool, if any, did the response use, and what were the outcomes and lessons learned from its application?

Sustainability 9. What measures were adopted to ensure that the outcomes of the response would continue after the project ended?

Coherence 10. To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

Coherence 11. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID-19 in delivering socio-economic support to Member States?

Gender, human rights and disability

12. To what extent were gender, human rights and disability perspectives integrated into the design and implementation of the project? What results can be identified from these actions?

C. EVALUATION APPROACH AND METHODOLOGY The evaluation will be a transparent and participatory process involving the Project’s implementing entities and key stakeholders. It will be conducted based on gender and human rights principles and adhere to the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation. The evaluation will apply a mixed-method design, including a combination of quantitative and qualitative analysis to inform findings.It is anticipated that travel of the evaluation team may take place in support of elaborating case study/ies, as well as to meet key project stakeholders in Geneva (UNCTAD and ECE). The selection of potential case study/ies and travel requirements will be developed as part of the inception report. Following a preliminary documentation review and a limited number of inception meetings with the core project team, the Evaluation Team will develop an inception report for the evaluation, which will include the finalized overall scope and focus of the evaluation, evaluation questions and methodology, including information on data sources and collection, sampling, key indicators, stakeholder mapping, selection of case study/ies, survey design, and the evaluation timeline. The tentative methodology for the evaluation is presented in Table 4.

Table 3: Tentative methodology for the assessment

a. A desk review of Project documents, including documents/data related to: o Project-level planning, implementation and results achievement, including but not limited to:

▪ Concept note, Phase 2 project proposal, and Phase 3 budget and outputs ▪ Progress report for Phases 1 and 2 (both financial and substantive/narrative report) ▪ Final report (both financial and substantive/narrative report) ▪ Meeting minutes, including the minutes of the bi-weekly/monthly DA network meetings ▪ Monitoring reports ▪ Information on non-DA resources, financial and in-kind, brought in by the participating entities ▪ Information on resources, financial and in-kind, contributed by partners/donors (including information requested under the “supplementary funding” section in the progress reports, which is often incomplete) ▪ Beneficiary/user feedback collected, including, but not limited to, workshop survey results, user feedback on publications, advisory services, guidelines, methodology documents, etc. ▪ Requests for assistance/services received ▪ List of activities completed and details about each activity, including but not limited to:

▪ Agenda, participant lists (name, title, division/unit, organization, country, gender, email address), report and any outcomes document, for each workshop/meeting ▪ Description of each advisory service, beneficiaries (including contact details of the contact persons) and any outputs/deliverables produced ▪ List and description of tool(s), research papers, policy briefs, studies published and information on how each product was disseminated and/or used, list of recipients/users of the product (e.g., dissemination lists)

▪ Documentation related to broader projects or sub-projects of the participating entities of which the Project or its component(s) has constituted an integral part or which are linked to and/or build upon/succeed the work undertaken as part of the Project ▪ Documents and literature related to the Project context ▪ Relevant web and social media metrics related to the outputs of the project;

o Project strategic documents, including but not limited to: ▪ General Assembly's Resolution on Global Solidarity to fight the coronavirus disease 2019 (COVID-19) (A/RES/74/270);

▪ Secretary General's report on "Shared responsibility, global solidarity: Responding to the socio-economic impacts of COVID-19"; ▪ UN Framework for the immediate socio-economic response to COVID-19; ▪ 2021 Programme budget and mandate of implementing entities; ▪ COVID-19 Socio-Economic Response Plan of participating countries.

b. Questionnaires/surveys (in appropriate languages in addition to English) to relevant stakeholders in countries participating in a sample of project activities; c. Telephone, online or in-person interviews with key stakeholders, including but not limited to:

o Project Coordination Team and project focal points of implementing entities o DA-PMT o DA focal points in participating entities o Sample of UN Resident Coordinators/Country Teams, as appropriate o Sample of key global partners o Sample of country-level stakeholders (mainly stakeholders from key beneficiary countries)

d. Case Study/ies, which may include for example a detailed examination of a particular intervention, or of project activities at a regional or national level.

In addition to assessing the mainstreaming of gender, human rights and disability perspectives in the design, implementation and monitoring of the Project (evaluation question 12), the evaluation will integrate these perspectives in the management of the evaluation, data collection and analysis, as well as the development of the evaluation report. Gender balance will be given full consideration in the composition of the Evaluation Reference Group, elaborated in Section D1 (Evaluation management), and the Evaluation Team. Data collected and analyzed in the course of the evaluation will be disaggregated by gender to the extent possible and whenever appropriate, and the evaluation findings, conclusions and recommendations to be presented in the evaluation report will reflect a gender analysis. The evaluation will be carried out according to the UNEG ethical principles and standards.169 The evaluators should demonstrate behavioural independence, impartiality, credibility, honesty, integrity and accountability in conducting the evaluation/assessment to avoid biasing the findings. The evaluators must also address in the design and conduct of the evaluation procedures to safeguard the rights and confidentiality of information providers. The Evaluation Manager will be responsible for ensuring that the Evaluation Team conducts the work assignments without any undue interference from those who were responsible for the implementation of the Project.

D. ORGANIZATION OF THE EVALUATION D1. EVALUATION MANAGEMENT

The independent final project evaluation will be managed/coordinated by UNCTAD’s Independent Evaluation Unit, with the support of an Evaluation Advisory Committee (EAC) that comprises a representative each of the evaluation units of the partner entities (DESA, ECA, ECE, ECLAC, ESCAP, ESCWA) and the Evaluation Officer with the CDPMO/DESA. The EAC primarily serves a quality assurance function and facilitates support to the Evaluation Team as necessary. An Evaluation Reference Group (ERG) consisting of a representative from each UN partner entity (e.g., the DA Focal Point for each entity) and the DA-PMT will review and contribute inputs to key steps in this evaluation such as the TOR and draft final report. Both the EAC and the ERG commit to submitting substantive comments on a timely basis, and comments will be invited on a ‘non-objection’ basis (no response = agree) so that the process is not delayed for an unnecessarily long time. An independent Evaluation Team will be convoked to undertake this assignment. The Evaluation Team (ET) is responsible for conducting the evaluation, applying the methodology as appropriate and for producing the evaluation report. All team members, including the Team Leader, will participate in briefing and debriefing meetings, discussions, and will contribute to the evaluation with written inputs for the draft and final report. A selected number of the evaluation team members will participate in the mission travel(s) if applicable. The ET and the Evaluation Manager will agree on the outline of the report, in consultation with

the EAC early in the evaluation process. The ET will develop its own evaluation tools and framework, within the available timeframe and resources. The team is fully responsible for its report, which may not reflect the views of any of the implementing entities of the project. The evaluation report is subject to quality control by the Evaluation Advisory Committee and clearance by the Evaluation Manager, as set out above. The Team Leader guides and coordinates the team member(s) in their specific work, discusses their findings, conclusions and recommendations and prepares the draft and the final report, consolidating the inputs from the team member(s) with his/her own. The members of the evaluation team should possess a mix of evaluation skills and technical or sectoral/thematic knowledge relevant to the evaluation. In putting together the team, adequate linguistic, geographic and gender representation will also be key considerations. The Evaluation Team will be provided full access to all project reports, documentation, and stakeholder lists and contact information. The Project Coordination Team are required to submit to the evaluation manager project documentation, including data and information residing with the other participating entities, in the last month of the project if possible, if not, immediately following the completion of the project, as well as support the evaluation process, including through facilitating the evaluators’ access to the project’s beneficiaries and other key stakeholders. The roles and responsibilities in the evaluation process are described below:

Evaluation Manager (UNCTAD) will: • Prepare the draft evaluation TOR and revise/finalize based on inputs received • Prepare the TOR for each member of the Evaluation Team (Team Leader, Team Member and Expert(s)) • Recruit and manage the Evaluation Team • Backstop the evaluation process, including supporting the development and administration of surveys, support outreach of the evaluation team to project stakeholders, and access to secondary data listed in Table 3. • Oversee/provide quality assurance to the evaluation and the development of the evaluation report • Facilitate the work of the Evaluation Advisory Committee and the Evaluation Reference Group • Be responsible for clearance of the evaluation report • Support the development of a management response to the evaluation report, including an implementation plan • Organise a virtual workshop on evaluation findings and lessons learned.

Evaluation Advisory Committee comprises a representative each of the evaluation units of the partner entities (ESCWA, ECA, ECE, ECLAC, ESCAP, DESA) and the Evaluation Officer with the CDPMO/DESA. The EAC primarily serves a support and quality assurance function. Specific responsibilities of the EAC include:

• Review and approve the evaluation TOR; • Advise on the selection of the evaluation consultant(s) to ensure that the selection is based on the required skills and qualifications; • Support facilitating access from their respective entities to relevant project documentation and stakeholders; • Review and comment on the inception and evaluation reports; and • Monitor and conduct periodic follow-ups on the implementation of evaluation recommendations addressed to the parties within their entities.

Evaluation Reference Group, consisting of a representative from each UN partner entity (e.g., the DA Focal Point for each entity) and the DA-PMT,170 will review and contribute inputs to key steps in this evaluation such as the TOR and draft final report. The ERG’s key function is to enhance the relevance, credibility and transparency of the evaluation process. Specific responsibilities include:

• To review the draft evaluation ToR and provide substantive feedback; • To facilitate access from their respective entities to relevant project documentation and stakeholders;

• To review the draft evaluation report and provide substantive feedback, including coordinating feedback from other sections, units and offices from headquarters and from the field to ensure quality and completeness; • To participate in the validation meeting of the final evaluation report; • To play a key role in disseminating the findings of the evaluation and implementation of the management response.

Project Coordination Team will: • Facilitate the Evaluation Team’s access to relevant Project documentation and stakeholders, including through:

o Collecting and compiling requested data and information from the participating entities, as requested by the Evaluation Manager o Providing an updated list of stakeholders, and facilitating access to the sample of stakeholders that the Evaluation Team may wish to interview o Facilitating the administration of questionnaires to workshop participants in the participating countries o Ensure the cooperation and contribution of the relevant staff of the implementing entities to the evaluation process, as requested

• Lead the preparation of a response to the recommendations directed to the participating entities, including an implementation plan

DA-PMT will: • Participate in the Evaluation Reference Group • Provide guidance on the allocation of the evaluation budget • Organize a virtual meeting with DA focal points to discuss the key lessons from this evaluation as well as from other COVID-19 joint project evaluations and how to incorporate them in future programming, implementation, monitoring and evaluation of DA-funded projects.

D2. EVALUATION TIME FRAME

The evaluation will be conducted from December 2022 to August 2023. The evaluation process will involve five phases with the tentative timelines as below in Table 5 (the timelines may be adjusted should any exigencies arise):

Table 5: Evaluation phases and tentative timelines Phase Timelines

1. Preparation

August 2022 – December 2022 • Preparation and finalization of evaluation TOR • Establishment of the Evaluation Reference Group • Recruitment of the Evaluation Team • Prepare package of documents required by the Evaluation Team

2. Inception

December 2022 -March 2023 • Preliminary documentation review and preparation of inception report by the Evaluation Team, including development of data collection instruments (questionnaires/ surveys, interview guides) • Brief visit to Geneva (3 days) to meet with UNCTAD Evaluation Unit and key project stakeholders (UNCTAD and ECE) • Draft inception report due: 10 February 2023

• Evaluation Manager review and inception report revision by Evaluation Team: 13 - 22 February 2023 • Reviews by Project Coordination Team, project focal points of implementing entities, and Evaluation Advisory Committee (in parallel): 23 February – 7 March 2023 • Draft final inception report due: 14 March 2023 • Final inception report approved: 20 March 2023

3. Data collection and analysis March- May 2023 • Desk review of remaining Project documents, including requesting additional documentation • Online surveys of stakeholders • Interviews with stakeholders • Data analysis and triangulation

4. Report preparation and reviews May – July 2023 • Data analysis and triangulation • First draft evaluation report due: 16 June 2023 • Evaluation Manager review and report revision by the Evaluation Team: 19 – 28 June 2023 • Reviews by Project Coordination Team, project focal points of implementing entities, Evaluation Advisory Committee and Evaluation Reference Group (in parallel): 29 June – 11 July 2023 • Revised draft evaluation report due: 18 July 2023 • Final evaluation report with annexes: 25 July 2023

5. Dissemination and follow-up August 2023 and onwards • Presentation to the Project Coordination Team, project teams of implementing entities and development and approval of a management response, including an implementation plan for recommendations • Virtual workshop on evaluation findings, lessons learned and follow-up with the DA Focal Points: April 2023

D3. EVALUATION TEAM DELIVERABLES The Evaluation Team will be composed of a team of three consultants (evaluators), namely Team Leader, Team Member and a Gender and Human Rights (HRGE) Expert who also plays the role of Team Member. The two Team Members will report functionally to the Team Leader. The Team Leader will report to the Evaluation Manager. Each of the Evaluation Team has a set of deliverables as described below: Deliverables for Team Leader and Team Member

o Initial review of key Project documents (preliminary document review) o Preparation of an inception report with a finalized evaluation scope and focus, evaluation questions and methodology, including information on data sources, sampling and key indicators, stakeholder mapping/analysis, selection of case study/ies, as well as survey design

o Desk review of remaining Project documents o Data collection and analysis based on the finalized methodology o Preparation of an evidence matrix presenting a summary of evidence collected through each data collection method by evaluation question o Development of a draft evaluation report, based on the template presented in Annex 1, for review by the Evaluation Manager, Project Coordination Team, project focal points of implementing entities, the EAC and the ERG o Revision/finalization of the evaluation report, including all annexes, based on comments received o Preparation of a 3-page summary of the evaluation report and a presentation (PPT) on key findings, conclusions and recommendations o Presentation of evaluation report and discussions with relevant stakeholders such as Project Coordination Team, project teams of implementing entities, DA focal points of participating entities and DA-PMT.

Deliverables for Gender and Human Rights Expert/Team Member

o Initial review of key Project documents (preliminary document review), including identifying gender equality, human rights and disability inclusion dimensions and issues for consideration; o Preparation of an inception report with a finalized evaluation scope and focus, evaluation questions and methodology, including information on data sources, sampling and key indicators, stakeholder mapping/analysis, selection of case study/ies, as well as survey design. Where applicable, gender equality, human rights and disability inclusion considerations will be integrated in the evaluation scope of analysis; evaluation criteria and questions design; methods and tools, and data analysis techniques; o Desk review of remaining Project documents; o Data collection and analysis based on the finalized methodology which would be gender sensitive; o Preparation of an evidence matrix presenting a summary of evidence collected through each data collection method by evaluation question; o Development of a draft evaluation report, based on the template presented in Annex 1 of the Terms of Reference for the evaluation, for review by the Evaluation Manager, Project Coordination Team, project focal points of implementation entities, the EAC and the ERG. The analysis of gender equality, human rights and disability inclusion dimensions and issues should be integrated in the report as an independent section and to the extent possible, these issues should be mainstreamed throughout the report, including in the evaluation findings, conclusions and recommendations. o Revision/finalization of the evaluation report, including all annexes, based on comments received; o Preparation of a 3-page summary of the evaluation report and a presentation (PPT) on key findings, conclusions and recommendations. o Presentation of evaluation report and discussions with relevant stakeholders such as Project Coordination Team, project teams of implementing entities, DA focal points of participating entities and DA-PMT.

E. COMMUNICATIONS AND DISSEMINATION PLAN: The results from the evaluation including key lessons learned, best practices and recommendations will be shared widely with participating entities, partners and stakeholders, and member States. In particular, the following modes of communication could be used:

e. A workshop with all relevant stakeholders to present the key findings, recommendations and lessons learned. The evaluation report will be presented at a workshop attended by the implementing entities, the DA-PMT and other relevant stakeholders for discussion and validation. The implementing

entities will be given the opportunity to present their management response, including an implementation plan for the recommendations; f. A separate virtual meeting will be held with the DA focal points to discuss the key lessons from the evaluation as well as from other COVID-19 joint project evaluations and how to incorporate them in future programming, implementation, monitoring and evaluation of DA-funded projects and projects. g. A copy of the final evaluation report will be published on UNCTAD’s website and the websites of the partner implementing entities, as appropriate; and h. The key findings from the evaluation report will also form a key input to the programme-level evaluation of the DA’s response to COVID-19 to be initiated by the CDPMO/DESA. i. Other communication briefs and products will be produced as appropriate.

EVAL_Trade_UNDA2023W_EvalReport_April2024

United Nations Development Account Terminal Evaluation of Project 2023W “Global Initiative towards post-Covid-19 resurgence  of the MSME sector” (2020-2023) Report completed: April 2024 Evaluation conducted by: Patrick Breard, Ariane Agnes Corradi, Jude Kallick 1 

Languages and translations
English

1

United Nations Development Account

Terminal Evaluation of Project 2023W

“Global Initiative towards post-Covid-19 resurgence of the MSME sector”

(2020-2023)

Report completed: April 2024

Evaluation conducted by: Patrick Breard, Ariane Agnes Corradi, Jude Kallick

2

Evaluators: Patrick Breard, PhD, Team Leader

Ariane Agnes Corradi, PhD, Team Member Jude Kallick, MA, Team Member

Evaluation Manager: Nishta Keeble, Chief, Independent Evaluation Unit, Office of the Secretary-

General, UNCTAD

Contact: Thomas Callaghan, Associate Programme Management Officer, Independent Evaluation Unit, Office of the Secretary-General, UNCTAD. Tel: +41 22 917 5295 Email: [email protected] This report was commissioned by UNCTAD. The findings, conclusions and recommendations of this report are

those of the external evaluator and do not necessarily reflect the views of the UNCTAD.

3

Table of Contents

Lists of figures, tables, and boxes.............................................................................................................................................. 4

List of Acronyms and Abbreviations ........................................................................................................................................ 5

Executive summary ....................................................................................................................................................................... 6

1. Introduction .................................................................................................................................................................................11

2. Description of the Project........................................................................................................................................................11

2.1 Background ...........................................................................................................................................................................11

2.2 Project objectives and expected accomplishments ................................................................................................ 12

2.3 Project strategies and key activities .............................................................................................................................13

2.4 Beneficiaries and target countries................................................................................................................................. 14

2.5 Key partners and other key stakeholders .................................................................................................................. 16

2.6 Resources ........................................................................................................................................................................... 16

2.7 Link to the Sustainable Development Goals (SDGs) ................................................................................................ 17

2.8 Innovative elements ......................................................................................................................................................... 17

3. Evaluation objectives, scope, and questions .................................................................................................................... 17

3.1 Purpose and objectives .................................................................................................................................................... 17

3.2 Evaluation scope, criteria, and questions .................................................................................................................. 18

4. Methodology ............................................................................................................................................................................. 19

5. Findings ....................................................................................................................................................................................... 21

5.1 Relevance ............................................................................................................................................................................. 21

5.2 Coherence .......................................................................................................................................................................... 24

5.3 Efficiency ............................................................................................................................................................................ 29

5.4 Effectiveness ...................................................................................................................................................................... 32

5.4 Sustainability .................................................................................................................................................................... 59

5.5 Gender, Human Rights, and Leave No One Behind ................................................................................................ 64

6. Conclusions ............................................................................................................................................................................... 73

7. Recommendations .................................................................................................................................................................. 76

Annexes .......................................................................................................................................................................................... 78

Annex 1: Project’s Theory of Change ................................................................................................................................. 79

Annex 2: Evaluation matrix.................................................................................................................................................. 80

Annex 3: Data collection instruments .............................................................................................................................. 86

Annex 4: List of documents reviewed ............................................................................................................................... 97

Annex 5: List of individuals interviewed .......................................................................................................................... 99

Annex 6: Evaluation surveys ............................................................................................................................................. 100

Annex 7: Evaluation Terms of Reference ........................................................................................................................ 134

4

Lists of figures, tables, and boxes List of Figures Figure 1: Overview of project activities 13 Figure 2: Allocation of project funding 16 Figure 3: Perceived relevance of the project according to the survey 22 Figure 4: Project outputs per implementing UN entity and phase 27 Figure 5: Number of project outcomes covered by the implementing UN entities 28 Figure 6: Perceived complementarity of the project with other interventions 29 Figure 7: Surge Project outputs delivered per implementing entity and implementation phase 37 Figure 8: DEPAR number of users 38 Figure 9: Effectiveness results of outcome 1.A 38 Figure 10: Effectiveness results of outcome 1.B 40 Figure 11: Effectiveness results of outcome 3 45 Figure 12: Effectiveness results of outcome 4 47 Figure 13: Effectiveness results of outcome 5 48 Figure 14: Outputs per project outcome and implementing entity 53 Figure 15: Project contributions to formulate and implement enabling policies to MSMEs 54 Figure 16: Contribution of the project to improve the resilience and competitiveness of MSMEs in post-. COVID-19 resurgence in a sustainable way 60 Figure 17: Contribution of the project to better outcomes for women and youth entrepreneurs and other vulnerable groups in a sustainable way 61 Figure 18: Contribution of the project to formulate and implement enabling policies for MSME post-COVID-19 resurgence in a sustainable way 61 Figure 19: Contribution of the project to improve the resilience and competitiveness of MSMEs in post-COVID-19 resurgence in a sustainable way 62 Figure 20: Perceived accessibility of project activities for women, youth, people with disabilities or other vulnerable groups 69 Figure 21: Perceived contribution of the project to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups 70 List of Tables Table 1: Project’s beneficiary countries 15 Table 2: Project clusters and lead and participating entities 16 Table 3: Project's implementation phases 17 Table 4: Evaluation questions 18 Table 5: Groups of informants and number of consultations 19 Table 6: Survey recipients and respondents 20 Table 7: Planned and delivered outputs per project phase 37 Table 8: Level of achievement of the Surge project indicators 50 List of Boxes Box 1: ECE’s guidelines and best practices – from a general approach to supporting country-specific responses 55 Box 2: UNCTAD Cross-border Trade and Gender Initiative 68

5

List of Acronyms and Abbreviations

ADB Asian Development Bank

AfDB African Development Bank

DA Development Account

DESA Department of Economic and Social Affairs

EAC Evaluation Advisory Committee

ECA Economic Commission for Africa

ECE Economic Commission for Europe

ECLAC Economic Commission for Latin America and the Caribbean

EMPRETEC Emprendedores (entrepreneurs) and tecnología (technology)

ERG Evaluation Reference Group

ESCAP Economic and Social Commission for Asia and the Pacific

ESCWA Economic and Social Commission for Western Asia

ET Evaluation Team

GDP Gross Domestic Product

GHRP Global Humanitarian Response Plan

HIC High-income Country

HRBA Human Rights-Based Approach

ICT Information and Communication Technology ILO International Labour Organization

IMF International Monetary Fund

LDC Least Developed Country

LIC Low-income Country

LLDC Landlocked Developing Country

LNOB Leaving No One Behind

MIC Middle-income Country

MNE Multinational Enterprise

MPTF Multi-Partner Trust Fund

MSME Micro-, Small and Medium Enterprises

PMT Project Management Team

PRODOC Project Document

RC Resident Coordinator

SDG Sustainable Development Goal

SIDS Small Island Developing States

SPRP Strategic Preparedness and Response Plan

SRO-SA Subregional office for Southern Africa

TCS Technical Cooperation Section

TOC Theory of Change TOR Terms of Reference

UN United Nations

UNCTAD United Nations Conference on Trade and Development

UNDP United Nations Development Programme

UNIDO United Nations Industrial Development Organization

UNOSSC United Nations Office of South-South Cooperation

UNSD United Nations Statistics Division

WHO World Health Organization

WTO World Trade Organization

6

Executive summary Project overview

The project “Global Initiative towards post-Covid-19 resurgence of the MSME sector” aimed at enhancing the resilience of Micro, Small, and Medium Enterprises (MSMEs) in developing countries and economies in transition, especially in the wake of the global COVID-19 crisis. The primary objective of the project was to develop and implement capacity-building tools for both governments and MSMEs. These tools were designed to mitigate the economic and social repercussions of the pandemic, focusing on MSMEs’ contribution to the SDGs. Operationally, the project was structured according to the Entrepreneurship Policy Framework (EPF) of UNCTAD. The EPF aims to support developing country policymakers in the design of initiatives, measures and institutions to promote entrepreneurship. The EPF features 6 key components: 1: Formulating a National Entrepreneurship Strategy; 2: Optimizing the Regulatory Environment; 3: Enhancing Entrepreneurship Education and Skills Development; 4: Facilitating Technology Exchange and Innovation; 5: Improving Access to Finance; and 6: Promoting Awareness and Networking. Building on these components1. The project was operated under five clusters, addressing critical areas of MSME recovery. These clusters included mobilizing entrepreneurial ecosystems, simplifying business registration processes, improving access to finance and financial literacy, increasing access to technology and innovation, and enhancing access to markets. The activities were designed to create an enabling entrepreneurship ecosystem by improving regulatory environments, providing access to innovation, technology, finance, and markets, and reaching out to vulnerable groups such as women and informal workers.

The project commenced in May 2020. The initial completion date was planned for March 2022 but this was extended to June 2022. It underwent three phases involving project startup and approval, initial project delivery and design of scaling up initiatives, and the launch of new activities based on previous results. Beneficiaries of this project included governments, policymakers, MSMEs, business associations, support services, and relevant technical staff in institutions and ministries. The project provided support to 96 beneficiary countries, employing various interventions such as advisory services, training, workshops, seminars, and online platforms. Jointly implemented by UNCTAD, DESA, and the UN regional commissions for Africa (ECA), Europe (ECE), Latin America and the Caribbean (ECLAC), Asia and the Pacific (ESCAP), and the Arab region (ESCWA), the project received financial allocations totalling $4,490,500, with each participating agency implementing its allocated component and managing day-to-day project operations. UNCTAD served as the lead coordinating entity. Key partners in this initiative included UN Development Partners, UN Resident Coordinators, UN Country Teams, UNITAR, UNDP, UNOSSC and ITC. Moreover, the project collaborated with EMPRETEC Centres and other stakeholders such as business incubators and support services. In alignment with the post-2015 development agenda, the project aimed to contribute directly to specific SDGs, particularly SDG 4 (Quality Education), SDG 8 (Decent Work and Economic Growth), and SDG 9 (Industry, Innovation, and Infrastructure).

Evaluation purpose and scope

This evaluation analysed the relevance, coherence, efficiency, effectiveness, and sustainability of the project and its mainstreaming of gender, human rights and disability considerations. The evaluation was carried out between February 2023 and September 2023 following a structured process of data collection and analysis, which included desk review, key informant interviews and online surveys. The evaluation conclusions and recommendations are intended to inform the work of UNCTAD and other UN implementing entities as well as the Development Account Programme Management Team (DA-PMT), which will conduct a synthesis from this evaluation and those of the other four DA Covid-19 projects.

1 According to the Project Document for Phase 2, interventions were selected based on several criteria including to target capacity building areas that are proven to have key impacts on entrepreneurship and MSMEs promotion as based on the formulation and implementation of UNCTAD’s Entrepreneurship Policy Framework (EPF): enabling ecosystem and facilitation of entrepreneurship competencies/motivation, access to finance, technology and markets, networking.

7

Conclusions

Relevance

The project responded to the needs of Governments and MSMEs. Interventions contributed to and were informed by needs and impact assessments. Recipients of project interventions were often involved in or contributed to the design and delivery of project activities and outputs. On various occasions, project outputs were tailored to the needs of target users and beneficiaries, being countries or vulnerable groups. The relevance of the interventions was somewhat mitigated by the mandates and capacities of the implementing UN entities. The latter are equipped for supporting medium- and long- term policy changes rather than immediate crisis responses. Other challenges included the restrictive Covid-19 measures and the short time frame for implementation of the project. Nonetheless, on the whole, the implementing UN entities exercised adaptive management to ensure high relevance of their interventions. The phased approach, in particular, provided the opportunity to introduce new activities not foreseen at an earlier stage.

Coherence

The project was anchored in the mandates and comparative advantages of the implementing UN entities. Selected interventions followed previous work and specific requests from member States and were often coordinated with or complementary to existing activities. At conception, the target of the project was to address specific constraints faced by MSMEs. It was designed around the Entrepreneurship Policy Framework to show the complementarities across interventions that each implementing UN entity would deliver in response to regional priorities and capacities. Some collaborations between implementing UN entities were pursued and complementarities established, but on a limited level. A few synergies in the form of joint outputs were integrated in the 3rd phase of the project. By design, technical task forces created through the project were expected to bridge the implementing UN entities. However, this proved difficult to launch and sustain, owing to several constraints, including a highly ambitious objective given the global crisis and competing priorities within each agency. Nevertheless, the coordination and complementarities established by the project among the implementing entities contributed to expanded dissemination and outreach of outputs, avoided overlaps between them, and allowed significant knowledge exchanges aligned with and supportive of a “One UN” system. On the other hand, collaboration of the implementing UN entities with other UN organisations, including UNCTs, was limited. Despite this shortcoming, at national level, target recipients of project outputs found the interventions complementary to those of other UN agencies.

Efficiency

Project coordination was complex and demanding, involving seven UN entities and the target delivery of around 100 outputs, of which some were composed of many activities. Nevertheless, the coordination of the project was found to be efficient overall given the circumstances and resources available. The coordination of the design phase of the response was actively supported by the DA-PMT. Coordination of project implementation was led by UNCTAD in collaboration with a Steering Committee that met on a regular basis, most often bi-monthly. UNCTAD’s Budget and Project Finance Section (BPFS) produced the project’s financial monitoring table every month, which included the expenditure data of UNCTAD as well as the other implementing entities. A monitoring dashboard was created by UNCTAD to facilitate the tracking of UNCTAD’s project delivery. Data from some of the other implementing UN entities was added, when provided, at two reporting points during the lifetime of the project.

The project would have benefited from the allocation of more resources towards the coordination and monitoring of overall implementation. Across the implementing UN entities, the project was considered complex and hard to follow outside of the interventions under their direct control. Some staff perceived that there were too many participants in the coordination meetings, that the project lacked a Theory of Change, and that it was primarily a compilation of activities rather than a cohesive and mutually reinforcing bundle of interventions supporting in depth any given country. Some staff in the UN Regional Commissions questioned the efficiency cost of a global or inter-regional response versus regional responses. The project was implemented in three phases, which were supportive of adaptive management and perceived by staff as the most rational approach considering the circumstances. However, it was also pointed out that clearer visibility from the onset on the funding available for future phases would have facilitated planning and the search for synergies.

8

Effectiveness

The Development Account programme, as a funding modality, was quick to respond and adapt to the COVID- 19 crisis. The prompt availability of resources and rapid orientation toward a global response contributed to the rapid launch of a wide range of interventions. However, questions remain as to its adequacy for addressing the immediate needs of a crisis in comparison with other UN agencies and mechanisms with a mandate for humanitarian response. The governance of this project, with DA-PMT’s oversight, the management structure with UNCTAD in the lead coordinating role, and each implementing entity leading the operational implementation at the regional level, ensured the global and regional scope of project delivery. At the operational level, implementing entities relied more on partnerships with local and regional institutions rather than coordination within the larger project management structure. The project governance and management mechanisms evolved in tandem with the COVID-19 crisis and the project’s emerging needs. The result was a high delivery rate but with limited inter-agency collaboration in the process. Yet, despite limited collaboration, this independent evaluation identified that the project was effective in producing identifiable results at the outcome level.

Overall, the project delivered 85% of the planned outputs. Phase 3, particularly, delivered the highest number of outputs but had the lowest outcome effectiveness rate, mostly because of budget cuts and some outputs still in progress at the time of this evaluation. There is evidence of the effective improvement of national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship and MSME promotion (outcome 1.A) [such as the adoption of a revised entrepreneurship strategy by South Africa, Uganda and Seychelles]; improved resilience and competitiveness of MSMEs (outcome 1.B) [for example with 64.5% of participants opening a new business after attending Empretec-based training]; facilitated MSME registration and formalization through e-platforms (outcome 2) in El Salvador, Benin, Cameroon and Mali; improved access to finance, including upskilling in financial literacy (outcome 3) [for example with 97% of the MSMEs in Latin America that attended trainings reporting improved financial literacy in accounting and reporting and also improved capacity to manage financial resources]; increased MSME access to innovation and technology (outcome 4) [including through capacity development on green technologies for SMEs in Southern Africa]; and increased access to local, regional and international markets through digitalization and non-tariff measures (outcome 5) [for instance with case studies on competition in Thailand, South Africa and Brazil, followed by webinars and the creation of an online course on SMEs and competition policy, primarily for government officials].

In relation to the project indicators, the assessment of their effectiveness was based on limited information, proxies, and expert judgement since they often lacked SMART criteria. It was possible to infer that 85% of them showed some level of progress, either through objective measures (e.g., sales increase) or proxies (e.g., post-training satisfaction assessments and survey responses). The high effectiveness results can be partly attributed to the conceptual framework provided by the EPF, as well as the cluster-based approach with each cluster addressing specific regional constraints and responding to demand from member States. In addition to clustering about 100 outputs from seven implementing entities into 5+1 outcomes, the EPF provided a coherent thread to relate outputs that cut across outcomes. Further analysis would be needed, however, to clearly identify which of those outputs are the most suitable or adaptable for a crisis response.

The project improved the capacity of policymakers in designing and implementing policies supportive of MSMEs, especially in terms of contributing to a country’s government responses, country-specific studies, technical assistance, training courses, and the development of digital tools and regional interventions. This improved capacity is reflected in beneficiaries’ reports of how they have been considering the needs of MSMEs, including those led by women and youth, in their daily work. These results could become more transformative and sustainable with more institutional support to networks of policymakers and communities of practice for mutual and regional learning on supportive policies for MSMEs, as well as by mainstreaming gender and Leaving No One Behind (LNOB) approaches in government officials’ capacity-building.

Sustainability

The project developed approaches, tools and capacities that are being transferred at multiple levels. At the policy level, the addition of more functionalities to e-government tools, expanding training courses to more countries, and the establishment of agreements to implement the project recommendations are all strong evidence of the sustainability of the project’s achievements. At the behavioural level, beneficiaries are applying new knowledge and skills to their daily work, be it by including MSMEs’ issues in policy making or by improving MSMEs’ management, performance, outputs and, by extension, resilience. Challenges to this transfer of

9

knowledge relate to the lack of financial resources, need for follow-up mentoring, and lack of a favourable business environment.

Gender, human rights, and disability

The overall project design was well aligned with the two key UN documents that provide clear direction for the integration of a human rights-based approach, which include addressing the aspect of gender equality and Leaving No One Behind. Although these aspects were well-articulated in the project documents, strong follow-through and documentation of their implementation was limited to a few project components. For those components that did include these aspects, gender was more commonly integrated and youth to a lesser degree. There was very little evidence of the outreach, participation, or impact of project activities for people with disabilities or other vulnerable groups. The UNCTAD component on Women and Cross-Border Trade targeting five countries in Africa was a good practice in how to design and implement this type of intervention with a strong reflection of the needs of some of the most vulnerable women across border areas.

Recommendations Recommendation 1: (i) UNCTAD should further leverage the experience gained through the project to map out how components of the Entrepreneurship Policy Framework relate to and can be best positioned to support MSMEs in their recovery from different types of crises and (ii) DESA and the Regional Commissions should add their analysis of how their work can contribute towards the objective. UNCTAD, DESA and the Regional Commissions could build on the project to map their areas of intervention in support of the MSME sector, as well as capacities and knowledge on the implementation of the EPF components, including in a crisis context, and opportunities for broadening the uptake of EPF components at the regional level based on national needs. UNCTAD, DESA and the Regional Commissions could further consider identifying areas of joint interventions that would trigger complementarities and synergies between the agencies. This could involve collaborating on the development of capacities of all national and sub-national actors, i.e. Governments, MSMEs, other partners (e.g. Chamber of Commerce, Business Incubators, etc.) in line with United Nations Sustainable Development Cooperation Frameworks (SDCF) in respective countries; strengthening or contributing to facilitate access to finance for the MSMEs (including seed money, grants, access to credit, etc.); improving the coordination of MSMEs related policies across ministries; increasing interventions at the local level, such as by supporting NGOs or MSMEs outside of the main cities. The application of a human rights- based approach, gender responsiveness and inclusion of other vulnerable groups (LNOB) should be ensured. Recommendation 2: UNCTAD should continue building on the momentum generated by the project to continue fostering knowledge exchanges and promoting the Entrepreneurship Policy Framework. UNCTAD, DESA and the Regional Commissions should identify means to more meaningfully continue to share good practices and lessons learned on the demand-driven support they provide to the MSME sector, including on areas such as green/circular economy; innovation policies and ecosystems; MSME formalization and strengthening MSME capacities for the empowerment and leadership of women, youth and groups in vulnerable situations. UNCTAD and the Regional Commissions could also propose organizing regional events and/or a global conference to promote entrepreneurship policy and further advance the goals and impact of the Surge project. Furthermore, learnings from the experiences with the Surge about external partnerships should encourage UNCTAD to promote the EPF and entrepreneurship development to other UN agencies also engaged in this area (e.g. ILO, FAO, UN Women, etc.), to development banks, or to related initiatives such as the recent Global Accelerator on Jobs and Social Protection. UNCTAD could also consider joining and supporting events that promote the achievement of SDG 8.3 as an avenue to further promote the EPF. Recommendation 3: The DA-PMT should develop a clear framework for assessing the costs and benefits of implementing a global or inter-regional project versus regional projects. As a global crisis can affect regions and countries differently, global or joint projects should be developed only when there are clear benefits of joint implementation. Thus, a global or inter-regional vs. regional response would not be a priori decision, but a result of a clear assessment. A specific guideline or framework should be developed for this. Some of the assessment criteria could include the need or demand from member States for an integrated response; the range of common versus entity-specific activities and added value of complementary interventions (i.e. in terms of geographical coverage, reach of the target population, capacity, and/or coverage of multiple reinforcing technical areas, or networks and partnerships); capability to execute joint interventions (e.g. in terms of time, resources, logistics, and flexibility to pivot interventions without bureaucratic procedures);

10

coordination costs (which increase with the number of participating UN entities); project inception modalities and governance and knowledge management requirements to facilitate collaboration and knowledge sharing; scaling and sustainability plans; expected results of spreading resources versus concentrating on fewer countries; and so forth. The time taken to conduct such assessments will increase project coherence and effectiveness and facilitate the identification of the financial and human resources needed for project implementation. The network of DA Focal Points is a key existing asset for this assessment. Recommendation 4: Implementing UN entities should ensure that they have a comprehensive Results Framework for the entire project as well as an adequate monitoring plan, with indicators that are designed to support the ongoing monitoring. The results framework should have one objective and ideally have one outcome per cluster/workstream. The objective should state the intended goal of the project, describe the overall achievement targeted by the project, involving a process of change aimed at meeting the needs of identified beneficiaries, and reflect the overall funding available to the project. Each objective should include reference to the project’s beneficiaries and its substantive focus. The objective should not attempt to explain the ways in which the project intends to achieve the objective (i.e. it should not include the word ‘through’ or describe the internal work of the UN using verbs such as ‘support’, ‘facilitate’ or ‘contribute’). The outcomes (OCs) should describe the changes that are expected to occur as a result of the completion of outputs. The OCs should be achievable within the project’s timeframe and budget, and should be specific enough to be measured by the associated indicators of achievement. The indicators of achievement (IAs) should provide measures for monitoring progress towards achieving the OCs and reporting on them after completion of the project. Every indicator needs to provide clearly defined baselines, units of measurement and targets, detailing the quantity, quality and timing of expected results. The monitoring, evaluation, accountability, and learning (MEAL) system should also be designed to capture HRBA, gender and LNOB aspects. In phased interventions or during project execution, any changes in the logframe if/when pivoting activities should be clearly explained to the wider team. Recommendation 5: Implementing UN entities should ensure that sufficient resources are allocated to project coordination, technical collaboration, and partnership building. The absorption capacity of implementing entities can be challenged by crisis response projects which add to the planned programme of work. This is further compounded by projects that come with an extensive UN partnership and a global scope. Sufficient resources should be dedicated to global coordination and to building global partnerships with strategic stakeholders (e.g., UN organisations engaged in supporting the MSME sector; development banks). Capacities should also be directed to supporting technical collaboration and the staff implementing interventions, including towards synergy or liaison with the UNCTs. When designing the project, implementing UN entities should consider featuring coordination and partnership-building in the Theory of Change or logframe of the project. Tools to support continuous connections and knowledge exchange, and to ensure institutional memory should be part of the response package, such as a project website, SharePoint space for all team members, and a Yammer network or Teams channel. The integration of cross-cutting aspects (HRBA, gender responsiveness, LNOB) also requires expertise with sufficient and dedicated time and resources. Guidance could be development to project managers on how to do this.

11

1. Introduction

1. This Evaluation Report presents the independent evaluation of the United Nations Development Account Project 2023W – “Global Initiative towards post-Covid-19 resurgence of the MSME sector” (the MSME Surge project or project). The objective of the project was to develop and implement capacity-building tools for governments and micro, small and medium enterprises (MSMEs) to facilitate their resurgence from the COVID-19 crisis and strengthen the resilience of MSMEs in developing countries and economies in transition. The project was implemented by UNCTAD, UN DESA and the UN regional commissions for Africa (ECA), Europe (ECE), Latin America and the Caribbean (ECLAC), Asia and the Pacific (ESCAP), and the Arab region (ESCWA). Project implementation started in May 2020 and ran until June 2022.

2. The final evaluation of the MSME Surge project aims to assess the results of the project and establish the link between the achievements and activities of the intervention; to assess the response delivery and external coordination of the project, including the extent of gender, human rights and disability mainstreaming; and to identify good practices and lessons learned from the project that could feed into and enhance the implementation of related interventions.

3. The evaluation started in February 2023 and concluded in September 2023. The evaluation conclusions and recommendations are intended to inform the work of UNCTAD and the implementing UN entities, as well as the UN Development Account Programme Management Team (DA-PMT).

2. Description of the Project

2.1 Background

4. The first diagnosed cases of ‘viral pneumonia’ became public on 31 December 2019. On 30 January 2020, WHO declared the coronavirus outbreak a public health emergency of international concern (i.e., WHO’s highest level of alarm) and soon after it launched a COVID-19 Strategic Response and Preparedness Plan (SPRP) and asked the UN Secretary-General to activate the UN crisis management policy2. COVID-19 was declared a “pandemic” on 11 March 2020. On 19 March 2020, the SG report “Shared Responsibility, Global solidarity: Responding to the socio-economic impacts of COVID-19”3 launched a call for action with a focus on people – women, youth, low-wage workers, small and medium enterprises, the informal sector and on vulnerable groups who were already at risk. The UN published the Global Humanitarian Response Plan (GHRP) for COVID-194 on 25 March 2020, initially calling for USD 2.01 billion. On 2 April 2020, the General Assembly adopted the Resolution on Global Solidarity to fight the coronavirus disease 2019 (COVID-19)5. On 5 April the COVID-19 Response and Recovery Multi-Partner Trust Fund (MPTF) was established to channel funds for a system-wide immediate socio-economic response to COVID-19. The UN Framework for the immediate socio-economic response to COVID-196 further set out the structure for the United Nations’ urgent socio-economic support to countries and societies in the face of COVID-19. The Framework presented five streams of work connected by a strong environmental sustainability and gender equality imperative to build back better. One pillar of the Framework focused on protecting jobs, supporting small and medium-sized enterprises, and informal sector workers through economic response and recovery programmes.

5. From its onset, the advent of COVID-19 confronted governments, the development and the humanitarian community and society with a deteriorating socio-economic context. COVID-19 affected poverty, trade and employment levels, and put at risk the achievement of the SDGs. The World Bank estimated the COVID-19 pandemic to have pushed an additional 119 million to 124 million people into extreme poverty

2 The UN Crisis Management Team brings together 23 UN entities to coordinate a comprehensive whole-of-UN response. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/interactive-timeline/#event-47. 3 https://www.un.org/sites/un2.un.org/files/sg_report_socio-economic_impact_of_covid19.pdf 4 UN OCHA. 2020. Global Humanitarian Response Plan for COVID-19. Geneva. 5 United Nations. 2020. Resolution adopted by the General Assembly on 2 April 2020. A/RES/74/270. General Assembly. New York. Link: https://undocs.org/en/A/RES/74/270. 6 UN. 2020. A UN framework for the immediate socio-economic response to COVID-19. April 2020. New York.

12

in 20207. The IMF estimated a 3.5 percent contraction in global GDP in 20208. UNCTAD reported on the use of trade policy instruments with trade-restrictive effects in over 180 countries, most of them in developing countries9. ILO estimated that 8.8 per cent of global working hours were lost in 2020 relative to the fourth quarter of 2019, equivalent to 255 million full-time jobs10. At the beginning of the pandemic, most of these job losses affected women, and, by August 2020, the participation of women in the labour force was over 15 per cent lower than the baseline – male participation was about 10 per cent lower11. Annual unmet SDG financing needs before COVID-19 were USD 2.5 trillion. The pandemic created additional needs in 2020 for USD 1 trillion in COVID-19 spending while the drop in external private resources for developing countries was USD 700 million12.

6. The COVID-19 crisis plunged the global economy into a deep recession and micro, small and medium enterprises were amongst the hardest hit, with decreased liquidity or cash flow availability, decreased demand for products and services, and decreased supply of inputs13. According to UNCTAD’s analysis of the World Bank data in selected countries, on average 13% of small-size firms were temporarily closed due to COVID-19 pandemic, almost 75% of SMEs experienced a decrease in demand for their products and services, over two-thirds of SMEs experienced decrease in supply of their inputs14. A survey implemented by UNECA and IEC Ltd of African MSMEs at the start of Covid-19 found that four-fifths of the survey respondents indicated being significantly affected by the current COVID-19 crisis (rating the effect as highly severe or severe). MSMEs make up 90 per cent of the economic engine of developing countries and are a major source of employment and self-employment, including for vulnerable groups, such as women, elderly and youth. Trapped in economic stagnation due to large-scale lockdowns, millions of MSMEs became the most vulnerable to COVID-19 within the private sector. At the sectoral level, demand and supply stagnation was seen in many industries. MSMEs are major players in non-essential services,15 which became the most affected sectors by lockdown and other isolation measures. In addition, many micro and small businesses are in the informal economy, which was hit first and hardest by the pandemic.

2.2 Project objectives and expected accomplishments

7. The main objective of the project was to develop and implement capacity-building tools for governments and MSMEs to facilitate the resurgence and strengthen the resilience of MSMEs in developing countries and economies in transition. These capacity-building tools sought to mitigate the economic and social impact of the global COVID-19 crisis and to facilitate the contribution of MSMEs to SDG implementation.

8. The initiative brought together UNCTAD, UN DESA and the UN regional commissions for Africa (ECA), Europe (ECE), Latin America and the Caribbean (ECLAC), Asia and the Pacific (ESCAP), and the Arab region (ESCWA) with funding provided by the UN Development Account (DA). The project expected to ensure both global reach and regional presence, international cooperation, and exchange of knowledge and good practices from all over the world. The project also aimed to establish a coherent approach towards MSME resurgence reflecting the holistic and comprehensive nature of efforts needed to ensure and facilitate MSMEs’ green, resilient and inclusive recovery.

7 https://blogs.worldbank.org/opendata/updated-estimates-impact-covid-19-global-poverty-looking-back-2020- and-outlook-2021 8 IMF. 2021. World Economic Outlook Update. January 2021. Washington. 9 UNCTAD. 2021. Impact of the COVID-19 pandemic on trade and development: Recovering, but unevenly - Situation as at 31 March 2021. https://unctad.org/programme/covid-19-response/impact-on-trade-and-development- 2021#aTradePolicy 10 ILO. 2021. ILO Monitor: COVID-19 and the world of work. Seventh edition. 25 January 2021. Geneva. 11 UNCTAD. 2021. Impact of the COVID-19 pandemic on trade and development: Recovering, but unevenly - Situation as at 31 March 2021. https://unctad.org/programme/covid-19-response/impact-on-trade-and-development- 2021#aTradePolicy 12 OECD. 2020. Global Outlook on Financing for Sustainable Development 2021. Paris. 13 UNCTAD. 2021. Impact of the COVID-19 pandemic on trade and development: Recovering, but unevenly - Situation as at 31 March 2021. https://unctad.org/programme/covid-19-response/impact-on-trade-and-development- 2021#aTradePolicy 14 UNCTAD. 2021. Global Initiative towards post-Covid-19 resurgence of the MSME sector. MSME surge project - Addendum Phase 3 Project Proposal. Geneva. 15 The exact definition of these services varies by country, but they are generally recreational businesses, such as accommodation, catering, entertainment and tourism.

13

2.3 Project strategies and key activities

9. The project followed the conceptual approach of clustering interventions under the overall umbrella of the Entrepreneurship Policy Framework (EPF)16 of UNCTAD. The five clusters addressed the most critical areas of MSME recovery:

1- Mobilize entrepreneurial ecosystem and strengthen business skills 2- Simplify business registration and facilitate formalization 3- Improve access to finance/financial literacy 4- Increase access to technology and innovation 5- Enhance access to markets

10. Activities were to be delivered through coordinated and complementary efforts of participating agencies covering key interrelated components of an enabling entrepreneurship ecosystem, such as improving a regulatory entrepreneurship environment and mobilizing entrepreneurship potential, enhancing MSMEs’ access to innovation and technology, finance and markets, including at a policy level and at MSME level, and reaching out to most affected target groups, including women and informal workers. The project also included a number of activities to facilitate the green and sustainable recovery of MSMEs.

11. The project started its implementation in May 2020 and was scheduled to conclude on 31 March 2022, but received approval in February 2022 for an extension until 30 June 2022. The project was developed and implemented under three phases:

• Phase 1 (May-June 2020): The start-up of the project was approved by the DA Steering Committee based on a concept note presenting an overview of the project intended to be implemented over an 18-month period (to December 2021), the intended outcomes for Phases 1 and 2, and the detailed budget for phase 1 only.

• Phase 2 (July-December 2020): The Project Document submitted for approval of Phase 2 was expected to cover Phases 2 and 3, with the expectation that it would be revised in September 2020 to include the finalized Phase 3 activities and budget, and the project’s monitoring and evaluation plan, for Phase 3 approval.

• Phase 3 (initially January-December 2021, extended twice, first to March 2022 and then to June 2022): The addendum to the Project Document focused interventions on scaling up the implementation of the online outputs developed in Phases 1 and 2 and launching new activities in selected countries based on demand and results in the previous phases.

12. Over its 3 phases the project implemented more than 290 activities (Figure 1). Activities were sometimes multi-faceted, for example with reports being used to feed training materials, conferences, or webinars. Activities involved varying levels of effort, from one-time webinars to reports or online platforms requiring several months of development.

Figure 1: Overview of project activities (indicative17).

16 Entrepreneurship Policy Framework and Implementation Guidance | UNCTAD 17 Some activities were described by the UN entities with interchangeable terms, such as webinars and trainings, while some other activities were not necessarily described, such as disseminating a report through a workshop. The evaluation reconstructed the number of activities based on a review and interpretation of the project logframe.

14

Source: Independent evaluation, 2023.

13. The Project Document did not formulate a Theory of Change (TOC) but presented a results framework that the evaluation used jointly with inputs provided by informants to reconstruct a simplified (draft) TOC at evaluation (Annex 1).

14. According to the Project Document (Phase 2), the monitoring arrangements planned for the project included:

● Bi-monthly online meetings of the Project Steering Committee: Representatives from all seven entities were sought to monitor project delivery to ensure smooth and steady implementation. Representatives were also expected to meet on an as-need basis for important decisions that affected all seven entities.

● Inputs and feedback from the Project Design Team: Representatives from UNCTAD, DESA, ECA, ECE, ECLAC, ESCAP and ESCWA, who participated in the project design and its adjustments as needed, in accordance with the COVID-19 situation, were considered to be a key part of the project to evaluate and act upon stakeholder commentary.

● Task Forces: Task Forces comprised of experts from the partner entities were to be established to provide ongoing coordination of work of the five clusters.

● Annual progress reports: Annual or phase-based progress reports were to be provided to the DA- PMT on all aspects of project implementation.

● Success stories: based on request and template provided by the DA-PMT.

2.4 Beneficiaries and target countries

15. The main direct beneficiaries targeted by the project included: Governments, policy makers, Micro-, small and medium enterprises (MSMEs), Business or industry associations, Business support services, Commercial Registry Offices, Technical/experts staff in relevant institutions and ministries, and Multinational Enterprises (MNE).

7

101

29

90

40

17

8

2

0 20 40 60 80 100 120

Online systems and platforms (e-Registration systems, knowldge hubs, websites, search engine, etc.)

Trainings (Workshops, online trainings, online toolboxes, etc.)

Webinars (Global, regional, national; series or unique events)

Reports (Books, Assessments, Policy Reports, Guidelines, Case Studies, Maps, etc.)

Online training tools (e-Learning programmes, course materials, self-training, etc.)

Conferences (Meetings, roundtables, inter-governmental discussions, online events, policy dialogues, etc.)

Technical assistance (Entrepreneurship strategies, national policies, etc.)

Networks (Community of Practice, Policy Network)

The project conducted more than 290 activities

15

16. The beneficiary countries covered different geographical regions. The project compiled a list of 96 beneficiary countries in the final report, as shown in Table 1. These countries were supported by the Surge project, to varying degrees, either through physical/in-person activities or through online modalities. Some of the forms of interventions (in-person or online) included advisory services, trainings, workshops, seminars, conferences and events, platforms and systems. In many cases, implementing entities also provided direct technical assistance to countries through policy advice or in the form of assessment reports, research studies, or tailored guidelines18. Project partners organized online seminars targeting specific countries and also regional or global audiences in many cases.

Table 1: Project’s beneficiary countries.

Africa

1. Algeria 2. Angola 3. Benin 4. Botswana

5. Cameroon 6. Egypt 7. Eswatini 8. Ethiopia

9. Ghana 10. Kenya 11. Lesotho 12. Malawi

13. Mauritius 14. Mozambique 15. Namibia 16. Nigeria

17. Seychelles 18. South Africa 19. Tanzania 20. The Gambia

21. Uganda 22. Zambia 23. Zimbabwe

Americas

24. Argentina 25. Bahamas 26. Barbados 27. Belize

28. Bolivia 29. Brazil 30. Chile 31. Colombia

32. Costa Rica 33. Cuba 34. Dominica 35. Dominican Republic

36. Ecuador 37. El Salvador 38. Grenada 39. Guatemala

40. Guyana 41. Jamaica 42. Mexico 43. Nicaragua

44. Panama 45. Paraguay 46. Peru 47. Saint Kitts and Nevis

48. Saint Lucia 49. Trinidad and Tobago 50. Uruguay 51. Venezuela

Asia

52. Afghanistan 53. Armenia 54. Azerbaijan 55. Bangladesh

56. Bhutan 57. Brunei Darussalam 58. Cambodia 59. China

60. Georgia 61. India 62. Indonesia 63. Islamic Republic of Iran

64. Jordan 65. Kazakhstan 66. Kyrgyzstan 67. Lebanon

68. Lao PDR 69. Maldives 70. Malaysia 71. Mongolia

72. Myanmar 73. Nepal 74. Pakistan 75. Palestine

76. Philippines 77. Saudi Arabia 78. Singapore 79. Sri Lanka

80. Tajikistan 81. Thailand 82. Türkiye 83. United Arab Emirates

84. Viet Nam

Europe

85. Albania 86. Belarus 87. Bosnia and Herzegovina 88. Montenegro

89. North Macedonia 90. Republic of Moldova 91. Romania 92. Russia

93. Serbia

Oceania

94. Fiji 95. Papua New Guinea 96. Tuvalu

18 Some of the countries that received direct technical assistance include for example Armenia, Bangladesh, Benin, Bosnia and Herzegovina, Botswana, Brazil, Cameroon, El Salvador, Georgia, Kazakhstan, Kenya, Kyrgyzstan, Malawi, Mali, North Macedonia, Republic of Moldova, Serbia, Tajikistan, Tanzania, Thailand, The Gambia, Ukraine, Zambia.

16

2.5 Key partners and other key stakeholders

17. Jointly implemented by UNCTAD, DESA, ECA, ECE, ECLAC, ESCAP and ESCWA, the project built upon the comparative advantages and mandates of the participating agencies to provide immediate advice, capacity-building and support to governments and MSMEs during the ongoing global pandemic. Broadly, the roles and lead entities, based on the mandates and expertise of the participating agencies, for each of the five project clusters and workstreams were as presented in table 2.

Table 2: Project clusters and lead and participating entities. Source: Project document Phase 2.

Project cluster/workstream UNCTAD DESA ECA ECE ECLAC ESCAP ESCWA

Overall coordination19 O Project cluster/workstream 1 Entrepreneurship and business skills promotion

O O O O

Project cluster/workstream 2 Business facilitation/formalization

O O O

Project cluster/workstream 3 Access to finance/financial literacy

O O O

Project cluster/workstream 4 Access to technology and innovation

O

Project cluster/workstream 5 Access to markets

O O O O O O

O: Lead institution O: Participating institution

18. UNCTAD was the lead project coordinating entity20. Day to day management of the project was entrusted to each implementing agency for its own component. Each participating agency managed and implemented its component with funds allocated to it. Within UNCTAD, project oversight was ensured by UNCTAD’s Enterprise Branch within the Division on Investment and Enterprise.

19. Key project partners included the UN Development Partners with the UN Resident Coordinators (UNRCs), UN Country Teams (UNCTs), UNITAR, UNDP, UNOSSC, ITC. The project was also implemented with the EMPRETEC Centres21. Other key stakeholders included technical/expert staff in relevant institutions, ministries, and business incubators and support services.

2.6 Resources

20. Overall, the United Nations Development Account (UNDA) allocated a total of $4,490,500 under this project. Concretely, UNCTAD received $2,671,000, ECLAC received $134,000, ESCAP received $240,000, ECA received $467,000, ECE received $448,500, ESCWA received $370,000 and DESA received $160,000 (Figure 2).

Figure 2: Allocation of project funding.

19 Phase 1 of the project was co-led by UNCTAD and ESCWA. 20 UNCTAD and ESCWA co-led phase 1 of the project. 21 EMPRETEC is a flagship capacity-building programme of UNCTAD for the promotion of entrepreneurship and micro, small and medium-sized enterprises (MSMEs) to facilitate sustainable development and inclusive growth.

17

Source: Final report.

21. The project budget was approved and distributed by phase (Table 3). The budget for phase 1 was $325,000 and the budget for phase 2 was $1,310,000. Budget for phase 3 was reduced in November 2011 from a provisional $3,165,500 to $2,855,500 (see section 5.3).

Table 3: Project's implementation phases

Phases Implementation period Effective budget allocations

I May-June 2020 $325,000

II July-December 2020 $1,310,000 III January 2021-June 2022 $2,855,500

2.7 Link to the Sustainable Development Goals (SDGs)

22. The project aimed to contribute to the post-2015 development agenda by making direct contributions to the sustainable development goals (SDGs), in particular goal 4 on “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” (target 4.4); goal 8 on “Promote development- oriented policies that support productive activities, decent job creation, entrepreneurship, creativity and innovation, and encourage the formalization and growth of micro-, small- and medium-sized enterprises, including through access to financial services” (target 8.3); and goal 9 on “Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation” (targets 9.3 and 9.5).

2.8 Innovative elements

23. The project demonstrated several innovative elements. At the institutional level, the project relied on increased consultative processes and a phased approach which enabled adaptive management. At the programmatic and technical levels, the project contributed to the introduction of MSMEs into policy discussions in response to the pandemic crisis, and e-government services. At the operational level, the project developed innovative approaches and tools to respond to country needs, such as online delivery of trainings and knowledge sharing events.

3. Evaluation objectives, scope, and questions

3.1 Purpose and objectives

$160,000

$370,000

$448,500

$467,000

$240,000

$134,000

$2,671,000

$0 $2,000,000 $4,000,000

DESA

ESCWA

ECE

ECA

ESCAP

ECLAC

UNCTAD

3.56%

8.24%

9.99%

10.40%

5.34%

2.98%

59.48%

DESA

ESCWA

ECE

ECA

ESCAP

ECLAC

UNCTAD

0% 20% 40% 60% 80%

18

24. As per the TOR (Annex 7), the final evaluation of the MSME Surge project has the following specific objectives:

● Assess the results and establish the link between achievements and activities of the intervention;

● Assess the response delivery and external coordination22, including the extent of gender, human rights and disability mainstreaming; and

● Identify good practices and lessons learned from the project that could feed into and enhance the implementation of related interventions.

3.2 Evaluation scope, criteria, and questions

25. The evaluation covers the duration of the project from April 2020 to June 2022. The evaluation focuses on the countries covered by the project. The evaluation analysed unintended positive (or negative) outcomes beyond the 96 countries identified by the project as beneficiary countries. The evaluation conclusions and recommendations are intended to inform the work of UNCTAD and other UN implementing entities as well as the Development Account Programme Management Team (DA-PMT), which will conduct a synthesis from this evaluation and those of the other four Covid-19 surge projects. The evaluation took a retrospective and forward-looking approach. It was summative but also formative aiming to support UNCTAD and UN partners in their next steps.

26. The evaluation addressed the questions in table 4. The evaluation questions were unpacked into an evaluation matrix with suggested measures (Annex 2) and guided data collection methods (Annex 3).

Table 4: Evaluation questions.

CRITERIA KEY QUESTIONS

RELEVANCE

1. To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID-19?

2. To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g., COVID-19 Socio-Economic Response Plan)?

COHERENCE

3. To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

4. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID-19 in delivering socio-economic support to Member States?

EFFICIENCY

5. How well coordinated was the process for the response among the entities implementing the joint project?

6. How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

EFFECTIVENESS

7. To what extent did the programme (Development Account) and project governance and management structures and processes enable, or hinder, the effective implementation of the joint project and the achievement of its results?

8. To what extent has the project contributed to the expected outcomes as enunciated in the project document?

9. How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

10. What innovative approaches or tools, if any, did the response use, and what were the outcomes and lessons learned from their application?

22 The OIOS COVID-19 response evaluation protocol identifies the following three cross-cutting focus areas: 1) response delivery; 2) external coordination (or “Delivering as one”); and 3) business continuity. “Response delivery” is further defined as consisting of delivery of: 1) the existing mandate needed to implement previously mandated activities in the new environment created by the pandemic; and 2) the COVID-19 specific response (health and non-health) needed to address the pandemic specifically. See OIOS (October 2020), “COVID-19 Response Evaluation Protocol”, para 3-4.

19

SUSTAINABILITY 11. What measures were adopted to ensure that outcomes of the response would

continue after the project ended? GENDER, HUMAN RIGHTS AND DISABILITY

12. To what extent were perspectives on gender equality, protection of human rights and reaching underserved groups integrated into design and implementation of the project? What results can be identified from these actions?

4. Methodology

27. The evaluation collected and analysed data from a range of sources to deepen understanding and triangulate the assessment. The following data collection instruments were used:

• Desk review: Study of secondary resources (Annex 4) as per the project management process and logframe in order to validate achievements, including UN GA resolutions, UN strategies and policies, documents/data related to project-level planning, implementation and results achievement (including post-training assessments, whenever data allowed23), project outputs and monitoring reports, and external reports (Annex 2).

• Interviews and focus groups: Interviews were conducted with a selected number of staff, partners, and stakeholders (Annex 5). Contact details of UN staff and partners were provided by UNCTAD (38 target informants), ECLAC (10), UNECE (10), UNECA (8), DESA (5), ESCAP (5), ESCWA (3). All prospective informants were invited to inform the evaluation, with at least one follow-up reminder message sent in absence of any response. The evaluation interviewed 58 informants. The canvas for semi-structured interviews was tailored to ensure specific relevancy to the selected stakeholders. To optimize time, a few interviews took the form of focus groups. Consultations were conducted virtually with Zoom or Teams. The following consultations per stakeholder group were conducted (Table 5):

Table 5: Groups of informants and number of consultations.

Informant groups Number of informants

Project Coordination Team 10 Project focal points of implementing entities (i.e. members of Task Forces) 12

DA-Project Management Team and DA focal points in participating entities 8

UN and Development Partners (UNCT members, UNDP) 3

Governments (Ministries, national institutions) and policy makers 6

MSMEs, Business or industry associations, Commercial Registry Offices 8

Technical/experts staff in relevant institutions, including EMPRETEC Centres 11

Total 58

• Survey: The evaluation carried out six external surveys (Annex 6) to inform the assessment of the relevance, coherence, and effectiveness of the project and its contribution to outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups. The surveys targeted participants to project activities and beneficiaries. The lists of contacts were provided by UNCTAD, DESA, ESCAP, ECLAC, ECA, and ECE (Table 6)24. The sampling strategy was convenience sampling. In addition, in consultation with UNCTAD Independent Evaluation Unit, the evaluation used purposive sampling to exclude from the lists provided by the UN entities (i) the staff from the same UN entities who were part of those lists as having attended or contributed to the events; (ii) participants to Trainings of Trainers workshops as their role was to train project beneficiaries; (iii) participants to one-time only webinar sessions as having had very limited exposure to the project.

23 Not all post-assessment data could be used by the evaluation team, because some were responded by a very small number of training participants, in relation to the total number of participants in a given training. 24 ESCWA was requested to provide a list of survey recipients but did not share one.

20

The surveys were composed of questions that were common to all six questionnaires and some specific to each of the 5+1 project outcomes25. The questionnaires were made available in English. They were translated in Spanish for outcomes 3 and 5 as target recipients were primarily in the Latin America and Caribbean region. The surveys were anonymous and remained open for 2 weeks, from Thursday 20 July to Friday 4 August. Two reminder messages were sent to increase the response rate. The surveys were launched to a combined list of 1454 persons from at least 120 countries (information on the localisation of survey recipients was not available for all recipients), with 89 messages bouncing back. Altogether, the surveys compiled feedback from 133 respondents. Responses were received from at least 53 countries (some participants did not indicate their location). Responses were received from at least 62 males and 42 females (some participants did not indicate their gender). All survey questions were optional. Questionnaires partially completed were kept in the batch of results when they contained information that was judged credible and meaningful. Statistics were calculated on the basis of the number of valid responses per question and not on the basis of the overall number of respondents to the surveys. The overall response rate to the email surveys is circa 9.7%. The surveys presented the opinion of those who responded but not of the entire list of recipients of the questionnaires nor of all the beneficiaries of the project.

Table 6: Survey recipients and respondents.

Outcome Email addresses Bounced back Valid email Valid responses Percentage

1A 574 38 536 52 9.70% 1B 258 24 234 40 17.09% 2 21 0 21 4 19.05% 3 219 4 215 11 5.12% 4 98 1 97 13 13.40% 5 284 22 262 13 4.96%

Total 1454 89 1365 133 9.74%

• Cross-cutting thematic analysis on human rights, gender and inclusion: As an integral part of the above-mentioned methods of data collection, the evaluation team explored the extent to which gender equality, human rights and leaving no one behind perspectives were considered in the design and implementation of the project. The UN framework for the immediate socio-economic response to COVID-1926 guided the assessment as it outlines how the responses to the pandemic should be consistent with international human rights standards, ensure gender equality and special measures to protect the most vulnerable and marginalized groups, such as women, youth, and migrants.

28. The evaluation used a combination of complementary tools for analysis of the data collected.

● Qualitative analysis of data, secondary resources and interviews/focus groups. The evaluation used ATLAS.ti for coding qualitative inputs.

● Quantitative analysis including systematizing post-training assessments, indexing and cross- tabulations of survey data, and quantitative coding of survey qualitative inputs.

29. The evaluation confronted several constraints or limitations that were mitigated as follows:

● Remote data collection: The evaluation did not include country visits that would allow for face- to-face interviews and direct observations, which may have limited the collection of evidence

25 The project outcome 1 was subdivided into outcome 1A and 1B. 26 A UN framework for the immediate socio-economic response to COVID-19, April 2020 https://unsdg.un.org/sites/default/files/2020-04/UN-framework-for-the-immediate-socio-economic-response-to- COVID-19.pdf

21

at the outcome level. Mitigation: Use of complementary data collection methods to enrich and triangulate findings.

● Limited availability of informants: The interviews (individual and group) are a key instrument for this evaluation. Given turnover and time since the project was completed, some country staff were not available for interviews as they, for example, had retired or left the organization, did not recall exactly what happened, or had too much work or competing demands for their time. Mitigation: Support was requested from the Project Manager for securing interviews.

● Lack of generalisation of survey results: Convenience sampling cannot ascertain that survey recipients were statistically representative of the entire community of project beneficiaries. Furthermore, reminder messages mitigated but did not prevent any non-response bias. Respondents also required to have internet access. The surveys presented the opinion of those who responded but not of all the beneficiaries of the project. Surveys were used as one among other sources of findings.

● Limited evaluability of some indicators: Because some indicators do not comply with the SMART criteria and are elaborated in a broad way, it was difficult to identify the evidence that answers to them. Additionally, they were associated with the work of specific implementing entities, rather than the project as a whole. This posed an additional challenge, especially when the entity associated with a certain indicator was not clear. Mitigation: Targeted consultation about these indicators, corresponding evidence directed at the responsible entity, and an inductive approach to associate results achieved within an outcome to specific indicators.

● Too short a time between end of project and the evaluation to observe impact: Since most of the project’s activities take time to show impact, this evaluation reports on indications of possible future impact on beneficiaries, governments, and partners. Mitigation: Self-reported survey for beneficiaries, and data triangulation generated this evidence.

5. Findings

30. This section presents the evaluation findings per evaluation criteria and question.

5.1 Relevance

To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID-19?

31. The project was designed to respond to demands of Member States to mitigate the effects of the COVID-19 pandemic. The project was based on ownership by beneficiary countries and relevance towards national and sub-national sustainable development priorities. A demand-driven approach was mainstreamed throughout the project lifecycle. The project was designed based on the request to UN Secretariat entities for assistance for MSMEs from Member States, as well as intergovernmental demands and resolutions on COVID- 19. The Project Document for phase 2 reported requests for support received from more than 70 countries across the 5+1 project outcomes. For example, the Governments of El Salvador, Benin, Cameroon and Mali, requested UNCTAD to create online services that streamlined and simplified business registration. The Governments of Malawi, Tanzania and Zambia, requested UNCTAD to scale-up the format of the cross-border trade trainings piloted in these countries and to replicate them at other borders in the same countries. Georgia and North Macedonia expressed interest for ECE to develop country-specific reports based on the findings of the regional Guidelines and Best Practices for MSMEs in delivering energy efficient products and in providing renewable energy equipment. Several staff from the implementing UN entities also mentioned that in-country interventions were conditioned by the receipt of a Letter of Agreement from Member States. In a few cases, informants further illustrated the demand-based approach of the project by indicating that some initial plans for collaboration were parked sometimes due to changing country needs and priorities.

32. The project conducted or contributed to needs and impact assessments to inform interventions. Across all regions the project collaborated with Member States and national stakeholders to conduct surveys, needs assessments, or impact studies that informed project design and implementation as well as national responses. In South-east and Central Asia, and the Pacific, countries such as Samoa, Bangladesh, Nepal,

22

Cambodia and Kazakhstan participated in ESCAP in-depth country assessments. In the Southern African region, ECA carried out a survey across its 11 member states to assess the impact of Covid-19 and the war in Ukraine on MSMEs in association with national business organizations, The survey gauged policies needed to foster regional integration and technology and innovation in post Covid-19 recovery strategies. ECE produced a rapid impact assessment series on end-to-end supply chain activities to support long-term development efforts for five beneficiary countries and presented them for intergovernmental discussion. ECLAC ran diagnostics on competition policy and consumer protection, and on SME support policies, to identify needs for project activities or products in target countries, including the needs of women and vulnerable groups. In Kenya, DESA and the Kenya Micro and Small Enterprises Authority assessed how MSMEs were affected by the pandemic. The survey results were disseminated to stakeholders, who were brought together to participate in a validation workshop to discuss most needed support systems and formulate policy advice to be shared with the Ministry of Cooperatives and MSME Development. In several cases informants also recalled that needs assessments are part of the project design methodologies implemented by UN entities. For example, the first step of the EPF is to assess the status of the national entrepreneurial ecosystem. According to the evaluation survey, the vast majority of respondents reported that project activities have responded to their priorities and were relevant to their work (Figure 3).

Figure 3: Perceived relevance of the project according to the survey (n=133 respondents).

Source: Independent evaluation survey, 2023.

33. Stakeholders, including policymakers, MSME entrepreneurs and development partners, participated in project design and implementation. On various occasions, evaluation informants stressed that the involvement of Member States and stakeholders in designing and implementing interventions was also a factor contributing to addressing their needs and priorities. In Kenya for example, in addition to ensuring buy-in from the Government before implementing cross-border trade trainings, UNCTAD relied on national institutions to facilitate project delivery on the ground. UNCTAD asked the Government to identify training participants and the Government requested local associations to identify participants and to help with logistics. A representative of the Ministry of Trade of Kenya travelled to the border to introduce the workshops. This was the first instance of high-level government support to the project. Several Empretec centres also mentioned that the identification of workshop participants was done in collaboration with government agencies and departments. In Zimbabwe, the Empretec centre selected primarily businesswomen and youth whose businesses were declining or faltering. Another example involved the creation of training materials on Core Indicators for Sustainability and SDG Reporting (GCI). A series of trainings of trainers (TOT) were organized by UNCTAD and participants’ feedback was used to revise the materials before training entrepreneurs.

34. Project activities and products were tailored to Member States and target beneficiaries. Project activities and products were often tailored to the needs of target recipients. Many regional reports were localized through national studies. For instance, ECE developed guidelines and best practices for MSMEs to

41%

44%

11%

1% 1%

1% 1%

The project activities responded to my priorities and were relevant to my work

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

23

assure resiliency and progress towards a circular economy in sustainable resource management and critical raw material supply chain solutions, as well as guidelines and best practices for MSMEs in delivering energy- efficient products and in providing renewable energy equipment27. Four countries of the ECE region were chosen as pilot countries for which the guidelines were customized, before spanning to other countries. In Tanzania, the Empretec centre delivered a training on entrepreneurship skills development that was based on the UNCTAD-EMPRETEC methodology but was adjusted to participants with limited literacy skills. UNECA delivered an online training course on the role of technology and innovation for MSMEs under Phase 2 and feedback from the training was used to develop a second course on role of green technologies in MSME development under Phase 3. For many activities and products, UN entities also ensured the translation of materials in official UN and local languages to facilitate access and outreach at global, regional and national levels. In the early months of the response, ECE partnered with UNDP (which had a Zoom license) to deliver workshops with simultaneous interpretation as this could not be accommodated by ECE online platforms.

35. Implementing UN entities adapted to maximize the relevance of interventions aimed for longer term change. While some Surge project outputs were found to be more crisis-response driven and with a shorter path to outcomes, such as the implementation of e-registration systems, other activities such as the design of a national entrepreneurship strategy were referred to as yielding an impact after a longer period of time. This is typical of DA projects and other socio-economic UN initiatives. Several interviewees were of the opinion that the UN socio-economic pillar did not generally have many crisis response products. Despite the achievements of the project (see section on Effectiveness), the capacity of the UN to respond to the needs and priorities of participating countries as a result of COVID-19 depended on institutional factors that were not always under their direct control. It was further indicated that at the onset of the pandemic, countries needed immediate financial support to help their companies and MSMEs to stay alive. Several survey respondents reported that complementing capacity-building activities for entrepreneurs with grants would have been beneficial28. Although the EPF and cluster-based approach proved an appropriate conceptual framework to articulate the response (see section on Coherence), the latter could have benefited from some prior analysis of the relief-recovery-development pathway to facilitate the identification of appropriate crisis mitigating interventions. Implementing UN entities therefore used adaptive management to ensure adequacy of the services in addressing demand-orientation and longer-term outcomes.

To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g., COVID-19 Socio-Economic Response Plan)?

36. The project was aligned with the COVID-19 Socio-Economic Response Plans (SERPs) in many participating countries, but the design occurred before the SERPs were finalized. The project was part of the general strategic plan of the United Nations called “A UN framework for the immediate socio-economic response to COVID-19”29 and specifically contributed to Pillar 3 “Economic Response and Recovery: Protecting Jobs, Small and Medium-Sized Enterprises, and Informal Sector Workers” of this system-wide framework30. This UN framework also served to guide the COVID-19 Socio-Economic Response Plans (SERPs) which were developed by UN Country Teams and Member States and stakeholders. However, while the concept note of the project was released in April 2020 and the Project Document for phase 2 in June 2020, many SERPs were published later, i.e. in or after August 202031. The design of Phase 3 of the project was finalized in early 2021 but most outputs were a continuation of previous work. Several informants also mentioned that the implementing UN entities have limited country presence, reducing the capability to mainstream the project in the SERPs. Despite these limitations, the evaluation reviewed a purposeful sample of 10 SERPs32 showing their systematic coverage of pillar 3 of the UN framework and some frequent references

27 UN Development Account project | ECE 28 See for example: Training and grant support for potential entrepreneurs and start-ups in the Sea of Azov region, Ukraine (ilo.org) 29 https://unsdg.un.org/sites/default/files/2020-04/UN-Framework-for-the-immediate-socio-economic-response-to- COVID-19.pdf 30 The UN framework built upon the UN General Assembly’s resolution on “Global solidarity to fight the coronavirus disease 2019 (COVID-19) (A/RES/74/270)” and on the Secretary-General’s report on “Shared responsibility, global solidarity: Responding to the socio-economic impacts of COVID-19”. 31 UNSDG | Resources Library 32 The evaluation selected 2 SERPs per region and countries where the project implemented interventions, i.e. Argentina, Bangladesh, Bhutan, BiH, Cameroon, Jordan, Moldova, Peru, Thailand, and Zimbabwe.

24

to or links with the outcome areas of the project, with a prevalence of connections with outcomes 4, 3 and 1A. The evaluation also found references sometimes to the project outputs. For example, the Socio-Economic Response and Recovery Plan for the Republic of Moldova (June 2020)33 was informed by the ECE’s survey- based assessment of the COVID-19-induced economic crisis and changes in non-tariff measures to contain the pandemic on MSMEs (May 2020) and by the Assessment on the impact of the crisis on female-owned enterprises (June 2020).

37. Independently of the SERPs, implementing UN entities considered the COVID-19 socio-economic responses of participating countries during project design and implementation. As noted in the earlier section, the project delivered many regional and national needs assessments and impact studies, which were a source for aligning project activities with the COVID-19 socio-economic responses of the participating countries. For example, UNECA organised, at the request of the Government of Mauritius and in association with the SADC Business Council, a regional seminar on the impact of Covid-19 on MSMEs in Southern Africa and the recommendations were intended to support the development of a roadmap for technical assistance at a regional level. In Latin America, ECLAC analysed the policies implemented by governments to face the MSME crisis generated by COVID-19. Experiences in countries having MSME fostering institutions were prioritized. Good practices were systematized and shared among public institutions in charge of MSMEs policies. Virtual meetings were organized to share implementation experiences about measures to help MSMEs in order to enhance the coordination of public policies for economic recovery. In the Gambia and Kenya, DESA ensured that the design and implementation of project activities were aligned with the national agendas to contribute to the relevance of the interventions and strengthen ownership by the countries. In the Gambia, work was carried out with the Ministry of Trade, Industry, Regional Integration and Employment (MOTIE), UNDP and the UNRCO. Project implementation served to strengthen the capacity of policy makers and other stakeholders to implement new policies and to pursue coherence among the already formulated policies in the country. Several evaluation informants indicated the project was relevant for bringing MSMEs to the policy debate where they were often absent.

5.2 Coherence

To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

38. The project was anchored in the mandates, comparative advantages, and programme budgets of the implementing UN entities, paving the way to complementarities with previous interventions. The initial concept note and Project Document presented a detailed review of the respective mandates and comparative advantages of the implementing UN entities as a rationale for their contribution to the project34. Among these

33 MDA_Socioeconomic-Response-Plan_2020.pdf (un.org) 34 Many factors were put forward to stress the institutional coherence and rationale for the project. Some highlights include: (i) UNCTAD is a focal point in the UN on entrepreneurship and MSME policy as formally mandated by two United Nations General Assembly resolutions on Entrepreneurship for Sustainable Development. UNGA resolutions /RES/71/221 and A/RES/73/225 both call on “the United Nations system, and in particular the United Nations Conference on Trade and Development, to continue to provide support to and assist member States, at their request, to identify, formulate, implement and assess coherent policy measures on entrepreneurship and the promotion of micro-, small and medium-sized enterprises”. UNCTAD also provides technical assistance and capacity-building support to a number of developing economies, including advice and training to policy makers to inform and guide the development and implementation of national entrepreneurship policies based on UNCTAD’s Entrepreneurship Policy Framework (EPF), which was endorsed by member states through a series of intergovernmental discussions and formally launched at the Ministerial Conference UNCTAD XIV in 2012. (ii) DESA brings relevant expertise from implementing the project “Enhancing national capacities for enhancing potentials of MSMEs in achieving SDGs in developing countries (MSME project)” funded by the UN Peace and Development Fund (PDF). (iii) ESCWA has developed and launched an SME information portal for the region with an aim to support entrepreneurs and small businesses access information needed to support their business. (iv) ECE has conducted rapid impact assessment of the COVID-19 pandemic on end-to-end supply chain activities, with a view to supporting national and regional recovery and long-term development efforts. (v) The ECA sub-regional office for Southern Africa leverages its mandate and implementation of a UNDA 13th tranche project for integrating a component to build capacities of MSMEs in Southern Africa to harness technological applications to address the impact of the pandemic. (vi) ECLAC has a long experience in studying the MSME performance in Latin America and in supporting public institutions in developing, implementing, and evaluating MSME fostering policies. (vii) ESCAP responds to ESCAP Resolution 70/5: Strengthening regional cooperation and capacity for enhanced trade and investment in support of sustainable development, and has planned to work with existing initiatives and organizations to support MSMEs in the region.

25

enabling institutional factors, references were primarily made to supporting policy makers on MSME development as well as to building capacities of MSMEs and entrepreneurs. Significant evidence was found showing that the project was consistent with and sometimes a continuation of previous activities. UNCTAD for instance relied on its network of Empretec Centres to deliver activities on the capacity-building of entrepreneurs. In collaboration with centres and experienced Empretec trainers, UNCTAD delivered on-line trainings and developed demand-driven, customized tools to assist the centres in coping with short-term and medium-term pandemic-induced challenges to their business operations. UNCTAD also built on earlier contacts to support the establishment of e-Registration platforms in El Salvador and Benin, which was accelerated by the pandemic. ESCWA grounded its proposed contribution to the 1st phase of the project in previous meetings and surveys. ECLAC delivered courses that were the result of prior studies and a long preparatory phase of getting to know the issues of SMEs in e-commerce. ESCAP’s “Bangladesh Startup Ecosystem Assessment Report”35 delivered by the Trade, Investment and Innovation Division followed earlier work from another division.

39. Interventions were often coordinated with and complementary to other internal activities that were underway in the implementing UN entities. Within each implementing UN entity, internal complementarity was sought with other projects. ECE for example used the impact assessments on the novel COVID-19 pandemic on end-to-end supply chain activities, conducted under UNDA 10 “Strengthening the national capacities of selected ECE countries for evidence-based regulatory and procedural trade policies to achieve SDGs”, to further draw two national assessments in Armenia and Moldova focusing on female-owned enterprises. DESA leveraged its expertise and understanding of country needs gained with the project “Enhancing national capacities for enhancing potentials of MSMEs in achieving SDGs in developing countries (MSME project)”36 funded by the UN Peace and Development Fund (PDF), DESA complemented the PDF Kenya and The Gambia to improve contributions of MSMEs to social and economic resurgence after the pandemic and strengthen their roles as enduring institutions for jobs and entrepreneurship to reach those further behind. ESCAP took advantage of the Asia-Pacific MSME Policymakers Network established with the project to present and launch the 2nd edition of the “Policy Guidebook for (M)SME Development in Asia and the Pacific”37, and its accompanying “Online Resource Centre for MSME Development”. Several knowledge products from ESCAP were also co-financed by the project and other sources. For instance, the publication on MSME Access to Finance: The Role of Digital Payments38 was funded from the Regular Programme of Technical Cooperation for the studies on Cambodia and Nepal, and from the Government of Canada (through Global Affairs Canada, in the context of ESCAP’s Catalysing Women’s Entrepreneurship Programme), for the studies on Bangladesh and Samoa. The UNECA Sub-regional office for Southern Africa leveraged the project to build the foundations of a UNDA 12th Tranche project on innovation and technology for SMEs, which had been approved. ESCWA presented the DEPAR portal39 and MSME toolbox40 at the first Arab SME Summit41, organized in Jordan from 30 October to 1 November 2022 by another ESCWA project.

40. The Surge project increased internal collaborations between departments in the implementing UN entities. Evaluation informants often shared the perspective that the project has contributed to increased communication between internal departments that were used to working separately. Several project outputs were developed on the basis of internal cross-collaborations between teams. In UNCTAD for example, the SME Entrepreneurship Branch and the Empretec centres collaborated with the Trade and Gender team, the Competition Branch, and the Accounting team. Staff from these sections participated in coordination meetings, exchanged tools and new materials, and were consulted to identify beneficiaries. The UNCTAD Business registration team also commended the project for providing an opportunity to have an open channel of communication with the Entrepreneurship section. This contributed to sharing experiences and good practices and for these colleagues to be better positioned to promote e-Registration. In ECE, project activities helped to break down silos and foster collaboration between different teams working in the area of sustainable trade, innovation, circular economy, rational use of natural resources, PPPs and women’s empowerment. In ESCWA, the work on the MSME toolbox involved 7 divisions, reflective of the 7 foci of the toolbox. Each specific team

35 Bangladesh startup ecosystem assessment report | ESCAP (unescap.org) 36 UNPDF | Enhancing national capacities for unleashing full potentials | United Nations 37 Policy guidebook for MSME development in Asia and the Pacific, 2nd edition | ESCAP (unescap.org) 38 ESCAP. 2022. MSME Access to Finance: The Role of Digital Payments, MSME Financing Series No.7. Bangkok. Available at: https://www.unescap.org/kp/2022/msme-financing-series-role-digital-payments. 39 Communities | DEPAR (unescwa.org) 40 MSME Toolbox Dashboard | DEPAR (unescwa.org) 41 Arab SMEs Summit | DEPAR (unescwa.org)

26

helped with research, data collection, and finalising materials. Weekly meetings were organized with the coordinators under each division. The teams used Trello to see what others were doing. In ECLAC, the International Trade and Integration Division and the Production, Productivity and Management Division jointly developed a course promoting digital economy and digital trade. In ECA, the ECA subregional office for Southern Africa (SRO-SA) and the ECA Digital Center of Excellence (DCE) collaborated to jointly produce a study on the “Role of Digitalisation in Strengthening Capacities of Micro, Small and Medium-size Enterprises (MSMEs) in Southern Africa”42. The ECA Subregional Office for Southern Africa collaborated with the ECA Africa Institute for Economic Development and Planning (IDEP) to deliver two training courses on role of technology and innovation for MSMEs. In some cases, informants mentioned that the Surge coordination meetings organized by UNCTAD helped identify other units from the same implementing UN entity involved in the project. Despite such examples, other staff also stressed that the overall emergency context meant there was little time to pursue internal synergies as teams concentrated most often on activities under their direct responsibility and on achieving immediate results.

41. The project enabled some level of technical collaboration and complementarities between implementing UN entities. According to the concept note, the project was conceived to propose a coherent approach towards MSME resurgence reflecting the holistic and comprehensive nature of efforts needed to ensure and facilitate the MSMEs’ green, resilient and inclusive recovery. Activities sought complementary and synergies in impact among participating agencies covering key interrelated components of an enabling entrepreneurship ecosystem. Inter-agency collaborations expanded the dissemination of agency-specific products while coordination and communications helped to avoid overlapping activities. Some joint products were also developed by the implementing UN entities, most often involving UNCTAD as one of the partners (see also next paragraph). UNCTAD and ESCWA, for example, cooperated to inform and build awareness on the e-registration platform for MSMEs. ESCWA leveraged US$ 30,000 from the project to organize a capacity development workshop with UNCTAD to familiarize member states in the Arab region with the e-registration system. This generated interest and ESCWA was later able to mobilise US$ 600,000 from different sources for two e-registration projects in Jordan and Syria43. UNCTAD also produced a global report on market access and competition policy44 to which ESCWA and the other Regional Economic Commissions contributed through advice, inputs, data, and review of the document. In Europe, ECE’s COVID-19 impact assessment targeting MSMEs in Georgia45 spurred discussions with the Ministry of Economy and Sustainable Development leading to the integration of Georgia’s non-tariff measures (NTM) information into UNCTAD’s NTM database46,. In Romania, ECE took up the Empretec methodology and provided grants for the Asociatia pentru Antreprenoriat din Romania (APAR) to conduct two in-person workshops of 9 days each which were attended by 60 MSMEs. In coordination with the Regional Economic Commissions, UNCTAD organized a series of regional capacity-building events to raise the awareness of policy makers on the role of competition-related policies for access to markets in the post-COVID-19 resurgence of MSMEs. UNCTAD, ESCAP and the Trade and Competition Commission of Thailand (TCCT) organized, for example, a conference in June 2022 on the Contribution of Competition Policy to the Resurgence of MSMEs post-COVID-1947, to which DESA and ESCWA also contributed. UNECA and UNCTAD co-organised a Regional Policy Dialogue on “The role of Competition Policy in supporting MSMEs economic recovery in the post COVID-19 crisis” and collaborated on a case study on South Africa for the UNCTAD global report on “How Covid-19 affects MSME access to markets and competition: a review of key issues and recommendations for future action”. Evaluation informants also mentioned that the project provided a unique and successful opportunity to learn about the work of the other implementing UN entities in the area of entrepreneurship and MSME development, therefore contributing to the “One UN” agenda. The project website48, developed to gather outputs and increase their visibility and sustainability, was also commended by many informants. However, most of these collaborations had a

42 The role of digitalisation in strengthening capacities of Micro, Small and Medium-Size Enterprises (MSMEs in Southern Africa to take advantage of the AfCFTA): ECA Sub-Regional Office for Southern Africa (SRO-SA) & ECA Digital Centre of Excellence (uneca.org) 43 ESCWA also mobilized an additional US$300.000 for an e-registration project in Sudan but the political context in the country did not allow for implementation. 44 The covid-19 pandemic impact on micro, small and medium sized enterprises: Market access challenges and competition policy (unctad.org) 45 Impact_COVID-19_Georgia.pdf (unece.org) 46 UNCTAD TRAINS 47 Conference on the Contribution of Competition Policy to the Resurgence of MSMEs post-COVID-19 | UNCTAD 48 Home | UNCTAD

27

bilateral character and the evaluation noted the difficulties for Regional Commissions to collaborate with one another.

42. Several factors limited opportunities for amplifying synergies between interventions and for establishing a stronger global coherence. From a total of 100 project outputs formulated across the three phases of the project logframe, only 6 were identified by the evaluation as joint outputs from 2 implementing UN entities (Figure 4, in yellow), out of which just 4 were effectively delivered (i.e., UNCTAD and ECLAC on outcome 5). Joint outputs were formulated for phase 3 of the project, owing to several factors such as the short duration of phases 1 and 2 and the need for a sufficient amount of time to design joint interventions identified after a national agenda, the number of implementing partners progressively increasing between phases 1 and 2, or the limited cross-institutional awareness of each partner’s expertise on and capacities for supporting MSMEs.

Figure 4: Project outputs per implementing UN entity and phase.

Note: Joint outputs are referred to in yellow. Source: PRODOC.

43. Staff from different Regional Commissions attempted to establish synergies with other Regional Commissions for this project but with limited success. ESCWA for example reached out to ECE and ECA to seek their prior experience in supporting MSMEs with a view to uploading their materials on the DEPAR portal to be used in the region and globally. Despite several meetings and some exchange of materials, there was no concerted effort to populate the DEPAR portal. Several evaluation informants in UNCTAD and the Regional Commissions seconded the perspective of an ECE staff member that “everybody was too busy and there was no time to make the effort of cooperation”. According to a staff member, if “capacity development by Regional Commissions is to be jointly done, this is a long process cycle” not suited to a crisis situation. Evaluation informants also mentioned that the geographical focus and mandates of the Regional Commissions created various challenges to collaboration as the Regional Commissions come with different areas of expertise and niches. Furthermore, needs of Member States also varied across and within regions. In addition, although the EPF was found to be a comprehensive and flexible framework, none of the implementing UN entities outside of UNCTAD had the capacity to cover all pillars. On average, UN entities covered about 2 out of 5+1 project outcomes (Figure 5), i.e., they remained geographically and technically focused. Accordingly, several informants shared the perspective that the project was a compilation of activities and outputs based on a unifying and valid conceptual framework (i.e., the EPF) and cluster-based approach but with limited synergies, moderate coherence, and high overall complexity. According to a staff member at ESCAP, “The project did not install joint indicators and reporting. If there was to be a joint event, it would indicate specifically who would deliver what. It was not about the project producing a pool of papers without referring to one or another agency; products were assigned by entity. From then on, it was clear agencies would have different streams of work.” Some evaluation informants questioned the cost and benefit of a global project in a crisis context compared to several inter-regional projects (see section on Efficiency), despite this project enabling strong knowledge exchange between participants.

Figure 5: Number of project outcomes covered by the implementing UN entities.

28

Source: Final report. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID-19 in delivering socio-economic support to Member States?

44. The project triggered some collaboration with other UN entities at national, regional, and global levels. At global level, UNCTAD partnered with UNITAR to provide the e-learning course “Policymaking: Entrepreneurship for Sustainable Development”49, which was followed by 317 policymakers from over 104 countries. ESCAP also published a series of knowledge products on MSME Financing50 that benefitted from a partnership between ESCAP and the United Nations Capital Development Fund (UNCDF). Complementarities with other UN organisations were also at national level. In several cases, the UN RCOs and UNCTs were referred to as project partners. In the Gambia, the Ministry of Trade, Industry, Regional Integration and Employment together with DSDG/DESA and the UN Country Team organized a high-level policy implementation forum in December 2020 on supporting MSMEs in the post-Covid-19 pandemic era, focusing on promoting MSME formalization51. In Europe and Central Asia, UNDP was often involved substantively and/or logistically in the delivery of online or hybrid workshops and dissemination events. In Armenia for example, ECE, UNDP, and other partners organized in March 2022 a conference on “Unveiling Market Opportunities for Boosting Residential Energy Efficiency and Alleviating Energy Poverty”52. UN Women also contributed to the development of a roadmap in Moldova based on the recommendations outlined in a study on women-led MSMEs, conducted by the project. Country office staff from several UN organisations including FAO, UNDP, OHCHR, and UNHCR also attended some of the online trainings or events delivered by the project. Despite such examples, evaluation informants were often of the opinion that the project had not maximised partnerships with other UN entities owing to several constraints or limitations, including the respective crisis response priorities and challenges in UN development agencies, limited staffing capacities from the project to focus on partnership building at global level, limited funds directed towards creating synergies with other agencies, lack of dedicated outputs and indicators referring to joint work with other agencies, and the lack of country presence of the implementing UN entities (see also §46).

45. The project was implemented with a range of partners that complemented the responses of the UN entities. Besides UN entities, project implementation involved a range of other partners at international, regional and national levels that helped to complement the responses from other UN organizations. Expert speakers in ECE events included, for example, important players in greening the MSMEs agenda, including the International Chamber of Commerce (ICC), World Trade Organization (WTO), Organisation of Economic Co- operation and Development (OECD), European Bank for Reconstruction and Development (EBRD), and a number of national authorities and private sector players. In Southern Africa, the project contributed to strengthening the ECA SRO SA partnership with the SADC Business Council and national chambers of commerce and industry in all 11 Southern African countries. Joint events were organized by ECA and the SADC Business Council at the SADC Industrialisation Week in Malawi and the Dubai Expo 2020. ECA also developed a digital platform on technology and innovation for MSMEs to be hosted by the SADC Business Council. According to the evaluation survey, the majority of respondents were of the opinion that the project complemented interventions implemented by other international organizations working in their

49 E-Learning Course on Policymaking: Entrepreneurship for Sustainable Development | UNCTAD 50 ESCAP’s MSME Financing Series | ESCAP (unescap.org) 51 High Level Policy Forum on MSMEs | United Nations Development Programme (undp.org) 52 From buildings to small companies, UNECE supports countries to reap benefits of improving energy efficiency | UNECE

1 1 1

5

1

2

3

2

1 1

6

1 1

2 2

5

1

0

2

4

6

UNCTAD DESA UNECA ESCAP UNECE ESCWA ECLAC

Phase 1 Phase 2 Phase 3

29

country/region (Figure 6). As illustrated by a private sector representative in North Macedonia, “A lot of organizations were conducting projects regarding COVID. However, this one was the only one providing actual information on practical ways to mitigate some challenges arising from the pandemic”.

Figure 6: Perceived complementarity of the project with other interventions according to the survey (n=130 respondents).

Source: Independent evaluation survey, 2023.

46. Partnerships with other UN entities and organisations proved challenging to harness. Despite the above and some other cases, the evaluation found that the project was not able to establish a cohesive eco- system of strategic partnerships. The initial concept note of the project considered that project governance would establish working groups on each outcome area, including “Digital economy solutions for advancing SME resilience (ITC, ITU, UNCDF, UN/CEFACT)”, “Resource Efficient and Cleaner production (led by UNEP and UNIDO)”, “Zoning/urban issues (led by UN-Habitat)”, “Employment (led by ILO)”. The Project Document for Phase 2 of the project also mentioned that “Close cooperation and partnerships should also be sought with World Bank, IFC.” This proved over-ambitious. In practice, the evaluation found limited global collaboration with UN entities also engaged in responding to the crisis and providing support to MSMEs, such as the ILO53, UN Women54, or FAO55. The evaluation also did not find evidence of the project attempting to leverage funding from other UN COVID-19 response mechanisms, such as the UN COVID-19 Response & Recovery Multi- Partner Trust Fund (MPTF) which financed projects at the national or subnational level in some of the countries where the project was implemented. Some informants stressed that Member States were looking for financial resources to support MSME resurgence, which is rather in the realm of the development banks. However, connections with the World Bank and the regional banks were limited. Partnerships appeared, therefore, primarily based on a bottom-up approach, relying on legacy collaborations or immediate opportunities, but not necessarily globally strategized and pursued. It should be noted though that the project did not dedicate specific resources to such activities (see next section).

5.3 Efficiency

How well coordinated was the process for the response among the entities implementing the joint project?

47. Despite unprecedented circumstances with the Covid-19 pandemic, and the lack of clear dedicated resources, the administrative, management and financial processes for the response were well coordinated overall. According to the initial concept note, Phase 1 of the project aimed to implement two components.

53 wcms_854253.pdf (ilo.org) 54 WEA-ActionBrief3-SMEs.pdf (weps.org) 55 COVID-19 Response and Recovery Programme | Resource Mobilization | Food and Agriculture Organization of the United Nations (fao.org)

28%

43%

18%

3% 2% 1%

5%

The project complemented interventions by other international organizations working in my country/region

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

30

Component 1 was to be led by ESCWA and planned to establish a virtual knowledge hub. Component 2 was to be led by UNCTAD and targeted the provision of hands-on assistance to countries on several sub- components grouped as 1+5 thematic clusters56. Although the narrative of the concept note did not identify an agency in charge of the overall coordination of the project, the budget featured the output “Coordination and proposal preparation” for which UNCTAD was to be provided US$ 27,000. This lack of clarity in the initial setup of the coordination of the project was reflected in the evaluation interviews, informants from the various implementing UN entities, including UNCTAD and ESCWA, having different perspectives on the scope of work and responsibilities for the coordination of the process during the first phase of the project (see also section on Effectiveness). The Project Document for Phase 2 clarified that UNCTAD would be the lead agency for the overall coordination of the project. This role was reflected in the budget of the project, with UNCTAD receiving US$ 40,000 for “Output –.1 - Overall coordination”, but not included in the project logframe. A Project Steering Committee was to monitor project delivery and make necessary adjustments to ensure smooth implementation 57. In practice, evaluation informants distinguished three different periods regarding the coordination process. The first period coincided largely with the preparation and implementation of Phase 1 of the project (from April to June 2020) during which the DA-PMT was reported by UN entities’ informants as having a very active coordination role. The second period spanned from August 2020 until April 2021, covering Phase 2 and part of Phase 3 of the project. It had regular (i.e., about bi-monthly) Project Steering Committee meetings coordinated by UNCTAD. Then, from April 2021 until the closure of the project, a third period saw the Project Steering Committee meetings attended by a larger number of participants (up to 40 participants). Project Steering Committee meetings held during the 1st, 2nd and 3rd phases were not technical but provided room to discuss and align activities. According to an informant who attended the 3rd phase meetings, “they were also used by UNCTAD to propose participants to build synergies on topics, but the project was so broad that this seldom happened (sic)”. Most often, the evaluation informants reported that these 3rd phase meetings were useful to learn about the activities of other implementing entities. However, the complexity of the project also made these 3rd phase meetings at times quite long and difficult to follow. Furthermore, the different time zones made it challenging for all staff to participate. In terms of administrative and financial management, project staff in UNCTAD reported an efficient and supportive DA team. This was valued as, according to a staff in UNCTAD, “the UN requirements to obtain the funds are onerous. There is a need for expertise for administrative management and administrative rules to prepare requests and reporting.” Monthly financial statements were provided to project management in UNCTAD. The DA also helped in understanding who among the UN entities was behind schedule and who was on track. According to an informant in UNCTAD, “In the project management meeting, one UN entity could have a narrative saying that all was going great but then, with the DA, we would see the financial delivery was only 12%. Then we knew we had to push delivery. The DA provided the implementation rate. Very helpful.”58 To support its internal coordination, UNCTAD specifically designed in April 2021 a new product, the project monitoring dashboard, consolidating information on UNCTAD’s planned outputs and how each intervention was progressing. Several informants from UNCTAD reported this as a very useful, centralized tool. The dashboard was regularly updated, despite being demanding to maintain. Collecting information from staff was time-consuming. Furthermore, the dashboard was very detailed, recording information at the activity level (some outputs could entail a dozen activities) and across 11 data fields. In preparation for the progress report for Phases 1 & 2 in May 2021, the dashboard was used to compile information from all the implementing UN entities. Despite most informants reporting appropriate and efficient project coordination considering the circumstances, informants in UNCTAD, and across the partner entities, shared the perspective that the project could have benefited from additional staffing capacities dedicated to global coordination as well as to coordination with countries and UNCTs, and to partnership building.

48. The technical coordination of the response proved sometimes difficult to install and sustain. According to the concept note (Phase 1), working groups and task forces were to be formed around thematic clusters to support a coordinated approach and complementary efforts among participating agencies. It was

56 The concept note indicated the creation of the following clusters: A- Access to Resources (Component 1); B- Entrepreneurship and business skills facilitation; C - Business facilitation; D- Access to Finance/financial literacy; E- Access to Technology; F- Access to Markets. 57 As per the Project Document for Phase 2: “Project Steering Committee - Representatives from UNCTAD, DESA, ECA. ECE, ECLAC, ESCAP and ESCWA will have online meetings once in two months to monitor project delivery to ensure smooth and steady implementation. Representatives from all seven entities will meet on an as-need basis for important decisions that affect all seven entities.” (p. 53). 58 As a side note, DA Focal Points reported that meetings of the DA Focal Points network became more dynamic and substantive with the response to the Covid crisis.

31

initially envisaged that members of these task forces would include the implementing UN entities as well as other UN partners. The Project Document (Phase 2) narrowed down the prospective setup for the clusters’ technical coordination by referring to Task Forces comprised of experts from the implementing entities. According to an informant, “The discontinuation of the original concept note was positive because it was unrealistic. The problem is that staff members were on other jobs, no one exclusively working on the project. It was time-consuming to coordinate and staff had different agendas and there were different priorities between partners.” In practice, this cluster-based thematic coordination was loosely implemented but fluid, sometimes involving sharing of information between several implementing agencies, though more often one- on-one bilateral and direct exchanges. As noted earlier, interviewees reported information sharing with UNCTAD and sometimes directly between the Regional Commissions “but then entities were doing their own part”. One of the cluster leads for example proposed that the Regional Commissions undertake a global analysis on market access and COVID-19 to develop a joint policy paper, but this did not work: “Colleagues wanted it but there were so many things to do. Project managers did not approve it and it was not in the logframe.” In that sense, the technical coordination of the project and its contribution to the efficiency of the response was eventually more closely aligned with the aim stated in the initial concept note as “(t)he effort will help to enhance coordination in the area of the MSME promotion in the UN system, and avoid confusion, duplication, and waste of resources”. Informants in UNCTAD and the Regional Commissions shared the perspective that directing adequate staffing capacities specifically to technical coordination may have facilitated the establishment of stronger synergies between the implementing entities. Furthermore, despite most UN informants being supportive of the global architecture of the project, including as a modality to enlarge partnerships and seize the benefits of a “One UN” system approach, a few staff from the Regional Commissions suggested to commit more time in the future to analyse the added value of global/inter-regional versus regional projects. It was argued that coordination costs increase with the complexity of the project and with the number of actors involved. As indicated by a staff in ECE, options to balance would include designing a global project or inter-regional or regional projects “and then coming together for dissemination at global level". Therefore, “the added value of a global project should be thoroughly assessed before project design. But during Covid it was a global challenge. […] Regional projects might have been an option. If going through inter-regional projects, their added value should also be well assessed. But in crisis, with the short timing and need to bring everybody under the project, one needs to cut some corners.”

How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

49. The phased approach was found to be a valid mechanism to cope with the uncertainty of the crisis. The project was developed and implemented under a three-phase budgeting and programming approach: Phase 1 spanned over 2 months (May-June 2020), Phase 2 was implemented in 6 months (July-December 2020), and Phase 3 was provided a total of 18 months (initially January-December 2021, extended twice, first to March 2022 and then to June 2022). On the programming side, during Phase 1 various entities were able to leverage existing products to deliver some immediate responses to countries that had requested assistance. In general, during Phase 2, the project contributed to develop tools, deliver technical support, and customizing materials required by countries. Phase 3 enabled implementation at country level of the tools developed during phase 2 and, building on country experience, to collect good practices and lessons learned that further informed policy makers. Technical staff interviewed by the evaluation were largely positive about the 3- phased approach. Interviewees recalled that this was a very difficult period and that it was unclear when the pandemic would end. Although a clear project plan for the entire period would have been beneficial, the context was too unstable to warrant it. According to a staff member in one of the Regional Commissions, the 3-phased approach “was the optimal and rational thing to do”. It was well aligned with the uncertain circumstances and supportive of a learning process. According to staff at UNCTAD, “Phases helped adaptive management” as “each phase was an innovative stage that gave a chance to introduce new things not foreseen at the beginning”. Incidentally, as put forward by a staff member in one of the Regional Commissions, it shows that the UN had an agile way of thinking to cope with the challenge. On a few occasions, a more nuanced assessment was conveyed by interviewees. Several informants reported that the phased approach added to the complexity and uncertainty of the project and made it more difficult to understand its design, and that a Theory of Change was missing.

50. The project would have benefited from providing clearer visibility on the funding available. From a financial standpoint, the five Development Account COVID-19 projects, including this project, were funded from various DA funding sources. This entailed relying on interest savings, residual balances from the 10th and other closed tranches, and savings from the 11th tranche. Accordingly, the overall available funding was

32

confirmed by phase. It was determined in late 2021 that the combined proposed phase 3 budgets submitted by the five projects exceeded the funding that became available by 1 million USD. As a result, the projects were requested in November 2021 to reduce their proposed phase 3 budgets, and for this project, the budget was reduced by $310,000, from a total of $3,165,500 to $2,855,500. UNCTAD received the final instalment for phase 3 in December 2021. Due to the requirement to reduce the budget and revise the planned activities for phase 3, all five projects were extended by three months to March 2022. However, according to informants at UNCTAD, this initial 3-month extension did not fully account for the late final instalment in the context of a global crisis. A UN staff member mentioned, for example, that if implementing UN entities “want to recruit a consultant, they need funds, then to identify candidates, then 2-3 weeks to issue the contract, then the consultant needs to do the work. That can take 3 months in normal circumstances”. However, the context of the pandemic imposed additional delays. Some activities that were planned to be delivered in person had to be delivered online instead in 2022, which required time for reprogramming based on the new budget. Furthermore, counterparts were also challenged as there were uneven capacities in beneficiary countries to put in place some of the interventions. Accordingly, activities for which disbursements were made shortly before the closure of the project in June 2022 were still underway several months later. Informants indicated that this contributed to the delayed completion of the final report of the project, together with limited resources for project monitoring and reporting in the Regional Commissions, particularly when the DA Focal Points were not associated with project implementation (see next section). Some staff also felt that the phased approach created a lack of visibility on the funds that would be mobilized by the DA. This was sometimes found to be a challenge when designing the project, as it prevented setting clear expectations and a clear design. Better visibility on the resources in the pipeline could have helped to construct a different logframe, which would have been designed at phase 2 and used at phase 3 with little revision. According to a UN staff member, a clearer view on the funding available for the entire duration of the project could have helped the teams to consider additional scenarios, such as focusing some of the activities on a more limited number of countries with greater depth.

5.4 Effectiveness

To what extent did the programme (Development Account) and project governance and management structures and processes enable, or hinder, the effective implementation of the joint project and the achievement of its results?

51. The Development Account programme, as a funding modality, was an effective tool in supporting the design of a global response to support the resurgence of MSMEs post-COVID-19, with remaining questions about it being the most adequate tool in comparison with other UN agencies and mechanisms with a mandate for humanitarian response. DA is a regular budget capacity development programme. It funds, among others, operational workshops, advisory services, and travels to countries to deliver capacity-building activities. Projects are approved by the General Assembly as part of the programme budget for the first year in the biennium. The project implementation period (from receipt of funds by implementing entities to the financial closure of the project) usually is 4 years, following a design or planning period that takes about 1.5 years. 59 Given project specificities and circumstances, and, possibly, limited knowledge of the DA rules and procedures, evaluation participants reported different views on the duration of DA-funded project cycles, the requirements for partnerships and indicators, and the DA governance structure. The converging perception is that a regular DA-funded project cycle is long and onerous to plan for. Nevertheless, to provide a swift and global answer to the COVID-19 crisis, an adaptive management approach was adopted. These procedures were made flexible, the design phase was significantly shortened, and a phased planning was introduced. For the five COVID-19 projects, the Surge project included, the GA approval of project concepts was skipped, while the endorsement by the DA Steering Committee remained and the use of residual balances from closed and closing projects to fund the five projects was retrospectively reported in the fascicle for 2022, which was submitted to the GA in 2021.60 Hence, the concept note of the Surge project and 1.5 years of implementation

59 Under regular tranches of DA, the design or planning phase of projects starts with the launch of a DA new tranche by the USG for Economic and Social Affairs and the development of concept notes by 10 DA entities, and ends with the approval of the project documents by the DA Programme Management Team and the endorsement of DA budget and funded projects by the General Assembly. 60 A report commentator referred that “this has happened in the past when developing projects using residual balances from the tranche budget, but has never happened, nor is allowed to happen, for regular tranche projects.”

33

period was endorsed to initially respond as quickly as possible to the urgent requests from Member States to address the socio-economic fallouts of the pandemic. It was widely reported by informants that this was the best approach to the circumstances, as no one knew how long the crisis would last, with expectations that it could be just a few months. An advantage of this alternative path to access residual balances from closed and closing projects, reported to this evaluation by a DA representative, is the almost immediate availability of resources. This promptness is praised by stakeholders in many implementing entities as a ‘unique’ achievement.

52. Nevertheless, when the DA was compared with other UN agencies and mechanisms with mandate and resources to provide humanitarian assistance in a crisis context, informants referred to the difficulties to be as prepared as these other agencies. The strategy, then, was to focus on post-crisis recovery and development, as a suitable DA approach to actively support countries and, specifically to the Surge project, MSMEs. Yet, informants from the regions referred that adaptations to the project document guidelines kept the highly demanding requirements of the fund and this was assessed by them as an issue to be reviewed if the fund is to be used in future crises. A better crisis-fit approach would have involved an even more agile track, and a focus on deliverables of immediate effect to mitigate the crisis’ impact on MSMEs and government policies (e.g., enhanced access to emergency finance and digital tools for trade). Informants shared expectations about the DA-PMT’s review of the five COVID-19 evaluations toward helping to inform the DA’s role in dealing with future shocks.61 Among members of the project coordination team, for instance, there are ideas about the usefulness of the Entrepreneurship Policy Framework as an overarching tool with relevant potential outputs for crisis response, as discussed later. Considering that the EPF is the key framework of UNCTAD to fulfil its role as the UN focal point for entrepreneurship and MSME policy for sustainable development (RES/71/221 and A/RES/73/225), EPF-related ideas can be considered as a tool to support crisis response funded by the UNDA.

53. Project governance and management structures and processes, overseen by the DA-PMT, supported a quick and overall effective response in light of the UNDA standards, identifying available resources and potential partnerships that ensured the delivery of an impressive number of outputs and achievements at the outcome level, even if joint delivery between implementing entities took time to come to fruition and some DA-FPs were ambivalent about the extent to which they were involved. Addressing a crisis of global scope such as the COVID-19 pandemic was new to the DA’s management structure. Seeking to find quick and effective ways to deal with it, the Head of the Capacity Development Programme Management Office of DESA, in his role of assisting the DA Programme Manager, sent out a memo to the DA focal points in the 10 implementing entities on 18 March 2020, inviting them to a Skype call to discuss the DA's COVID-19 support to Member States, in particular the possible reorientation of the ongoing 10th, 11th and 12th tranche projects, on 20 March 2020. The meeting minutes indicate that the participants were informed of two proposals for potential response initiatives from ESCWA (on SMEs) and from DESA (on statistics). A follow-up meeting was convened on 26 March, where five other project proposals were discussed, and other implementing entities were requested to provide comments on those proposals, as well as submit any additional proposals. After these extensive consultations with DA-implementing entities, DA-PMT selected five topics considered as the most relevant to a socio-economic response to COVID-19 and built on comparative advantages of DA- implementing entities. This included strengthening the capacity and resilience of MSMEs to mitigate the economic and social impact of the pandemic – ESCWA’s proposal. The preliminary idea of this project, as reflected in its 1-page concept note, focused on digital tools to support SMEs. UNCTAD further enriched it with additional ideas, such as the e-registration platform and entrepreneurship for SMEs. The project concept note finally took shape with the inputs of more entities, including DESA and other four regional commissions. As more implementing entities joined the project, UNCTAD supported the clustering of activities in 5+1 outcomes, with implementing entities taking outcome leadership according to their expertise and capacities. This structuring of the project governance, with DA-PMT’s oversight, and the management structure with UNCTAD in the overall coordinating role and each implementing entity leading the operational implementation at regional level ensured the global and regional scope of the project delivery. At the level of the ongoing processes, implementing entities relied more on partnerships with local and regional institutions rather than coordination within the larger project management structure. The online meetings of the steering committee

61 An updated information about this review received in feedback to this evaluation report refers that “this programme- level evaluation had to be cancelled due to the delay with the completion of four of the five project evaluations (only one has been completed to date, and two are expected to spill into 2024). A meeting of the DA focal points to discuss the findings of the five project evaluations will be organized by DA-PMT as soon as they are completed; the DA's role in dealing with future shocks would certainly be discussed at this meeting.”

34

and the smaller-scale coordinating role of the task forces to weave efforts worked to a limited extent, as these meetings were not as regular as planned in the project document. The result was a high delivery rate with limited inter-agency coordination in the process. Yet, this independent evaluation identified that the higher- level governance and management structures and processes were effective in bringing the project together with identifiable results at the outcome level.

54. Given the uncertainties about how the pandemic would evolve, and considering the pressing immediate needs, a first phase of the project was launched with a limited number of outputs (n=7), in May 2020. It resulted from meetings of the DA Network in March and April 2020. From April 2020, these became bi-weekly meetings of the DA focal points. It is noteworthy that this process was not recollected as such by the informants of this evaluation. There is a perception that not all DA-FPs were involved from the start, that this inclusion was gradual and followed advances in the project. Informants acknowledged that these meetings also included staff from the implementing entities to discuss the next steps to be taken, the different country and regional needs, and the installed regional expertise to propose responsive actions62. This trickled down into several smaller-scale bilateral meetings at the regional and local levels. It became clear that the DA’s mandate on development had to be taken into consideration when planning the project outputs and adapting activities associated with crisis response (e.g., providing direct health support was out of the scope of possible answers). The DA’s development mandate, instead, could support the strengthening of institutions that needed preparedness and resilience to design supportive measures in a post-pandemic recovery perspective. Hence, out of five projects, the Surge was one with the objective of developing and implementing capacity-building tools for governments and MSMEs to facilitate the resurgence and strengthen the resilience of MSMEs in developing countries and economies in transition. There was a need to bring the relevant expertise of different entities quickly on board. Because of its global mandate and expertise on entrepreneurship and MSME policy, UNCTAD became the lead agency of the Surge project, and coordinated the elaboration of a project document to deliver a global response with more entities. For the first time, the DA-PMT was intensively involved in the oversight and monitoring of project implementation, with more direct contacts with the technical cooperation staff – rather than going through organizational-level DA focal points. “The DA PMT was in contact with the DA-FPs but had more contacts with the project managers than it would normally have.” (DA-PMT representative). This was made possible by the small number of projects responding to the COVID-19 crisis. This centralized approach allowed for a swift response at a global level, with a record time endorsement of the concept note and the Phase 1 budget by the DA Steering Committee. DA-PMT’s oversight and direct contact with technical staff resulted in the institutionalization of periodic meetings with DA-FPs and other stakeholders. UNCTAD DA-FP, represented by the Technical Cooperation Section (TCS), served as a bridge between the DA-PMT and UNCTAD project teams, especially at the design stage. UNCTAD TCS also provided substantive support to the UNCTAD project coordinator in the draft of the concept note of the MSME Surge project. During the implementation stage, UNCTAD DA-FP participated in several project Steering Committee meetings and addressed questions from project teams. UNCTAD, as the leading entity, and under the guidance of the DA-PMT, communicated directly with the project focal points to monitor progress. This monitoring was supported by UNCTAD Budget and Project Finance Section through the sharing of financial information on implementation rate by the implementing entity. This, and the evolution of the pandemic, brought more clarity about the needs and the implementing capacity of UN entities and member states in responding to the crisis. It supported the design and implementation of phases 2 and 3. There is consensus between informants that the introduction of the phased planning was consistent with the uncertainties of the pandemic crisis, and they would not recommend another way to respond to it, despite reporting some bottlenecks.

55. There are different perspectives on how well the DA procedures were adapted for crisis response. Following the requirements for joint proposals as much as possible, according to informants, felt like squeezing a 4-year long implementation period into 1.5 years. It was suggested by some informants that, instead of a concept note and a PRODOC, a crisis response could have required only one of them. Outcomes were over- ambitious for a 2-year project, but project planners felt they had to comply with the DA requirements, even if this would imply challenges to deliver at the outcome level. Indeed, some outputs were delivered after the end of the project (see next section). There was and internal quality control of the PRODOC with multiple inputs, oversaw by the DA-PMT to ensure a ‘good enough quality’ level, however, a higher-level involvement of all relevant DA-FPs in dialogues and feedback loops from the start and the gender review and other quality assurance steps (i.e., internal clearance processes of the implementing entities of interim documents) could

62 Other informants referred to these meetings as more about sharing information than coordination between implementing entities.

35

have improved the quality of the PRODOC further. For instance, it could have ensured that project indicators cover the SMART63 criteria and be better aligned with a crisis response context – some were the same Empretec-based indicators in use for the past 10 years. In one example suggested by an informant, instead of having an indicator on increased sales of MSMEs, the project could have one about ensuring that a certain percentage of MSMEs that received project support remain in business. Informants confirmed that designing the logframe of the project was challenging because of the scale of the COVID-19 crisis, the number of implementing entities involved, and the uncertainty about the level of funding associated with the phased approach. The high number of planned activities and countries would imply a thin distribution of financial resources. What some project teams did to manage the uncertainty of resource availability and ensure delivery in the Surge project was saving resources from regular projects. Informants acknowledged that these managerial and processual arrangements were the best that could have been done during the crisis and the uncertainties associated with it, but also acknowledged that, even with some trimming in the proposed activities, the end result was less focused than would usually be the case. A lesson learned is the need to assess what crises the DA programme is equipped to respond to (e.g., economic, financial, market, health, climate, national, regional, global, etc.) and what tools can be immediately made available to countries in need. For instance, the e-registration platform was mentioned as an existing technology for e-government that was rapidly scaled up to respond to the needs of MSMEs in the COVID-19 crisis.

56. The project governance and management structures and processes were effective in supporting bilateral collaborations between UNCTAD and individual Regional Commissions in delivering project outputs, but there were limited project-level management mechanisms. Nevertheless, the operating mechanisms supported communication and seeded exchanges of information between regions with increased joint deliveries toward the end of the project. From the perspective of the DA-PMT, according to interview data, one of the purposes of bringing all implementing entities together in bi-weekly meetings was to foster dialogue and help collaboration for a quick response to the crisis. It seems that this did not represent an inter-entity coordination function of these meetings, but a sharing of knowledge to be taken up by other coordination structures. A project coordination team, with two representatives of UNCTAD and one of each of the other implementing entities, was set up. The actual coordination was mostly centralized in UNCTAD, as the project lead. The perception of some members of the PCT is diverse in relation to how the PCT worked, with some of them referring to themselves as focal points rather than members of the PCT, and others not providing substantive evidence on how the PCT, as a whole-of-project coordination structure, worked. Evidence from deliverables indicates that collaboration was bilateral, mostly between UNCTAD and individual Regional Commissions, and built on previous successful collaborations. Some informants referred to parallel streams of work without coordination with the work of other Regional Commissions. Others saw the Surge project as an opportunity to learn about what other Regional Commissions were doing and to set the stage for future collaboration. For instance, ESCAP, ESCWA and ECA exchanged information about the DEPAR portal, exchanged materials, and participated in meetings; ECE and ESCAP acknowledged that mutual cooperation can be facilitated by them having member States being in both commissions. Interviewees from all implementing entities referred to attending events organized by other entities. An advanced level of cooperation was achieved by ECLAC and ECA (through its Africa Trade Policy Center (ATPC) and IDEP), on the online course material on competition policies delivered in Portuguese. They engaged in initial conversations and, because the pace of implementing this specific output was different in each entity, this was an asynchronous cooperation, mostly involving the sharing of course materials. Yet, this is reported by ECLAC as a successful South-South cooperation during a turbulent crisis context.

57. UNCTAD, as the lead agency, focused on bilateral cooperation according to regional expertise and planned outputs. UNCTAD cooperated with ECLAC, ESCAP, ESCWA, ECA, and ECE in the delivery of outputs such as “The COVID-19 pandemic impact on micro, small and medium-sized enterprises – Market access challenges and competition policy” (phase 1, output F7). With ECLAC, UNCTAD collaborated in delivering a training course on competition law and policy, an online course on competition and SMEs and cross-border trade, outputs on competition policy, and technical assistance for projects on Asia exports in the LAC region. With ESCAP, UNCTAD collaborated with the creation of the ASEAN SME policymakers’ network, and a discussion paper and meetings proceedings of this network. With ESCWA, UNCTAD collaborated in actions on topics such as e-registration, and the organization of an event on digitalization. This partnership grew stronger and new partnerships emerged in the area of competition. With ECE, UNCTAD collaborated with a report on SMEs, and counted on ECE speakers contributing to its workshops. This is consistent with UNCTAD’s traditional approach of engaging in bilateral cooperation, rather than interregional. A factor associated with

63 Specific, Measurable, Achievable, Relevant, and Time-Bound.

36

this approach is the source of demands for assistance, which normally are country- or regionally based. In addition to collaboration between implementing entities, the Surge project also promoted intra-agency cooperation. This was reported within UNCTAD, involving Empretec, the competition and consumer protection policies branch and the investment branch, within ESCAP, on the Bangladesh-related outputs, including the UNCTAD Voluntary Peer Review of Competition law and policy of Bangladesh, with a focus on SME, and within ECLAC, between the economic affairs and international trade divisions. This independent evaluation identified that, toward the end of the project, in phase 3, there were more joint deliverables involving different Regional Commissions (e.g., the online events on competition policy and MSMEs held in phase 3), which suggests that cooperation increased as the project advanced. A lesson learned from these collaborations is that cooperation depends on a combination of countries’ needs, sources of resources (e.g., donors’ requirements, modalities of funding and what they allow to be funded), continuous presence of staff and time for coordination. This makes planning for cooperation a challenge, especially under a global crisis of the COVID-19 scope. Nevertheless, higher collaboration could have been fostered had the project coordination team worked more effectively and planned joint outputs. Hence, despite Surge’s opportunities for cooperation, actual results of these initial efforts are at an early stage.

To what extent has the project contributed to the expected outcomes as enunciated in the project document?

58. Eighty-six outputs were delivered across the world’s regions, built on the expertise of UNCTAD and the Regional Commissions, and some degree of inter-entity collaboration. These deliverables composed a mosaic of actions that, on a global scale, contributed to achieving the outcomes enunciated in the project document. Implementing entities adapted resources and expertise that they already had and expanded (mostly bilateral) inter-entity partnerships to quickly attend to the most urgent needs of entrepreneurs and policymakers to support MSMEs in the first 2-3 months of the project. These actions comprised making information digitally available64, converting face-to-face training into online, developing digital platforms to support MSMEs and governments in responding to the crisis, and producing and disseminating assessment reports on the impact of COVID-19 on MSMEs. UNCTAD, ESCWA, and ECE delivered the eight outputs of phase 1, with some collaboration with the other entities to collect data for the development of these outputs. For the delivery of the Competition and Consumer Protection assessment report (output F7 of phase 1), under UNCTAD lead, for instance, all regional commissions were involved. Outputs in phase 1 were delivered not knowing how long the crisis would last but considering it could extend to 2021. As successive COVID-19 infection waves hit the world, the end of sanitary measures was not foreseeable. Phases 2 (planned to last until the end of 2020) and 3 (planned to last until the end of 2021) expanded the project to support more countries and MSMEs, and increasingly relied on inter-entity collaboration to ensure a high rate and quality of delivery. The PRODOC of these phases, then, involved seven implementing entities and shifted from general responses (phase 1) to the delivery of a wide range of country- and region-based interventions65. The project’s responses to these needs depended on the locally installed capacities to develop more resilient institutions and supportive policies to MSMEs. Some of these deliverables, mostly those related to online awareness- raising, policy dialogue and workshops, relied on inter-entity partnerships to be successfully delivered. Examples from phases 2 and 3 are the UNCTAD-ESCAP Regional Dialogue held in cooperation with the ASEAN Secretariat, the series of Online regional capacity-building events on the role of competition policies for access to markets in post COVID-19 resurgence of MSMEs (UNCTAD-ECE, UNCTAD-ECLAC, UNCTAD-ESCAP), the UNCTAD-ESCWA-ECA Regional Policy Dialogue: The role of Competition Policy in supporting MSMEs economic recovery in the post COVID-19 crisis, the Sub-regional workshop: Global initiative towards post- Covid-19 resurgence of the MSME sector – Women in informal and small-scale cross-border trade (UNCTAD- ECA), the online course Oportunidades y retos para Mipymes de América Latina en el e-commerce transfronterizo66 (ECLAC-UNCTAD), and the Global Policy Dialogue: Post COVID-19 Resurgence of MSMEs and Competition Policy (all seven implementing entities), The Surge project, therefore, effectively delivered a wide range of interventions (i.e., advisory services, trainings, workshops, seminars, digital portal, events, and

64 Smart search engine: Manara Portal (unescwa.org); MSME knowledge portal: https://msme-resurgence.unctad.org/ 65 From a project management perspective, phase 2 was to develop intervention tools and phase 3 was to implement them at country level. The analysis of the deliverables, however, shows a mixed picture, determined by the local capacities to move from the development of a tool to its implementation. Many outputs in both phases 2 and 3 combine, for instance, course development and delivery (e.g., OP 1.5 and 1.6, OP 3.4 and 3.5, and OP 5.10 in phase 2, and OP 1.14, and 5.3 in phase 3). 66 Translation: Opportunities and challenges for MSMEs in Latin America in the cross-border trade.

37

direct technical assistance through policy advice, assessment reports, research studies, and tailored guidelines), directly assisting 96 countries in specific ways. A good practice that comes from this approach is that a global response can be made of multiple local and regional actions that are the most relevant and impactful for its recipients during a crisis. Figure 6 shows the distribution of project-planned activities per phase and implementing entity. The number of outputs planned increased from 7 in phase 1, to 38 in phase 2 and 60 in phase 3 (+57.9% from phase 2), totalling 105 outputs, even under a budget cut of USD 310,000. UNCTAD was responsible for 50% of the outputs in phase 2 and 43% in phase 3.

Figure 7: Surge Project outputs delivered per implementing entity and implementation phase67.

Source: Project’s final report, 2022.

59. It is noteworthy that the number of outputs slightly changed between the PRODOC and the project final report. Additionally, this independent evaluation received evidence of deliverables completed after the conclusion of the final report, in 2022 (e.g., five outputs delivered by UNECA in phase III), and others that do not clearly fit in any of the project planned outputs (e.g., ESCAP’s papers on MSME Financing Series, delivered in 2022). Table 7 summarizes these changes and updates and presents the percentage of outputs delivered.

Table 7: Planned and delivered outputs per project phase. Phases PRODOC

outputs Final report

outputs Delivered outputs*

% delivered Not delivered**

1 7 8 8 100 2 38 39 36 92.3 4.3, 5.9, 5.13

3 60 54 42 77.8 1.25, 2.2, 2.3, 3.2, 3.3, 3.4, 4.1, 4.3, 4.5, 4.7, 5.5, 5.6

Total 105 101 86 85.2 * Based on evidence collected in this independent evaluation. **These outputs fall into the following situations: not delivered due to budget cuts, in progress (i.e., reports in the inception phase by the time of this evaluation), or missing information.

60. Overall, 85% of all project outputs were delivered. Phase 3 has the highest number of outputs and the lowest percentage of effectiveness, around 78%. It was impacted by the budget cuts, the longer time needed to conclude some outputs and the lack of information for this independent evaluation. In cases of budget cuts, this evaluation counted as delivered those outputs with at least 50% of the activities concluded. It is noteworthy that the delivery of some of these outputs was concluded after the end of the project68, given factors such as the pandemic barriers to implementation, coordination difficulties, personnel turnover, and

67 Figure 7 presents the number of outputs fully delivered as stated in the project’s final report. In addition, 6 outputs were still in progress when the final report was submitted (ECLAC 1 output, UNCTAD 1 output, UNECA 4 outputs in progress). 68 Interviewees used the project’s final report (2022) as reference for the end of the project, since some of their deliverables, pending then, were concluded afterwards.

Phase 1

Phase 2

Phase 3

0

5

10

15

20

UNCTAD DESA UNECA ESCAP UNECE ESCWA ECLAC

3

0 0 0 2 2

0

19

2 4 4 4

1 2

19

1 3 2

9 7

4

Number of project outputs delivered per phase

38

limited capacities of member states. Considering all these barriers, this independent evaluation considers the level of effectiveness high, since these rates of delivery contributed to the achievement of the main project objective through significant achievements in each project outcome, as described next.

61. National capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship and MSME promotion in post-COVID-19 resurgence (outcome 1.A) improved through e-learning courses, awareness-raising activities, technical assistance in designing and implementing national entrepreneurship policies, the creation of regional networks of policymakers, and knowledge production and hubs on MSME’s policies. Deliverables to support national capacities started in phase 1, with the launch of the DEPAR portal by ESCWA. This portal provides a wide range of tools and services to MSMEs, such as the MSME toolbox69, with business guides, inspiration stories, start-up tools, external tools and news from partners on innovation, technology, policy-making, etc. It allows visitors to become part of a community of practice, in which participants can share content, build their own community, create events, develop courses, and announce opportunities. One of the ideas for the portal was to provide a one-stop shop for materials from other Regional Commissions too, but, later, it became clear that this would lengthen the path70 for MSMEs and policymakers to access materials from their respective regions and this was not implemented. Figure 8 indicates a growing number of users to the portal, with about 2000 users four months after launch, and about 3000 users one year after launch. Most of these users are from Jordan (59%, according to data from 20 June 2023). There was no qualitative information to explain the quantitative trend.

Figure 8: DEPAR number of users.

Source: ESCWA.

62. Other deliverables under outcome 1.A included e-learning courses based on the EPF; a series of webinars to policymakers on entrepreneurship policies for MSME resurgence; technical assistance in designing and implementing national entrepreneurship policies; the creation of the Asia-Pacific MSME policymakers’ network and follow-up workshops; the creation of a knowledge hub on policy measures, tools and practices on MSMEs’ resurgence; regional workshops to promote dialogue on women’s role in small-scale and informal cross-border trade and tailored recovery; development of the ‘Compendium of Renewable Energy Technologies’ with an online toolbox and training materials on green recovery of MSMEs; policy roundtables with communities and SMEs; and design and implementation of entrepreneurship policies and strategies based on regional assessment. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 74% across project dimensions (Figure 9). It is noteworthy that, given the small response rate, survey results need to be interpreted with caution, keeping in mind that these findings reflect the perceptions of those who responded to the survey questions, and are not representative of all participants (See the methodology section for details).

Figure 9: Effectiveness results of outcome 1.A (n=51 respondents).

69 https://depar.unescwa.org/msmetoolbox. 70 This would increase the need to liaise with ESCWA to upload information according to regional needs (i.e., more time to have information available online), and could impose a barrier to MSMEs and policymakers from other regions to access their regional information (i.e., by adding ESCWA as an intermediary institution between local stakeholders and their respective RECs).

39

Source: Independent evaluation survey, 2023.

63. In the second half of 2021, the 5-week UNCTAD/UNITAR EPF e-course launched with the objective of enhancing the capacity of policy makers to provide policy response and support measures on MSMEs for post-COVID-19 resurgence and develop longer-term strategies for a resilient and competitive MSME sector. One of the course contents explicitly addressed how the environment for and role of entrepreneurship evolved during and in response to the COVID-19 pandemic. It was developed in English, French and Spanish, and covers all five pillars of the EPF. It was preceded by the launching of webinars with the participation of UNCTAD officers, UNITAR representatives, policymakers, and thematic experts. The UNCTAD / UNITAR 2021 Summary Report of this EPF online course presents data for each of its three versions. Results indicate a gender balance between the number of male (n=221) and female (n=223) registrants to the course in the English version, a higher number of female registrants (female=65, male=48) in the Spanish version, and a higher number of male registrants (male=78, female=52) in the French version. Between 5-7% of registrants indicated another gender in the three course versions. The majority of survey respondents, in the three versions, indicated that the course presented new information (>75%) and was relevant to their jobs/occupations (100%). The likelihood of survey respondents applying the information acquired from the course is above 96%, and all respondents would recommend the course to a friend or colleague. This e-course supported UNCTAD’s assistance to Uganda, Seychelles and South Africa in designing and implementing national entrepreneurship policies (NES) based on the EPF. The NES of South Africa focuses on Youth Entrepreneurship, the NES of Uganda on migrant and refugee entrepreneurs, and the NES of Seychelles on the marine biotechnology sector with reference to youth entrepreneurs. Although there was an impact assessment of the NES implemented before the COVID-19 pandemic in The Gambia, this was not delivered due to the budget cuts in phase 3.

64. A set of deliverables by ESCAP included the launch of the terms of reference of the Asia-Pacific MSME Policymakers’ Network (February 2021), followed by a concept note. Three online regional dialogues of the Network were organized by ESCAP in partnership with UNCTAD and ASEAN secretariats. They covered the digital economy (including how to support women entrepreneurs through ICT, business skills training and innovative financing), the role of competition policy in strengthening the business environment, both in March 2021, and the launch of the Policy Guidebook for MSME Development in Asia and the Pacific71, in May 2022. Documents produced to support the Network include a paper on the digital economy72, UNCTAD’s broad survey on the impacts of the COVID-19 on the business environment, particularly in relation to competition issues (2020)73, and the regional report ‘The Role of Competition Policy in Strengthening the Business

71 This book describes the MSME contribution to achieving the SDGs, with comprehensive and practical information to policymakers on MSME development. 72 Callo-Müller, M. V. (2020). Micro, small and medium enterprises (MSMEs) and the digital economy. 73 UNCTAD (2020). Covid-19: Firmer Action Needed to Better Protect Consumers. [online] https://unctad.org/news/covid-19-firmer-action-needed-better-protect-consumers.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

provided access to information and best practices on government support measures for post COVID-19

recovery

improved my knowledge on entrepreneurship/MSMEs policies design and implementation in the context of

post COVID-19 recovery

contributed to identifying new/improved policy measures for MSME promotion in post COVID-19

resurgence

The project...

Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

40

Environment for MSMEs in the ASEAN Region’ (2021)74. Following up on the second meeting, there was a report with recommendations to governments, competition authorities, development agencies, and multiple stakeholders on competition policy measures in a pandemic recovery context. Assessment data is available for the two first meetings and shows that more than 75% of the respondents indicated enhanced knowledge and skills, allowing them to effectively receive from and share information with others in the region regarding MSME policies (>67%), and providing opportunities to enhance regional cooperation in competition policy and law (>69%). Comments from participants referred to the advantages of sharing experiences with other countries, learning how they try to overcome MSMEs’ problems during COVID-19, and providing ideas on how to promote digitalization to MSMEs via education. Interview data indicates that the Network became inactive afterwards, since ESCAP dropped the MSME agenda after the project to focus on other demands.

65. In September 2021, the ‘UN-ESCWA Entrepreneurship Community Workshops and Policy Roundtables – How to improve existing government and non-government programs’ took place, with a set of interactive discussions on existing government programs, existing non-government programs, needed government programs, and needed non-government programs, with follow-up recommendations on these areas. Specifically on the health economy, a set of online conferences on ‘Entrepreneurship in the new health economy’ were delivered in October 2021. One targeted the Asia-Pacific region75, with 45 attendees, another targeted Latin America76, with 25 attendees, and another, Africa77, with 33 attendees. They were organized in partnership with private sector organizations in the health sector for policymakers. It is noteworthy that, except in the African event, these conferences did not include policymakers among their panellists. These conferences generated inputs to the publication ‘Entrepreneurship & innovation in the new health economy’78, launched by UNCTAD in 2022. This document highlights the role of SMEs and start-ups in the new health economy in developing countries, with main findings on the role of regulation in the e-health sector, and the role of public institutions in building bridges between different stakeholders.

66. The resilience and competitiveness of MSMEs in post-COVID-19 resurgence (outcome 1.B) improved through a large set of Empretec-related activities (including trainings tailored to low literate entrepreneurs), cross-border trade workshops for female traders, adaption of the Farming as a Business program, and country-tailored guidelines and best practices on circular economy and sustainable resource management. Outcome 1.B is the one with the highest number of deliverables (n=28), each encompassing sets of activities and supporting MSMEs in multiple countries. They can be grouped in four packages: Empretec (responsible for most of these deliverables and demonstrating the effectiveness of UNCTAD’s pre-existing capacity to deliver trainings, including onsite, during the COVID-19 pandemic), informal and small-scale cross-border trade for women traders, UNCTAD’s Farming as Business program, and Guidelines and best practices on circular economy and sustainable resource management79. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 68% across project dimensions (Figure 10).

Figure 10: Effectiveness results of outcome 1.B (n=39 respondents).

74 ESCAP (2021). The Role of Competition Policy in Strengthening the Business Environment for MSMEs in the ASEAN Region. 11 February 2021. NOTE: there is no authorship in the document. This reference may need revising. 75 Scaling innovation in the new health economy – lessons learned from the Asia-Pacific region. 76 The rise of medtech in Latin America. 77 Unlocking digital health innovation in Africa. 78 UNCTAD (2022). Entrepreneurship & innovation in the new health economy. Geneva: UNCTAD. 79 Given the higher emphasis on government advice, more details on these guidelines are in the highlight box on the contribution of the Surge project to the country’s government responses to the COVID-19 crisis regarding the resurgence of MSMEs.

41

Source: Independent evaluation survey, 2023.

67. Effectiveness evidence of the Empretec package of deliverables indicates that development of the updated online training course (E6DM) was a strategy to facilitate post-COVID-19 entrepreneurship recovery. Ten Empretec webinars, jointly organized and delivered by UNCTAD and Empretec Centres between April and December 2020, were attended by Jordan, Malaysia, Ghana, Nigeria, Russia, South Africa, The Gambia, Argentina, Benin and Zimbabwe. In tandem, 5-day E6DM TOTs were delivered in seven batches with participants from 15 countries80, between June and October 2020. In total, 65 men and 32 women participated in these trainings. Training assessments show positive results, with more than 85% of the respondents rating the course content as good or very good, and as a useful opportunity to progress as an Empretec trainer or trainee trainer. The 21-day TOT, delivered in English, French and Spanish in November 2020, had 35 participants from African countries81 and Romania, and 35 participants from Latin America82. The content was assessed as good or very good by more than 83% of the respondents (16 in Africa and Romania, and 25 in Latin America), and useful for their progression as Empretec facilitators (>91%). Training survey comments highlighted, for instance, the online approach, the follow-up and participant-centered approach, and the possibility of interacting and sharing opinions with colleagues as the most interesting aspects of the training. In sequence, ten 6-day Entrepreneurship Training Workshops (E6DM) were delivered in seven countries83 between September 2021 and July 2022. The total number of participants across countries was around 30084, with about 50% of women (estimated on the data available) and some countries reporting the participation of youth entrepreneurs up to 54% (i.e., Ghana). When post-training assessment is available, results are positive with more than 88% rating the overall workshop as effective/useful/good or very good/excellent85. More than 84% rated the workshop content as good-excellent/relevant. Good-very good improvements in entrepreneurial skills ranged between 43% and 100%, good-very good relevance of the training to run the business was above 80%, and estimated good-very good impact on the future of their businesses was between 33% and 100% in a set of 10 possible impacts. Subsequently, UNCTAD delivered an additional online 1-day E6DM TOTs in September and October 2022, as a pioneer effort to homogenize the application of new versions of the Empretec method to different Empretec Centres and trainers. Participants totalled 109 (64 male, 45 female) from 19 countries86.

80 Argentina, Benin, Cameroon, Ethiopia, Ghana, Jordan, Kenya, Peru, Romania, Russia, Tanzania, The Gambia, South Africa, Venezuela, and Zambia. 81 Benin, Botswana, Ghana, Mauritius, Romania, South Africa and Tanzania. 82 Argentina, Chile, Colombia, Ecuador, Panama, Peru and Venezuela. 83 Benin, Nigeria, Cameroon, Zimbabwe, The Gambia, Ghana, and Malaysia. 84 Note: Some country reports do not inform the total number of participants. 85 Measurement scales could be different between countries. 86 Angola, Argentina, Botswana, Colombia, Ghana, Ecuador, El Salvador, Jordan, Kenya, Malaysia, Mauritius, Mozambique, Namíbia, Panama, Romania, Saudi Arabia, South Africa, Tanzania, and Uganda.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

provided information useful for starting a new business or expanding a business venture

improved my knowledge on entrepreneurship and business development

contributed to identifying new/improved measures to increase business sales

The project...

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

42

68. Other innovations to the Empretec programme implemented in 2021 were the development of a version for low-literate people and a full online training. Between April and May 2021, UNCTAD delivered three sessions of the TOTs for low-literate people, in English, French and Spanish, for 62 male and 58 female participants. Trained trainers, then, delivered three 8-day workshops for low-literate people, between March and July 2022, in Benin, Malaysia and Uruguay. The total number of participants was 72, 57% female. The assessment was positive87 in relation to the workshop in general, its content, and its relevance for the participants’ businesses. Regarding the development of the online Empretec training course using gamification and other IT-based tools88, there is evidence of three pilot workshops with entrepreneurs, delivered in hybrid format and in partnership with SEBRAE (Empretec host institution in Brazil) between December 2021 and August 2022. Each pilot tested the effectiveness of the digital tool and led to the identification of, and solutions to, technical issues. Interviews for this independent evaluation conducted with Empretec centers and UNCTAD staff indicated that the performance of Empretecos in the online course was below face-to-face training. An informant said that “workshops are better when people share in the classroom, with safe environment, with facilitator. Online does not facilitate that.” However, even if the online format is not suitable for a workshop that requires six full days of commitment, some procedures, e.g., tracking participants’ progress, and resources, such as web platform for Empretec centers, can be online. Another challenge of delivering the Empretec training online was conciliating the objective of reaching out to vulnerable beneficiaries with no access to the necessary technology. The solution, then, was to keep Empretec workshops offline. Only the TOTs remained online, to ensure there would be trainers when the presential workshops could be resumed.

69. The Surge project delivered a series of webinars on the role of Empretec on enhancing entrepreneurship promotion in post-COVID-19 resurgence. At the global level, it supported the virtual 7th Empretec Global Summit, ‘Boosting Entrepreneurship: The Contribution of Empretec into the Post-Covid-19 Resurgence of the MSME Sector’, in April 2021. The Summit was attended by over 570 participants from 64 countries and was interpreted in seven languages89. “It was also followed by 783 viewers on UNCTAD Facebook Live and almost 2,000 views were recorded on UN WebTV from 94 countries.” Another global event was the Global Entrepreneurship Congress, in Riyadh, in March 2022. UNCTAD contributed to the Ministerial Roundtable on Entrepreneurship. Regional webinars were held face-to-face in Colombia, in November 2021, with Empretec center’s representatives from Colombia, Argentina and Ecuador and entrepreneurs; and with the francophone Empretec centres, with 68 participants from Benin, Cameroon and Mauritius, in May 2022. Country webinars included an online event with NAWEN in Malaysia, in December 2021.

70. To expand knowledge on relevant topics of the modernized version of Empretec, UNCTAD delivered a series of ad-hoc online courses between August and October 2021. They covered business model generation (64 attendants from nine countries90), design thinking (83 participants from 11 countries91), entrepreneurship and marketing (Seychelles, no attendance data available), capacity-building for women in cross border trade (59 women participants in Kenya and Tanzania), and a 4-day TOT on sustainable businesses and SDGs (62 participants from 10 countries92). Post-training survey data (22 respondents) is available for the last course, indicating 90% of participants were satisfied with the training content and 86% agreed that sustainability as per the 2030 Agenda should be included in their Empretec training activities.

71. An impact survey conducted by UNCTAD in March 2023 collected data from a representative sample of Empretecos in Benin, Cameroon, Ghana, Nigeria, The Gambia and Zimbabwe. The Empretec programme was positively assessed by respondents (8.75 points in 10) and was considered better than the average of other business training tools by 54%. Most respondents reported sales growth after the ETW (67%), and 94% believed sales will increase in 2023-24. Among non-entrepreneurs, 60% reported an increase in their income levels after the workshop, 88% felt more motivated to venture after the ETW, 67% foresaw opening a business in the coming three months, and 91% of the Empretecos believed their employability level had increased after the ETW, a trend even stronger among women. Interviews with Empretec centers for this independent

87 100% of good-excellent ratings in Benin, and an average range between 4.5 and 4.7 in a scale of 5 in Uruguay. The training report for Malaysia did not present specific assessment data. 88 LMBInterativa (2022). Empretec Online. Delivery report. 89 Arabic, Chinese, English, French, Portuguese, Russian and Spanish. 90 Ecuador, Ethiopia, Ghana, Jordan, Kenya, South Africa, Tanzania, Zambia and Zimbabwe. 91 Ecuador, Ethiopia, Jordan, Kenya, Malaysia, Romenia, Saudi Arabia, South Africa, Tanzania, Zambia and Zimbabwe. 92 Angola, Argentina, Brazil, Colombia, Dominican Republic, Ecuador, Guatemala, Peru, Uruguay, and Venezuela.

43

evaluation indicated that, in Zimbabwe, for instance, women and youth facing business difficulties were the majority of those trained. Two of these participants are the founders of the Women in Agriculture Union. They learned how to keep records of funds received from members, and the advantages of doing so. Young entrepreneurs in agribusiness improved product processing knowledge, leading to expansion and new markets, in Zimbabwe. In Argentina, course participants prefer the online course format, even when their access to the internet is hard, because of the flexibility of time and schedule and the possibility to connect with people across the country. The number of participants is much higher in the online version in Argentina. In Jordan, the online TOT targeted vulnerabilities, such as internet connection and digital literacy. The online delivery was a challenge also from the Empretec centres’ perspective. More than three hours of online training was not possible; and the programme is based on learning by doing, i.e., it requires observation of behaviours. The new online tools helped to mitigate this challenge, but they did not always work because of poor IT literacy, people’s commitment, and internet connection.

72. The outputs related to the UNCTAD component on women and cross-border trade, were delivered in phases 2 and 3 of the Surge project. Evidence indicates 6-day presential workshops93 delivered in Zambia (36 participants), Malawi (32 participants), Kenya (64 participants) and Tanzania (64 participants), between February and November 2021. They tackled two main critical issues: lack of knowledge about trade rules and customs procedures, and lack of entrepreneurial skills. Workshop beneficiaries were mainly small- scale/informal cross-border traders, especially women, representatives of cross-border trade associations, representatives from the Revenue Authorities, and other authorities based at the borders. Workshop assessments were positive, with more than 45% of the participants rating that their knowledge about trade rules and customs procedures had improved very much or extremely. Most of them acquired confidence in crossing the border through normal channels (>61%) and will be able to formalize within the next 12 months (>54%). Assessment data for Kenya and Tanzania indicated that more than 80% of the participants rated the programme as excellent, the training content as very relevant for running their businesses, and more than 90% indicated the training will help them improve professional behaviour, management skills, business planning skills, establish a network with other business owners, learn about business opportunities and help learn about funding possibilities. The workshop’s methodology, content and results are compiled in the activity report94 commissioned by UNCTAD. Country-specific guides on ‘Women in informal cross-border trade: A small- scale trader’s guide to trade rules and procedures’ developed by UNCTAD provided tailored information for female traders in Kenya, Botswana, Tanzania, Malawi and Zambia. These guides, except for Malawi and Zambia, contain traders’ obligations regarding COVID-19 measures related to crossing the border and a checklist that includes COVID-19 certificates.

73. A 2022 impact assessment on project activities carried out by UNCTAD on women and cross-border trade95 indicated that 100% of the participants became very or extremely familiar with their rights as cross- border traders, 95% became very or extremely familiar with the obligations of cross-border traders, and 43% acquired considerable knowledge on the use of COMESA and EAC STRs and the SADC Trade Protocols. The enhanced knowledge about the use of official border points was very or extremely helpful for 92% of the respondents. “Some traders reported positive results in terms of increased profits, a larger clientele, and the ability to offer a more diversified range of products” associated with the enhanced use of official border points. Almost all participants (96%) reported that they had adapted their business model. Also, 80% of the respondents considered registering their businesses. Responses were positive on acquiring extremely good or very good understanding of the COVID-related rules (65%). Qualitative data indicated that the training helped trainees to be proactive in protective measures, such as in encouraging customers to wear masks and wash their hands. Regarding new strategies, “some traders turned to digital tools to continue their businesses, including Facebook, Instagram, WhatsApp groups, and online platforms.” Interviews with trainers conducted by this independent evaluation indicated that the results of the training for low literacy beneficiaries in cross- border trade were ‘amazing’, based on a follow-up four months after the training. Proper understanding of the trade rules and procedures, rights and obligations of the traders coupled with upgrading of their entrepreneurship skills helped traders to overcome the adverse trade environment caused by the Covid-19

93 They built on training activities completed by UNCTAD in 2019 in the framework of the project ‘Informal cross-border trade for the empowerment of women, economic development and regional integration in Eastern and Southern Africa’. 94 Tigere, F. (December 2021). Phase III Final Report – Global initiative towards post-COVID-19 micro, small and medium enterprises (MSME) sector: Kenya and the Republic of Tanzania. 95 UNCTAD (2022). Cross-border Trade in the Pre- and Post-Pandemic Environment Evidence from Malawi, the United Republic of Tanzania and Zambia. Geneva.

44

pandemic. Some were able to bounce back and build back their businesses. More information on female traders in cross-border trade is in the specific section on gender and LNOB.

74. The delivery of the Farming as Business (FaaB) programme took place in phases 2 and 3 of the Surge project. Adaptation of this programme referred to the online delivery of the TOT, between February and June 2021. Training material does not refer to the pandemic crisis96. The English version of the TOT trained 25 male and 28 female small farmers from 13 countries97. The French version trained 25 male and 28 female participants from Angola, Benin and Cameroon. The Spanish version trained 9 male and 7 female participants from eight countries98. Post-training assessments, available for the English (15 respondents) and French (4 respondents) versions, were positive. On average, 79%, rated the course content as good or very good, 84% found it a useful learning opportunity, and 95% considered the course useful for them to progress as Empretec trainer or trainee trainer. Following the TOT, a grant agreement between Empretec Ghana Foundation and UNCTAD allowed for training of 65 vegetable farmers in Ghana, in November 2021. A report on this training99 indicated that 90% of the participants were male and 10% female. There is no reference to resilience in post-COVID-19 recovery.

75. The Surge project implemented tools that facilitated MSMEs’ registration and formalization (outcome 2) through an e-registration platform, policy toolkits for governments on formalization, and virtual workshops for knowledge dissemination. Under this outcome, UNCTAD delivered the e-registration platform, implemented it initially in El Salvador (cuentamype100), and expanded implementation to Benin, Cameroon and Mali. DESA developed policy toolkits for the governments of Kenya and The Gambia in streamlining MSME formalization and inclusive delivery of formalization services. The dissemination of these toolkits was with virtual workshops in the two countries. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, but, because the number of respondents was between 3 and 4, no further conclusions can be drawn from our survey.

76. Interview data indicated that the pilot implementation of Cuentamype in El Salvador involved private and public institutions, and the signing of an inter-institutional agreement, considered one of the biggest project achievements in the country. It brought together, for instance, social security institutions, the Ministry of Finance, and municipalities, and attracted the interest of financial institutions in relation to providing credit to MSMEs. The preparation of the platform included the organization of sectoral working tables with these institutions, and a pilot, in 2020, with a group of 20-25 entrepreneurs for feedback, for around 3-5 months. After adjustments, the platform launched in a forum where entrepreneurs could learn about the tool. The project trained government technicians as trainers for the end-users of the platform – 3-4 training days of about 15 nationwide professionals. During technical visits to MSMEs (e.g., women’s owned beauty salons), specific needs of beneficiaries were identified, such as basic knowledge in accounting and individual monitoring, mostly for women. The answer to these needs included the development of a printed simplified booklet on accounting (i.e., a paper version of cuentamype), follow-up visits for six months, and the creation of a WhatsApp group for direct contact with the registry office (CONAMYPE) to get direct support in interpreting financial analysis outputs and suggest improvements to the platform. Out of about 400 trained entrepreneurs101, CONAMYPE estimates that at least half of them use the tool regularly (about 60% of them are women and covering entrepreneurs from 18 to 70+ years old). The e-registration platform was, then, expanded to Benin, Cameroon (guichet electronique) and Mali (guichet unique) in 2021. The online platform increased MSMEs’ registration 91% in Benin (84% women, 181% youth), and 40% in Mali (49% women, 110% youth). Replicating the case of El Salvador, these e-registration platforms are incorporating additional services, such as modifications in business registry and grant applications. Real-time data generated by these platforms have been used to design better entrepreneurship policies.

96 The trainer’s guides (i.e., coffee and dairy farming), in English and French, were produced before the pandemic and their content was not updated to the new situation. The guide on dairy farming in Spanish was issued in 2021 and reproduces the content of previous guides, also not referring to the COVID-19 pandemic situation. 97 Angola, Benin, Botswana, Ghana, Kenya, Malaysia, Myanmar, Panama, Saudi Arabia, South Africa, Tanzania, Zambia and Zimbabwe. 98 Argentina, Chile, Colombia, Ecuador, El Salvador, Peru, Uruguay and Venezuela. 99 Empretec Ghana (Feb.2021). Report on training of farmers on the UNCTAD Farming as a Business (FaaB) programme - Organized for vegetable growers in the Ayawaso West municipaliy – Greater Accra region. Ghana. 100 https://cuentamype.org/ 101 This included entrepreneurs who could not read or write and were not tech savvy. In these cases, entrepreneurs indicated collaborators to take part in the trainings.

45

77. Policy toolkits issued by DESA considered regional and country-based approaches. A regional toolkit was ‘Best Practices: Formalization of Micro-, Small and Medium-Sized Enterprises (MSMEs) in Africa’102, a document that covers a wide range of best practices associated with formalization (e.g., institutional frameworks, regulatory environment, supply and value chains, access to finance, human capital development, and social dialogue). It assembles data from African countries’ best practices, such as Kenya, Nigeria, Rwanda, South Africa, Tanzania, and Zimbabwe, and benchmarks on examples from countries in other regions, such as India, Sri Lanka, China, Argentina and Colombia. Country-based policy toolkits focused on Kenya and The Gambia and included policy documents and virtual or hybrid workshops for knowledge dissemination. In Kenya, the focus was on leveraging policies governing cooperatives of MSMEs, with the issuing of a report103, a toolkit with policy recommendations for the formalization and growth of MSMEs through a cooperative model104, policy guidelines for the formalization of MSMEs in Kenya, modelled on the guidelines for the formalization of cooperatives105, and the toolkit promotion in a virtual workshop in March 2021. A workshop assessment (responded by 60 participants, 41 male, 19 female from African, Asian and Europe countries106) indicated that 98% of the participants were satisfied or very satisfied with the event, and 100% considered the event relevant to their work. Later, in 2022, DESA supported an assessment of the impact of COVID-19 on MSMEs led by women and youth in Kenya 107 and stakeholder consultations, based on these assessments, in a hybrid workshop. For instance, 100% of the 22 respondents (14 male, 8 female) of the assessment of the workshop on the Assessment of the Covid-19 Impact on Women and Youth-led MSMEs in Nairobi, were satisfied or very satisfied with the workshop, and considered it relevant to their work.

78. MSMEs’ access to finance (outcome 3) improved through online training to policymakers and MSMEs (e.g., financial literacy, accounting, SDG reporting), publications, case studies, training of trainers, and national and regional workshops for knowledge dissemination. Outputs under outcome 3 were carried out by UNCTAD and ESCAP. They produced online training material for policymakers on MSMEs’ access to finance, developed training courses for MSMEs for financial literacy and accounting and reporting, including on the SDGs – supported by a guidance –, TOT’s courses on accounting and reporting for MSMEs, published a book on MSME finance, developed case studies on accounting and reporting for MSMEs, and delivered national and regional workshops based on the online materials produced. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 70% across project dimensions, keeping in mind that the number of respondents is low for robust conclusions and this result needs to be interpreted in qualitative terms (Figure 11).

Figure 11: Effectiveness results of outcome 3 (n=10 respondents).

102 DESA (2021). Best Practices: Formalization of Micro-, Small and Medium-Sized Entreprises (MSMEs) in Africa. 103 DESA (2021). Leveraging policies governing cooperatives to encourage the formalization of micro-, small and mediu- sized enterprises (MSMEs) in Kenya. 104 DESA (2021). Promoting MSME Formalization through the Cooperative Enterprise Model. 105 DESA (2021?). Policy guidelines for the formalization of micro-, small and medium enterprises in Kenya. 106 Kenya, South Africa, Burundi, Tanzania, The Gambia, Lesotho, Morocco, Philippines, Indonesia, Germany. 107 MSEA (2022). Assessment of the impact of COVID-19 among micro, small and medium-sized enterprises in Kenya and their resilience mechanism. Nairobi: Micro and Small Enterprises Authority (MSEA).

46

Source: Independent evaluation survey, 2023.

79. ESCAP led the delivery of a book that set out a post-crisis policy agenda for MSME finance, in 2021108. It addresses MSMEs’ finance gaps and the financial landscape for MSMEs before, during and after the pandemic crisis. The book draws policy recommendations and practical suggestions in MSME finance in relation to, for instance, business services ecosystem, regulatory sandboxes, supply chain and trade finance, financing women-led MSMEs, multi-stakeholder collaboration for mutual benefits, and regulatory framework for fintech. Two thematic studies followed-up, in 2022. One was on the role of banks in Asia in lending to MSMEs109, and the other on the role of digital payments110, including a topic on women-led MSMEs and SDG 5.

80. UNCTAD developed the guidance on core indicators (GCI) ‘Implementation of core SDG indicators for sustainability reporting by companies’ in English, Spanish, French, Arabic, Chinese and Russian, and corresponding training material in partnership with UNITAR. This guidance supported GCI workshops on accounting and reporting for MSMEs in Kenya (23 participants), China (150 participants), Latin America (55 participants, 14 countries), Francophone countries (122 participants, 11 countries) and the MENA (79 participants, 4 countries) and Eurasian (200 participants, 10 countries) regions, between February and November 2021. Across workshop sessions, assessments responded by 246 participants (78 male, 47 female111) indicated that more than 90% of them rated the overall course content as good-excellent/satisfied-very satisfied.

81. MSME’s access to technology and innovation (outcome 4) increased through a combination of survey-based reports assessing the impacts of Covid-19 and the Ukraine crisis on MSMEs at country and regional levels, regional seminars for knowledge dissemination and sensitization of policymakers on needs of MSMEs, online courses on the role of technology and innovation for entrepreneurs and policymakers, including on harnessing green technologies, and knowledge products on a range of areas including building linkages between MSMEs and multinational companies. Activities under outcome 4 were led by ECA, through its Subregional office for Southern Africa, and included surveys circulated to more than 3,000 MSMEs across its 11 member states in Southern Africa, to assess the impacts of the Covid-19 and Ukraine crises on MSMEs112, the development of an online training course for entrepreneurs and policymakers on the role of technology and innovation in addressing this impact (December 2021), an online training course on harnessing green

108 ESCAP (2021). Rethinking MSME Finance in Asia and the Pacific: A Post-Crisis Policy Agenda. Bangkok. 109 ESCAP (2022). Small and Medium-Sized Enterprises Lending Approaches: The Role of Banks in Asia. MSME Financing Series No. 6. Bangkok. 110 ESCAP (2022). MSME Access to Finance: The Role of Digital Payments, MSME Financing Series No.7. Bangkok: United Nations. 111 No gender disaggregated information for the Eurasian workshop. 112 The result was 11 country level reports which, at the time of the evaluation, were being finalized and not yet published.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

provided useful information on financial inclusion and MSMEs access to finance

improved my knowledge on accounting and reporting, including on the SDG reporting, and

improved my capacity to manage financial resources

contributed to identifying new/improved measures for financial inclusion and MSMEs access to finance

The project...

Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

47

technologies to enhance MSME competitiveness in Southern Africa (August 2022), two side events at the margins on the SADC Industrialisation Week on strengthening linkages between African MSMEs and Southern led MNCs and on the role of technology incubators in MSME development in collaboration with the SADC Business Council Southern Africa113. An additional unplanned output, at the request of the Government of Mauritius, was a regional seminar on “The impact of Covid-19 on MSMEs in Southern Africa: Country Experiences and a Focus on Building Back Better in Mauritius” was held at the margins of Dubai Expo 2020, in collaboration with the SADC Business Council and the Ministry of Industrial Development, SMEs and Cooperatives of Mauritius. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 83% across project dimensions, keeping in mind that the number of respondents is low for robust conclusions and this result needs be interpreted in qualitative terms (Figure 12).

Figure 12: Effectiveness results of outcome 4 (n=12 respondents).

Source: Independent evaluation survey, 2023.

82. Evidence from the technical report114 of the 4-week online training on the role of technology and innovation for African businesses in addressing the impact of COVID-19 indicated that 41 participants attended the course. Out of those who responded to the post-training assessment, 95% noted that the course modules were relevant to their professional needs and 90% felt satisfied with the relevance of the topic and the modules. Regarding the course on green technologies for African SMEs, the assessment report indicates that 30% of the 155 registrants completed the course successfully by passing the quizzes. Out of these, 45 responded to the post-course assessment. Results indicated that 93% of respondents rated the quality of the course as good or very good, 93% indicated that the content of the course is relevant to their work, and 100% indicated that they will use the knowledge acquired in this course.

83. MSME’s access to markets (outcome 5) increased through activities that combined the production of national and regional assessments, studies and reports with online events for knowledge dissemination of findings, and online trainings for policymakers and MSMEs at the regional level on topics such as competition policies and access to markets, agricultural quality and food loss, and the promotion of export capacity. Outcome 5 is the second in number of outputs planned (n=25) and the first in collaborative deliveries between UNCTAD and the Regional Commissions. Deliverables cover a wide range of activities on competition policies to regulate access to markets during the pandemic crisis and promote the resurgence of MSMEs in post- pandemic recovery. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 66% across project dimensions, keeping in

113 Updated information, received after the finalization of this independent evaluation, indicated that, as of October 2023, the two publications on strengthening linkages between African SMEs and Southern led MNCs and the role of technology incubators for MSMEs development in Southern Africa are completed and being professionally edited. Two other publications, finalized in 2023, are “Opportunities for MSMEs within the Blue and Green Economy: the case of SADC” and “The role of digitalization in MSMEs development in the context of the AfCFTA”. 114 IDEP, ECA (2022). The role of technology and innovation for African businesses in addressing the impact of COVID-19 – Technical report.

0% 20% 40% 60% 80% 100% 120%

provided useful information on MSMEs access to innovation and technology

improved my knowledge on how to increase MSMEs access to innovation and technology

contributed to identifying new/improved measures to increase MSMEs access to technology

The project...

Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

48

mind that the number of respondents is low for robust conclusions and this result needs to be interpreted in qualitative terms (Figure 13).

Figure 13: Effectiveness results of outcome 5 (n=12-13 respondents).

Source: Independent evaluation survey, 2023.

84. A good illustration of coordinated outputs to enhance access to markets to MSMEs is the joint work produced by UNCTAD’s competition and consumer protection policies branch with ECLAC, ESCAP, ESCWA, ECA and ECE. During 2020, multiple regional preparatory meetings between UNCTAD, the Regional Commissions and regional stakeholders discussed how to mobilize participation and build collaboration in designing and implementing these dialogues to raise awareness of the role of competition policies for post- pandemic access to markets by MSMEs. The resulting five online regional dialogues took place between March and April 2021 and had approximately 650 attendees across regions115. This inter-institutional partnership supported the delivery of a global report on the articulation between competition policy and MSMEs (produced in phase I, between June and September 2021)116. This global report provided a snapshot of the challenges MSMEs faced around the world to deal with the impact of the COVID-19 pandemic and drew key recommendations centered around levelling the playing field for MSMEs. It was disseminated with a global policy dialogue, organized by UNCTAD in partnership with ECLAC, ESCAP and ESCWA, in December 2021 (phase 3). This global dialogue explored how competition policy could support MSMEs post-COVID-19, with the presence of representatives and officials of government MSME bodies and competition agency representatives from across the world. It was divided into three sections, one with a focus on digitalization, another on access to finance and government support, and the third on inter-agency coordination between MSME agencies and competition authorities. It was delivered in English, French, Spanish, Chinese, Russian, and Arabic.

115 The UNCTAD-ESCAP-ASEAN dialogue was delivered in English, Thai, Burmese, Bahasa Indonesian and Bahasa Malay and had 149 participants. The UNCTAD-ECA dialogue was delivered in English and French and had 97 registrants. The UNCTAD-UNECE dialogue was delivered in English and Russian and had 140 attendees, 30% from outside the UNECE region. The UNCTAD-ECLAC dialogue was delivered in English, Spanish and Portuguese and had 115 registrants. The UNCTAD-ESCWA dialogue was delivered in English and Arabic and had 150 attendees. 116 UNCTAD (2021). How COVID-19 affects MSME access to markets and competition: A review of key issues and recommendations for future action. NOTE: the previous title is in a press release by UNCTAD and the next refers to the report shared with the evaluation as output F7. Alternate title and reference: UNCTAD (2022). The COVID-19 pandemic impact on micro, small and medium-sized enterprises. Market access challenges and competition policy. Geneva.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

activities and publications provided useful information on MSMEs access to markets

improved my knowledge on how to increase access to markets, including export and integration into value

chains

activities contributed to identifying new/improved measures on consumer protection and competition

activities contributed to identifying new/improved measures on agricultural quality and food loss reduction

The project...

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

49

85. Other cooperation-based and replicated outputs were the three country case studies on the impact of COVID-19 on specific SME sectors in Brazil117, Thailand118 and South Africa119, delivered between March and December 2021. They were implemented with the support of national institutions to ensure a grounded approach in understanding what competition policy is, how it is framed in the country, and how it can impact MSMEs in the context of the COVID-19 crisis. Each case study provided recommendations to competition authorities and representatives and support entities for MSMEs. A global event in June 2021, ‘DA COVID-19 technical cooperation project on Competition Policy & MSMEs’ resurgence’, disseminated the findings of the case studies and promoted policy advocacy on this subject to 238 attendees. Also, five regional webinars with competition authorities and MSMEs’ institutions delivered on how the recommendations of the global and the three national reports could support the resurgence of MSMEs. A follow-up hybrid global workshop, held in Bangkok in 27-28 June 2022, to discuss ‘The Contribution of Competition Policy to the Resurgence of MSMEs post-COVID-19’ had panellists from UNCTAD, ESCAP, DESA, ECLAC, ESCWA and African countries. This event covered recommended practices for digital platforms for MSMEs120 and presented the guidance document for Thai competition and SME institutions121 for 534 virtual participants. It was assessed as relevant (>95% of participants) and effective in enhancing skills, regional cooperation and methods for integrating MSMEs into regional value chains (>95%). This workshop presented the newly designed online course on SMEs and competition policy, primarily for government officials. The country case studies are cited as having inspired the course showing what capacities SMEs needed to engage in better trade practices. This 5-module course was delivered between November and December 2022 to 141 participants (73 male, 68 female) from 41 countries122. The post-course survey indicated that the experience was very/extremely valuable to 98% of the 72 respondents, 81% felt that the course provided them with learning opportunities not available otherwise, 82% felt able to serve the community with what they have learnt, and 68% considered the course relevant to advance in their career.

86. At the level of the indicators of achievement, despite somewhat lacking in specificity, and the high level of ambition and long-term-orientation for a crisis response, considerable progress is observed. The elaboration of the project indicators was reported by informants as one of the challenges in the Surge’s design phase, given the uncertain pandemic context, the lack of clarity on the resources that would be available for each phase of the project, an understanding that the DA requirements for indicators had to reflect development goals (adequate for longer projects123), and time pressure to deliver as soon as possible. As a result, the management decision was to focus on what would be right for MSMEs’ support from each entity’s perspective (i.e., separate streams of work), beneficial for countries, and already available as standard indicators for the entities. Reviews of the initial indicators followed advances in the project and the evolution of the pandemic situation. From phase 2 to phase 3, some indicators were revised to become more ambitious, since there was more time and resources available. According to members of the project coordination team, there was awareness of the broad scope of these indicators, that some of them were not crisis-driven, and it would be difficult to achieve them within a year. Nevertheless, there is also the shared perspective that they are needed to track longer-term effects of the project. A mitigation monitoring strategy by project managers used proxies or partial reporting from specific delivered outputs when short-term impact could not be measured. This is reflected in the Surge project’s final report, concluded in November 2022. Adding to this

117 UNCTAD (2021). Competition and Market Access Policies in the Resurgence of MSMEs in Post-Covid-19 in Brazil. 118 Charoenrat, T., Harvie, C. (2021). Analysis of the impact of COVID-19 on micro, small and medium-sized enterprises (MSMEs) in Thailand from competition policy and market access perspectives. 119 Nsomba, G., Tshabalala, N., Vilakazi, T. (2021). Analysis of the impact of COVID-19 on micro, small and medium-sized enterprises in South Africa. 120 UNCTAD (2022). Harnessing the interaction between digital platforms and MSMEs: Recommended best practices for digital platforms (RDP). 121 UNCTAD (2022). Guidance for action: Supporting MSMEs’ recovery from the COVID-19 pandemic. Prepared for the trade competition commission of Thailand. 122 Kenya, St. Kitts, Botswana, Philippines, Indonesia, El Salvador, Azerbaijan, Trinidad and Tobago, Kuwait, Belize, Brazil, Barbados, Suriname, Eswatini, Burkina Faso, Peru, Malawi. Côte d'Ivoire, Bahrain, Colombia, Russia, Malaysia, Zambia, Ghana, Jamaica, India, Tanzania, Spain, Grenada, Switzerland, Ethiopia, San Marino, South Africa, Nigeria, Senegal, Montserrat, Egypt, Saint Lucia, Zimbabwe, Madagascar, Liberia. 123 It is noteworthy that the DA guidance on the expected outcomes for PRODOCs of joint COVID-19 response projects states that “The outcomes (OCs) describe the changes that are expected to occur as a result of the completion of outputs. The OCs should be achievable within the project’s timeframe and budget and should be specific enough to be measured by the associated indicators of achievement.” Nevertheless, implementers seem to have been driven by an understanding that the PRODOC approval was conditional to a longer-term perspective on outcomes – an understanding that reflected on their indicators too.

50

evidence, this independent evaluation conducted additional desk review with the support of interviewees who shared the results of the latest outputs delivered, and a survey with relevant stakeholders. This evaluation corroborated comments on the project’s final report in relation to finding mismatches between indicators, project phases and some deliverables, as implementing agencies seem to have shifted activities around, repeated the same activities in different outputs or created new ones since the PRODOC. Other activities in the evidence pool were associated with more than one output and are reported here in the output where their content makes more sense. Also, this independent evaluation pooled together evidence from different implementing entities to make an expert judgement of the progress achieved per indicator, not being restricted to the implementing entity associated with specific indicators. This promoted a better global perspective of the results achieved. Importantly, since most quantitative indicators lack a baseline, the baseline used was the number of countries/participants engaged in the activities under that indicator. At a later stage, more evidence on sustainability and impact indicators will be reported in the UNCTAD Division on Investment and Enterprise’s Annual Research and Impact Report. Table 8 summarizes the findings per indicator at the time of this independent evaluation.124 Out of the 26 indicators of achievement, this independent evaluation located evidence of good progress for 11 of them. Some evidence of progress, from some of the implementing entities involved, but not all, or from a limited number of the beneficiary countries/participants engaged in activities under a specific indicator, was found for other 11 indicators. One indicator showed limited progress and activities related to it are continuing in another project. This independent evaluation could not locate evidence for other three indicators. Overall, this evaluation identified levels of progress for 84.6% of the performance indicators. Indicators under outcome 1A are the ones with more progress reported. Outcome 5 is the one with the most uneven performance in terms of evidence of progress identified by this independent evaluation.

Table 8: Level of achievement of the Surge project indicators. Expected Outcomes Indicators

(Important: each indicator was planned to reflect the work of one implementing entity and

was not designed to cover all participating countries)*

SMART* * criteria covered

Level of progress identified

Source of evidence

OC 1A: To improve national capacities on formulating and implementing enabling polices on green, resilient and inclusive entrepreneurship MSME promotion in post COVID- 19 resurgence

IA 1.1. At least 75% of the countries that participated in the project, have adopted and/or implemented a revised/improved policies aimed at building a resilient and competitive MSME sector in post COVID-19 resurgence

MR Significant progress.

DA 2023W Final report, data from UNCTAD, ESCAP and ESCWA

IA 1.2. At least 75% of policymakers that participated in the interventions of this component indicated their enhanced capacity on entrepreneurship/MSMEs policies design and implementation in the context of post COVID-19 recovery

MAR Significant progress.

DA 2023W Final report, data from UNCTAD, ESCAP, ESCWA, survey

IA 1.3 At least 75% of policymakers and other stakeholders of the ESCWA network indicated their improved access to information on best practices on government support measures for post COVID- 19 recovery

SMAR Significant progress.

DA 2023W Final report, data from ESCWA, survey

IA 1.4. At least 75% of policymakers, MSMEs and other key stakeholders indicated their improved access to the ESCWA knowledge hub on issues on the MSME post-COVID-19 resurgence

SMAR Significant progress.

DA 2023W Final report, data from ESCWA

IA 1.5. At least 75% of users of training materials developed under this component indicated their usefulness for their improved capacities in policy- making for enabling MSME environment

MAR Significant progress.

DA 2023W Final report, data from UNCTAD and ESCWA

IA 1.6. Positive feedback of users of publications under this component (assessment reports and other publications)

AR Some progress.

DA 2023W Final report, data from UNCTAD and ESCWA, survey

124 Importantly, the baseline for each indicator was the activity or activities associated with it, rather than the project beneficiaries as a whole.

51

OC 1B: To improve resilience and competitiveness of MSMEs in post-COVID-19 resurgence

IA 1.7. % of Empretec125 trainees expanding or starting a business venture, disaggregated by country and gender, including by people with low literacy

MAR Some progress.

DA 2023W Final report, data from UNCTAD and ECE

IA 1.8. % of Empretec-supported MSMEs able to increase sales in one year, disaggregated by country and gender

MART Some progress. Gender disaggregated data not available

DA 2023W Final report, data from UNCTAD and ECE

IA 1.9. Number of new jobs created by MSMEs which participated in the Empretec programme, disaggregated by country and gender

MAR Some progress. Gender disaggregated data not available

DA 2023W Final report, data from UNCTAD and ECE, interviews

IA.1.10. At least 75% of participants to the entrepreneurship training courses under this component indicated usefulness of the activities for increasing their entrepreneurship and business skills in the related areas by country and gender

MAR Significant progress. Lacking data by gender for some of the training courses.

DA 2023W Final report, data from UNCTAD and ECE, survey

OC2: To facilitate MSME registration and formalization

IA 2.1. Increased number of new businesses registering, disaggregated by country and gender of business owner

R Some progress.

DA 2023W Final report, data from UNCTAD

IA 2.2. Decreased administrative costs to start a business, disaggregated by country

R Some progress.

DA 2023W Final report, data from UNCTAD and interviews

IA 2.3. At least 75% of participants of the workshops on policy guidelines for MSME formalization stated their increased capacity to facilitate MSMEs formalization

MAR Significant progress.

DA 2023W Final report, data from DESA, survey

OC 3: To improve MSMEs access to finance

IA 3.1. Number of participating MSMEs with improved financial literacy and ability to prepare financial statements, disaggregated by country, and gender

MAR Some progress.

DA 2023W Final report, data from UNCTAD and ESCAP, survey

IA 3.2. At least 75% of MSME participants of online training courses state their improved financial literacy in accounting and reporting, including on the SDG reporting, and improved capacity to manage financial resources, by country and gender

MAR Some progress. Gender disaggregated data not available

DA 2023W Final report, data from UNCTAD and ESCAP, survey

IA 3.3. At least 75% of policymakers that participated in the workshop state their increased capacity to facilitate financial inclusion and MSMES access to finance

MAR Significant progress.

DA 2023W Final report, data from UNCTAD and ESCAP

IA 3.4. Positive feedback of users of the publication prepared under this component

AR Significant progress.

DA 2023W Final report, data from UNCTAD, survey

IA 4.1. Number and type of new/improved adopted measures to increase MSMEs access to technology

MR Data not available

-

125 Empretec is a flagship capacity-building programme of UNCTAD for the promotion of entrepreneurship and micro, small and medium sized enterprises (MSMEs) to facilitate sustainable development and inclusive growth.

52

OC4: To increase MSMEs access to Innovation and technology

IA 4.2 Digital community of practice on use of technology and innovation to build resilience to shocks is created with at least 100 members

MAR Limited progress.126

Data from ECA, interviews

IA 4.3. Positive feedback from users of online training materials and assessment report on their usefulness for increased access to innovation and technology

AR Significant progress.

DA 2023W Final report, data from ECA, survey

IA 4.4. At least 75% of participants of training activities under this component indicated usefulness for their increased access to innovation and technology, by country and gender

MAR Significant progress. Lacking data by gender on usefulness for training activities.

DA 2023W Final report, data from UNCTAD, survey

OC5: To enhance MSMEs access to markets

IA 5.1 Number and type of new/improved measures endorsed by policymakers on consumer protection and competition disaggregated by country

MR Data not available127

DA 2023W Final report, data from ECE

IA 5.2 Number and type of adopted new/improved measures on agricultural quality and food loss disaggregated by country

MR Data not available

-

IA 5.3. Increased number of MSMEs in regions targeted by interventions with increased access to markets, including export and integration into value chains, disaggregated by country and gender

R Limited progress. Gender disaggregated data not available

Anecdotal data from UNCTAD

IA 5.4. At least 75% of policymakers that participated in the workshops indicated their increased capacity to improve MSMEs access to markets, including by integration into the value chains

MAR Significant progress.

DA 2023W Final report, data from ECE and ESCAP, survey

IA 5.5. Positive feedback on usefulness of publications under this component (studies, guidelines, videos, recommendation) by users of these publications

AR Some progress.

DA 2023W Final report, data from UNCTAD and ECE, survey

Source: Evaluation team analysis of available evidence. *Given that each indicator refers to the work planned by a specific implementing entity, this independent evaluation was bound by the coverage of activities of the corresponding entity. Nevertheless, whenever possible, evidence from other implementing entities on the same content of the indicator was considered as evidence for that indicator. **Specific, Measurable, Achievable, Relevant, and Time-Bound.

87. The EPF and the cluster-based approach formed an effective framework to organise outputs delivered by different implementing entities under specific outcomes in the Surge Project, and, also, to support the inter-relatedness between outputs whose content cut across different outcomes. Nevertheless, questions remain on the extent to which this approach is suitable for a crisis response (see para. 89). The EPF and the cluster-based approach were introduced in phase 2 of the Surge project as a strategy to provide a global narrative and weave opportunities for collaboration, mostly evidenced by bilateral partnerships between UNCTAD and respective Regional Commissions, and by collaborations between different UNCTAD divisions (e.g., registration and trade). According to members of the project coordination team, UNCTAD already had the EPF as an agreed menu of best practices to support enterprise development, supported by latest resolutions of the UNGA128. Therefore, as the project moved from phase 1 to phases 2 and 3, the EPF components and the cluster-based approach became the design template for the logframe and structured activities around the five outcomes described previously. Activities delivered by different Regional

126 Update received from ECA in feedback to this evaluation report indicates that this work is continuing under the

UNDA13th Tranche project “Innovative approaches for MSME competitiveness to promote trade and inclusive

industrialization in Southern Africa in the Post-Covid context”. 127 There is evidence of national case studies and regional studies collecting measures taken by governments, but these measures do not result from interventions implemented by the project. 128 UNGA resolutions /RES/71/221 and A/RES/73/225.

53

Commissions and UNCTAD’s divisions, then, became conceptually connected, providing the UN with a clear picture of how country-level deliverables fit the global framework. As stated by a member of the PCT, “The original idea was to put together the areas of expertise of Regional Commissions and divisions within UNCTAD under the pillars of the EPF, but each pillar is a different type of product.” It is acknowledged that only UNCTAD has the capacity to deliver a full EPF project at a global level, as this is its flagship intervention model acknowledged in resolutions of the UNGA. Regional Commissions do not have this global mandate on entrepreneurship policy, so the project benefited from their mandates and expertise on MSME development129. Nevertheless, the implementation of the project by the Regional Commissions was constrained by their work programme for the year – previously agreed with governments –, limited staffing resources and online delivery. Figure 14 reflects how the EPF structured the delivery of the project’s outputs by different implementing entities.

Figure 14: Outputs per project outcome and implementing entity.

Source: Final report.

88. An example of delivery that cuts across the EPF pillars is ECE’s series of guidelines and best practices (located under outcome 1.B) that follow a template that covers topics across the spectrum of the EPF: policy, legal and regulations (outcome 1.A), business facilitation and business registration (outcome 2), access to data, information and knowledge, entrepreneurship skills facilitation, access to finance (outcome 3), access to technology (outcome 4), and market access and logistics and supply chains (outcome 5). Other examples indicate the potential for replicability provided by this framework, such as the national entrepreneurship strategies (outcome 1.A), the e-registration platform (outcome 2), and the case studies on competition and access to markets (outcome 5).

89. Interview data with members of the PCT indicates that the Surge’s implementation was consistent with the way UNCTAD implements the EPF, by assessing what exists at country level, making practical considerations on what can be done, and responding to countries’ demands. A risk of this EPF-centred approach in a global crisis response situation is not building enough synergy with partners and facing implementation difficulties imposed by competing local or regional priorities and uneven competencies on MSMEs. Interview findings corroborate this, with informants referring to challenges in balancing regional priorities and competencies with the delivery of outputs and some focal points in the Regional Commissions reporting a lack of knowledge on the role of the EPF in structuring the project. In these cases, project focal points reported the delivery of what was agreed with UNCTAD on specific outputs and outcomes without a coherent conceptual perspective. Some of them felt that decisions on the project were top-down to create a global response, and delivering under the given circumstances was challenging. Informants from different groups of stakeholders shared the perspective that, for a crisis response, most EPF outputs would be low to medium impact, since they follow a long-term development perspective (e.g., the development of national

129 The Subregional office for Southern Africa, for instance, has a mandate to work on MSME development as contained in the outcome statements of its Intergovernmental Committee of Senior Officials and experts Meetings.

0

10

20

30

40

Outcome 1A

Outcome 1B

Outcome 2 Outcome 3

Outcome 4 Outcome 5

Number of outputs per project outcome

UNCTAD DESA UNECA ESCAP UNECE ESCWA ECLAC

54

entrepreneurship strategies). So, in future crises, it would be an advantage to know from the start what EPF and MSME development-related outputs are fit for immediate results and how to implement them accordingly.

How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

90. The Surge Project contributed to the participating country governments’ responses to the COVID-19 pandemic crisis by improving policymakers’ capacity to design and implement enabling policies to MSME resiliency and post-COVID-19 resurgence through studies, technical assistance, training, digital tools, and interventions at the regional level. Survey results from this independent evaluation indicated that 71% of respondents agree or strongly agree that the project contributed to formulate and implement enabling polices on green, resilient, and inclusive entrepreneurship for MSME promotion in post COVID-19 resurgence (Figure 15). Given the small response rate, these findings cannot be generalisable to all project participants and represent the perceptions of those who responded to the question.

Figure 15: Survey responses on the project contributions to formulate and implement enabling policies to MSMEs (n=41 respondents).

Source: Independent evaluation survey, 2023.

91. This was achieved through higher level outputs, such as the Entrepreneurship National Strategies resulting from technical assistance of UNCTAD to the governments of Seychelles, South Africa and Uganda (outcome 1.A) and country-specific case studies with tailored recommendations and follow-up workshops for knowledge dissemination in the Asia and Pacific region (outcome 1.B, see Box 1 below) and Latin America, Southeast Asia and Africa (on the role of competition and consumer policies in Brazil, Thailand and South Africa, outcome 5). The production of assessments on the impacts of COVID-19 on MSMEs at the regional level (Southern Africa in outcome 4, multiple studies in Latin America in outcome 5), followed by policy advocacy supporting inter-institutional partnerships between different areas of government to align policies in efficient and effective ways and learning from exchanges with other countries represent a great contribution to institutional strengthening (e.g., Europe, Latin America). Informants referred to previously ‘unthinkable’ dialogues, collaborations, and policy alignment between different ministries, with policies for the MSMEs becoming part of the bigger inter-institutional government agenda. In Latin America, interview data referred to how SME policy capacity was initially poorly coordinated with other productive development policies. During the pandemic, however, the same institutions adopted a different working methodology by coordinating with other ministries, such as health, economy, and other productive ministries. The intention was to make the most of the money and avoid dispersed actions. The modalities of policy implementation were a) localized, with targets per sector, or certain geographical locations, b) made flexible and adaptable on a case-by-case basis, according to set targets and sectors most affected, and c) with support from local governments. This was confirmed by government officials in El Salvador and Peru. In Kenya, the survey results supported government action in facilitating access to finance to MSMEs through, for instance, the creation of a targeted fund to increase the access of MSMEs to digital platforms. Now, the new government is focused on MSMEs and DESA is supporting the design of a strategic plan aligned with that. In Zambia, the Ministry of Small and Medium Enterprises improved the participation of women in business through various cooperatives.

The project contributed to formulate and implement enabling polices on green, resilient and inclusive

entrepreneurship for MSME promotion in post COVID-19 resurgence

0% 10% 20% 30% 40% 50% 60% 70% 80%

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

55

Learning from other countries in webinars and regional dialogues is reported as an additional contribution to building and strengthening government capacities to respond to the crisis.

92. In relation to providing innovative tools to government, the development and implementation of specific tools, such as the e-registration platforms (outcome 2) in El Salvador, Benin, Cameroon, and Mali improved the delivery of public services to MSMEs through this digital technology. Real-time data on the number of businesses, categorization by type of business, locations, main business activity, gender, age, etc., from the e-government platforms, is reported to be used in the design of better entrepreneurship policies (outcome 4). The e-registration platform increased the number of MSMEs formalized by about 90% in El Salvador130 and Benin131, and 40% in Mali132, supporting mostly women and youth. Benin became the fastest country in the world to register a business simultaneously with 5 different public agencies (in less than 2 hours). These positive results led to requests from these countries to expand the functionalities of the platform, e.g., by providing analytical financial statements, modifications in the business registry, grant applications to access financial resources (outcome 3), and access to social security benefits for employers and employees. The Prime Minister in Cameroon is signing a new Decree on entrepreneurship making online business registration mandatory.

93. Another type of contribution, regarding capacity-building, refers to the development of the online training course on accounting and reporting on the SDGs (outcome 3), with a specific guidance in French, Russian, Arabic, and Chinese and training workshops for MSMEs in English, Spanish, French, Russian, Arabic and Chinese. This approach expanded the outreach of these courses to over 600 participants in 39 countries, indirectly contributing to countries’ advances on the sustainable development agenda. Having these relevant materials available in more than the three official UN working languages represents an additional contribution to governments, since it may expand opportunities for knowledge uptake through other initiatives in the country.

94. It is noteworthy that contributions to governments often take more time than the project duration, since deeper changes require rounds of consultation and chains of approvals. Hence, challenges to make transformational contributions to governments are often beyond the project’s managerial control, such as government’s capacities, institutional changes in governments due to shifts in power or political priorities (e.g., Georgia, Moldova, The Gambia, Brazil).

Box 1: ECE’s guidelines and best practices – from a general approach to supporting country-specific responses. In phase 1, ECE developed two general guidelines and best practices for MSMEs. One of them was on circular economy and sustainable resource management and critical raw material supply chain solutions133 and the other on delivering energy-efficient products and in providing renewable energy equipment134. They refer to the UNFC and UNRMS as a global framework that can help MSMEs to build innovative business models and gain from the new opportunities created by the pandemic crisis. Under outcome 1.B, this expanded to cover a wide set of studies and interventions aimed at supporting governments to develop an enabling environment for MSMEs (outcome 1.A) in the areas of circular economy and critical raw materials, and energy efficiency and renewable energy. Overall, these country-specific guidelines replicate the same structure of the original documents, allowing for country-specific analysis and recommendations, while providing information comparable between countries and sub-regions within the ECE region. They cover opportunities for entrepreneurs, business development managers, technical experts in existing and potential MSMEs, and financiers on the following topics: business facilitation and business registration (outcome 2), policy, legal and regulations (outcome 1.A), access to data, information and knowledge, entrepreneurship skills facilitation (outcome 1.B), access to finance (outcome 3), access to technology (including digitalization, outcome 4), market access, and logistics and supply chains (outcome 5). Recommendations to governments include providing immediate financial support to MSMEs in the event of a crisis (e.g., temporary tax waivers, temporary tax breaks, national financial programs to support MSMEs, offer payment delays, wage subsidies, line of credit and guarantee free loans to MSME), supporting MSMEs in networking and information gathering, and adopting policies to avoid workforce layoffs (e.g., offer employee development

130 In El Salvador, 12,000 new businesses formalized (40% women-led). 131 In Benin, 52,000 businesses formalized between 2019 and 2021, mostly by young entrepreneurs. 132 In Mali, business creations increased from 6,087 in 2019 to 14,796 MSMEs in 2021. 133 UNECE (2020). Guidelines and Best Practices for MSMEs to assure resiliency and progress towards a circular economy in sustainable resource management and critical raw material supply chain solutions. August 2020. 134 UNECE (2020). Guidelines and best practices for micro-, small and medium enterprises in delivering energy-efficient products and in providing renewable energy equipment.

56

programs, such as in digitalization, wage subsidies to cover temporal downsizing and financial support to laid-off workforce). Following the OECD strategic directions for SME development, these recommendations include the development and promotion of 1) an institutional framework and operational environment (e.g., coordination mechanisms for SME policy, collect relevant SME statistics and disseminate the results, improve business closure and insolvency procedures, use targeted public procurement for MSMEs, and a specialized clean energy financial institution to enable MSMEs as a driving force to clean and green economic recovery); 2) access to finance (e.g., public grants, especially for MSMEs working on low-carbon technologies, supply-side financial skills of banks, demand-side financial education for entrepreneurs, credit guarantee scheme, other non-bank financing for SMEs, with a clear governmental guidance); 3) skills and entrepreneurial culture (e.g., training needs assessments, implementation of vocation training policies, access to non-formal training, women’s entrepreneurship); 4) internationalization (e.g., export promotion activities and export-related skills, SMEs to adapt to DCFTA requirements, financial support to exporting SMEs, and SME integration into global value chains); and 5) innovation and R&D (e.g., legal framework for innovation, collaboration between industry and academia, and financial support to SMEs for innovation). Country-specific guidelines on delivering energy-efficient products and in providing renewable energy equipment were produced for Georgia and North Macedonia in phase 2, and expanded to Armenia Albania, Kyrgyzstan, and Republic of Moldova in phase 3. Likewise, customized guidelines on circular economy in sustainable resource management and critical raw material supply chain solutions were produced for Tajikistan and Ukraine in phase 2 and expanded to Bosnia and Herzegovina, Kyrgyzstan, Kazakhstan, and Serbia, in phase 3. Once the guidelines for a country were done, there were regular contacts with the government’s agency responsible for that piece, for dissemination and organization of the online training. These were about half-a-day training sessions, sometimes with presentations from the WB, or UNDP. There were a couple of events by the end of the project delivered face- to-face. Also in phase 3, an updated version of the guidelines and best practices for MSMEs in delivering energy-efficient products and renewable energy was delivered. This update responded to changes in the work environment of MSMEs in this sector in the ECE region, where countries laid out economic recovery through ‘green’ approaches. A follow-up on the original guidelines’ recommendations indicated that “Most of the countries in the ECE region have adopted various economic and operational tools for recovery from the COVID-19 crisis, helping MSMEs in the energy efficiency and renewable energy equipment sector to boost their growth and to bring the clean energy transition across the region. Intervention measures like low-interest financing, innovation grants and funds, operational relief funds, provision of subsidized/common manufacturing units and workplaces, etc. are being issued for supporting MSMEs during the COVID-19 crisis.” The assisted countries can now request technical assistance from the regular programme on technical cooperation to implement the recommendations in these guidelines.

95. The absence of strong policymakers’ networks on policies for MSMEs and of mainstreaming gender- and minority-inclusive approaches hindered the effectiveness of the Surge project in supporting governments’ responses to assist MSMEs recover from the COVID-19 crisis. The two initiatives on building policymakers’ networks, one by ESCAP and another by ECA, showed high relevance and appreciation by country stakeholders interviewed. In the case of ESCAP, after three meetings and a set of associated outputs, such as input studies and follow-up meeting reports, interest on the topic of MSMEs faded away and this network seems to be inactive now. In relation to ECA, the plan was to deliver a ‘Digital community of practice’ (Techni Africa portal) on technology and innovation for SMEs in Southern Africa. Progress was made in developing and validating the digital platform (TechniAfrica) that will host the community of practice, but it had not been operationalized by the closing of the Surge project.135 Another tool with potential to support strategic networks of policymakers is the DEPAR portal. Although it has the digital structure and input materials in place (e.g., the ecosystem maps), there seems to be a lack of institutional effort to build and maintain an active network, as a community of practice. Networks of policymakers could sustain mutual and regional learning on policies to support MSMEs in different sectors, including in cross-border trade.

96. The project could have improved its support to government capacities in cross-border trade by involving government officials, i.e., immigration officers, customs and police with crucial roles on CBT, in the training courses, as trainees, or having a dedicated training session for them. Interviewees referred to their lack of knowledge on CBT and business skills and negative attitudes toward cross-border traders as remaining barriers to facilitating cross-border trade, especially for women. This independent evaluation also found that

135 Update on the operationalization and launch of this platform was provided by ECA in feedback to this evaluation report. The actual operationalization and launch of the digital platform, titled “TechniAfrica”, to be hosted by the SADC Business Council, will be implemented under the UNDA13th tranche project “Innovative approaches for MSME competitiveness to promote trade and inclusive industrialization in Southern Africa in the Post-Covid context”.

57

some sectoral interventions on access to markets were gender-blind by design and they benefited women because a large proportion of workers in those sectors are female (e.g., cocoa and coffee in Peru). Informants referred to a lack of government interest in looking at gender issues amid a pandemic crisis, so implementing agencies, in those cases, focused on their priorities.

What innovative approaches or tools, if any, did the response use, and what were the outcomes and lessons learned from their application?

97. One of the key institutional innovations in the Surge project was the use of adaptive management through an increased consultative process, and a phased approach. The COVID-19 crisis was an opportunity for the DA programme to innovate on how it promotes sustainable development. Its orientation remains geared to the long run rather than to crisis responses, but the efforts to make the Surge project and the other COVID-19 response projects work created momentum to stimulate more joint projects, according to interview informants. They refer to the advantages of learning what other regions are doing and the possibilities to connect through digital technologies, reducing the costs of inviting someone from Asia to an event in Latin America, for instance. Informants also acknowledge the constraints to implementing such collaborations, such as distinct programmes of work in different entities, translation costs, time zone challenges, etc., but feel that the project promoted the interest and motivation for more synergies. The emerging benefits of, and new expertise, on joint projects through frequent consultation with DA focal points and relevant stakeholders (e.g., entities’ staff working on MSMEs) made this modality a good practice to be considered when planning regular DA projects that could benefit from joint implementation. The streamlining of guidelines136 to support the phased approach associated with the Surge project is also an innovative practice. The phased approach was qualified by several UN staff in the implementing entities as ‘unique’ in DA history and seen as one of the main innovative approaches at the institutional level.

98. The Surge Project developed innovative approaches and tools to respond to country needs, such as online delivery of trainings and knowledge-sharing events, the introduction of MSMEs into policy discussions in response to the pandemic crisis, and e-government services. Some of these innovative approaches and tools are being incorporated into the daily work of stakeholders. The most highlighted innovation by relevant stakeholders was the conversion of activities into the online format. Some advantages of this format were the massive extended access to training, inclusive during lockdowns (e.g., e-learning EPF TOT courses delivered, outcome 1.A, Empretec TOT courses, outcome 1.B, courses on role of technology and innovation and green technologies for MSMEs, outcome 4, course on competition policy for MSME resurgence, outcome 5), the opening of opportunities for exchanges between implementing entities (e.g., joint events and participation in each other’s events, all outcomes) and between policymakers (e.g., regional webinars, global conferences, all outcomes), and to bring together multiple stakeholders for wider discussions (e.g., webinars on the health economy in Africa, Asia and Latin America, outcome 1.A). At the content level, an innovative approach praised by different stakeholders was the focus on MSMEs in the context of competition policies (Latin America) and the inclusion of MSMEs in the existing circular economy and resources management tools (Europe) to answer to a global pandemic crisis. This approach had not been thought about and it was transformative to see it associated with the rapid digitalization of services, such as delivery services, and in policy guidelines on circular economy (outcomes 1.B and 5). This brought MSMEs to the mainstream debate on digitalization and competition policies and into ECE’s work on energy efficiency. Innovative online tools that generated rapid results were the e-government and e-registration tools (outcome 2), since they kept public services working during lockdowns and increased the interaction between MSMEs and public services in a digital one-stop shop that can be accessed by computer or mobile phone and saves on paper for accounting books. An additional gain is improving the digital literacy of users.

99. Respondents to the survey conducted by this independent evaluation listed other innovative or unique approaches and tools of the Surge project that they thought the UN should consider replicating or upscaling, with 57 contributions from 48 respondents. Here is a summary of these responses, categorized according to main themes:

136 The timeline for concept note and PRODOC development was shortened, and the concept note guidelines were

shorter than those for regular tranche projects (as the concept note was only subject to approval by the DA Steering

Committee, and not the General Assembly). The PRODOC template still included all the elements of the regular

tranche prodoc, and there was an additional Phase 3 budget proposal.

58

● Leaving no one behind (15 comments): reaching out to resource-poor areas in developing countries and seeking cooperation to do so and following up with them; business that clearly accommodates those with disabilities; supporting the local women and youth-led initiatives to increase accessibility of services by women and girls in hard-to-reach communities (e.g., rural areas); promoting and upscaling global entrepreneurship with e-commerce, especially for marginalized and vulnerable groups; considering the triple impact (economic, social and environmental) of women-led businesses; profiling women-led MSMEs was insightful and helped to develop legal amendments supportive to women employability.

“I live in a rural and poorest province in my country. I was impressed by the reach of the project because while my country (the capital city) is hosting many UN regional offices, UN programmes are unheard of in these parts, though I suspect they are visible elsewhere in the country and region.”

● Capacity-building for MSMEs, entrepreneurs and communities (12 comments): capacitating communities to engage in entrepreneurship; financial literacy training; entrepreneurial training (including on appropriate technology application and innovation); diversification of business opportunity innovations and mentoring of entrepreneurs.

● Scope of the project and mode of delivery (9 comments): online activities (i.e., courses, webinars and information dissemination); large-scale multifaceted project allowing for rapid intervention; the combination of independent study, YouTube videos, exercises, moderated online participation and tests facilitated greater engagement with the material, learning from the facilitator, and peer learning; room for project managers to try out new ideas, to innovate; undertaking a needs analysis and developing programs based on the needs of the MSMEs.

● Access to finance (5 comments): a simplified micro-lending business method, such as funding local small businesses with a day-to-day transaction of lending to collect daily loan offers with affordable interest rates; innovative digital finance solutions and regulatory frameworks to support both financial inclusion and growth of SMEs, especially those owned and led by women (new perspective for the private sector); increasing non-refundable investment support for SMEs in developing and underdeveloped countries; supporting MSEs to access grants; development of investment attraction mechanisms for the energy system.

● Institutional capacity-building and national capacities for policymaking (5 comments): opportunity or skills-sharing from neighbouring countries and building business relationships and Empretec relationships; face-to-face courses for policymakers to support SMEs; improving cooperation between countries; hybrid policy dialogues including the private sector; improving private sector governance to make it an engine of growth.

● Access to markets (4 comments): digitalization in a globalized world; reducing border barriers; the role of competition policy in the economic recovery of MSMEs (e.g., MSMEs continue to face challenges related to their interaction with bigger counterparts in the agricultural sector and current competition laws do not present effective solutions); the work dedicated to the analysis of structural barriers to trade helped to identify measures necessary for facilitation of exports in Africa.

● Guidance for crisis response (4 comments): recommendations to respond economically to a crisis; emphasis on sustainable and inclusive development; studies might contain innovative and/or unique conclusions and recommendations (should upscale and replicate); the whole approach was innovative, enabling one-stop shop for mechanisms and best practices towards post-COVID-19 resurgence of the MSME sector.

● Access to technology and innovation (2 comments): unique in tackling the novelty of COVID-19 for future resilience (upon lessons learned); the project's efforts to leverage technology and digital solutions for MSMEs are forward-thinking.

● Better policies and government strategies (1 comment): improving EE/RES (energy efficient/renewable energy) secondary legislation.

100. Challenges to the institutionalization of some of these innovations include the digital gap and the lack of technological infrastructure and literacy. Two examples illustrate these challenges. First, the pilot tests of the online version of the Empretec training indicated that, even if online training had worked well in training of

59

trainers’ courses, this is not the case for entrepreneurs. Empretec is a lengthy training programme that includes the observation of behaviour and the performance of team tasks, and the online format does not favour these characteristics of the program. Moreover, the course’s online games and activities would not be supported by the bandwidth and the computers the entrepreneurs have access to. Second, the implementation of the e- registration platform in El Salvador required many intermediary – presential and paper-based steps – to include female entrepreneurs of low literacy and technological skills. Implementers developed a paper version of the platform, where entrepreneurs could write down their financial information until they developed the financial and technological skills needed to use the platform. They needed a 6-month follow-up with visits by the implementers and, during training, they would require their help and the help of younger entrepreneurs to, for instance, create an e-mail account and a password and access it afterwards. Another challenge, at the implementing entities’ level, is access to digital platforms for large conferences and meetings. ECE, for instance, depended on UNDP to set up Zoom meetings with interpretation or had to hire interpreters, since the tools available to them would not offer this service.

5.4 Sustainability

What measures were adopted to ensure that outcomes of the response would continue after the project ended?

101. From the perspective of the implementing entities, the Surge project sowed the seeds for joint projects that are being planned or may emerge in the future. Lessons learned from the responses to the COVID-19 crisis seem to be part of a process of internal reflection by the DA-PMT to support better the design and evaluation of joint projects. The lessons learned from the 5 COVID-19 response projects will be reviewed and discussed at a meeting of the DA Network, as soon as all of these project evaluations are completed. Meanwhile, the management response to the final evaluation of the DA T10 Programme on Statistics and Data137, which report138 also took into consideration the impacts of the COVID-19 pandemic crisis, addresses actions to be taken in the course of 2023 and 2024 to improve and develop guidelines on planning, monitoring, and evaluation of joint projects, especially those with budgets above USD 1 million. In June 2023, DA-PMT expressed its intent through interviews to do more joint projects with a long-term view and responding to the evolving world. Since funds are non-earmarked, they can respond to the mandate of UN entities, including by operationalizing the HQ’s analytical work by cutting silos. Joint projects currently in the design phase (16th tranche) refer to the triple crisis – food, water, energy – and the food crisis response for Ukraine. The former has a global scope, covering urban resilience, energy security, financing etc., with a regional perspective (e.g., the climate crisis plays out differently in Europe and in Africa). DA-FPs interviewed acknowledge the uptake of learning from the COVID-19 response in the DA call for crisis-response projects. They refer to faster procedures, with an improved system of calls for proposals supporting the development of concept notes, project documents, and improved templates that are simpler and more concise. Reported improvements include clearer and more specific feedback of the DA-PMT to proposals, e.g., details on indicators, countries, logframe, including suggestions for aggregated activities. At the regional level, informants referred to how partnerships between Regional Commissions in implementing the Surge project have promoted learning about what others are doing and raised interest in seeking further partnerships. Some of the key factors that might limit or facilitate such joint initiatives, under regular project planning conditions, are the availability of resources, the Regional Commissions’ mandates, and their priorities in a moment in time. Future evaluations of joint projects that are planned under such regular conditions (i.e., not called for to a swift response to a major unpredictable global crisis) might benefit from considering evidence on these and other factors to demonstrate the added value of joint projects under more predictable global and regional contexts.

102. There is evidence of the sustainability of e-government tools, training activities and regional and national interventions, with measures related to expanding the functionalities of digital tools, training courses and the formalization of agreements between partner institutions. Across regions, e-registration platforms are being increasingly used by entrepreneurs, mostly youth, with their potential being acknowledged by MSMEs, governments and other relevant stakeholders, such as banks. The result is new requests from different countries to expand the functionalities of these digital platforms and the higher probability that they will remain active in the future. In Cameroon, for instance, the government is signing a decree to make e-registration

137 https://www.un.org/development/desa/da/wp- content/uploads/sites/52/2023/05/MR_to_Final_Evaluation_Report_1617A.pdf 138 https://www.un.org/development/desa/da/wp-content/uploads/sites/52/2023/01/DAT10-Programme-on- Statistics-and-Data-Final-Eval-Report.pdf

60

mandatory. Surge’s managers referred to the e-government platforms as a tool that can be available in crisis response to keep government offices operational. However, there is awareness that these platforms may not respond to all types of crisis.

103. In Southern Africa, the Surge project strengthened relationships between the ECA’s Sub-regional Office for Southern Africa and the SADC Business Council, with joint meetings on MSMEs being held to this day139. Recommendations from the side events held at the SADC Industrialisation Week 2021 and funded under the Surge project were incorporated into the key priority areas for stakeholders, including member states, in the Lilongwe Declaration, issued at the 5th Annual SADC Industrialisation Week (November 2021), under the section on SME and local development, gender, and youth.

104. The project raised the opportunity to pilot an UNCTAD/Empretec Farming as a Business training with small-holder vegetable farmers in Greater Accra, Ghana. However, despite the pilot’s success in empowering farmers who embraced the improved practices with measured increased yield and income, the sustainability of this pilot is not ensured, since its small scale may prove challenging in sustaining access to markets and pro-poor initiatives. In Nigeria, the Entrepreneurship Training Workshop has been recommended by former participants from the government, bringing in more government agencies and state governments to the program, thus enhancing its sustainability.

105. In Latin America, sustainability is rooted in strengthened links between government officials and their stakeholders, such as MSMEs, business associations, and training centers. Based on the courses of the Surge project, ECLAC has been supporting subregional and national initiatives for training MSMEs, using the tools generated to help and promote greater incorporation and formalization of MSMEs in the region, with special emphasis on those led by vulnerable groups such as women and youth. This follow-up work involves different divisions at ECLAC, such as international trade, social development and gender. Also, as a follow-up to the online generic courses on cross-border trade, ECLAC received sub-regional and country requests for tailor- made courses. This generated a customized course for Central America, organized in collaboration with the Regional Center for the Promotion of MSMEs (Cenpromype), on “the "Development of strategic capacities on public policies for electronic commerce" (2022140). This was followed by the planning of a Portuguese version of the course for Brazil.

106. In Thailand, after the end of the Surge project, the SME authority (OSMEP) and the Competition authority (OTCC) signed an agreement in August 2023 that incorporates SMEs into competition policies. This is evidence of the implementation of recommendations from the national case study discussed in the ‘Formal consultation between UNCTAD and Thai institutions: OTCC, NESDC, OSMEP, ISMED and SME Development Bank under the UN COVID19 Project: SME resurgence’ in which representatives of these two institutions were present.

107. The beneficiaries’ perspective on the potential impact and sustainability of results at the MSME level of the Surge project indicates potential long-lasting effects on resilience and competitiveness in post-COVID- 19 resurgence, through high-level achievements in each project outcome. MSMEs and government officials report on implementing activities learned during capacity-building, advantages of the new registration facilities, improved management practices, access to knowledge, empowerment of women and youth entrepreneurs, preparedness to respond to crises through innovation, access to finance and markets, and improved capacities of policymakers. Across the board, 73% of 106 survey141 respondents agree or strongly agree that the project contributed to improving the resilience and competitiveness of MSMEs in post- COVID- 19 resurgence (Figure 16). Given the small response rate, these findings cannot be generalisable to all project participants and represent the perceptions of those who responded to the question.

Figure 16: Contribution of the project to improve the resilience and competitiveness of MSMEs in post-. COVID-19 resurgence in a sustainable way (n=106 respondents). Source: Independent evaluation survey, 2023.

139 The latest one was a 3-day regional meeting, held in June 2023. ECA partners with SADC Business Council on technology and innovation for MSMEs in Southern Africa | United Nations Economic Commission for Africa (uneca.org) 140 136 participants mainly from Belize, Costa Rica, Dominican Republic, El Salvador, Guatemala, Honduras, and Panamá. 141 Survey conducted by this independent evaluation in July-August 2023, responded by responded by government, MSMEs, and other private sector-related stakeholders (e.g., business associations, commercial registry offices).

61

108. Most of these respondents also agree or strongly agree that the project impacted on better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups (64%) (Figure 17). Given the small response rate, these findings cannot be generalisable to all project participants and represent the perceptions of those who responded to the question.

Figure 17: Contribution of the project to better outcomes for women and youth entrepreneurs and other vulnerable groups in a sustainable way (n=105 respondents).

Source: Independent evaluation survey, 2023.

109. Survey results confirm the predominance of positive assessments of the potential impact of the project in each outcome. Because of the low number of responses in outcome 2 (between 2 and 3 respondents), outcome 3 (9 respondents), outcome 4 (11 respondents) and outcome 5 (between 8 and 9 respondents) to the questions on the potential impact of the Surge project, these results are considered as qualitative information that reflects the assessments of those who responded and do not represent a robust finding in quantitative terms.

110. Figure 18 presents the survey results for the impact of outcome 1.A, where 71% of respondents agree or strongly agree that the project contributed to the formulation of better policies for MSMEs.

Figure 18: Contribution of the project to formulate and implement enabling policies for MSME post-COVID- 19 resurgence in a sustainable way (n=41 respondents).

0% 10% 20% 30% 40% 50% 60% 70% 80%

The project contributed to improve the resilience and competitiveness of MSMEs in post COVID-19 resurgence

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

0% 10% 20% 30% 40% 50% 60% 70%

The project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and

marginalized groups

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

62

Source: Independent evaluation survey, 2023.

111. Figure 19 shows moderate impact of the project on creation of new jobs, increases in sales, and creation of new or expansion of existing business ventures, with between 42% and 57% of the survey respondents agreeing or strongly agreeing with the survey statements (outcome 1.B).

Figure 19: Contribution of the project to improve the resilience and competitiveness of MSMEs in post-COVID- 19 resurgence in a sustainable way (n=33 respondents).

Source: Independent evaluation survey, 2023.

112. Qualitative survey responses regarding the potential impact and long-lasting effects of the project for beneficiaries were mostly about gains to MSMEs’ resilience. Out of 47 comments, 25 referred to impacts for MSMEs. They cover multiple ways in which the project supported MSME’s survival and resilience. The main service of the Surge project was capacity-building for entrepreneurs, such as in relation to registration, internal management practices, good guidelines towards circular economy, knowledge sharing about critical issues (e.g., market dynamics and export opportunities and barriers), empowerment of women (incl. women with disabilities) and youth to do business, access to support mechanisms, and contribution to community development (incl. on green technology), and preparedness to respond to crises through innovation and reaching out to consumers. Regarding MSMEs’ operation, comments refer to increased access to finance and support programs, economic improvement through income generation activities, methods to improve access to markets and supply chains through innovation, facilitation of business operation through e-commerce and digitalization, and contribution to increase the number of MSMEs. The 16 comments on the results of the project that were beneficial to governments covered policy advice on national MSME policies (e.g., e- commerce, law enforcement, sustainable development, gender-responsive support, MSME formalization,

The project contributed to formulate and implement enabling polices on green, resilient and inclusive

entrepreneurship for MSME promotion in post COVID-19 resurgence

0% 10% 20% 30% 40% 50% 60% 70% 80%

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

0% 10% 20% 30% 40% 50% 60%

contributed to expanding my business venture or to start a new business

supported my business increasing sales

contributed to the creation of new jobs

The project...

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

63

new support schemes for MSMEs), e-registration, stimulus to public-private partnerships, awareness of the leveraging power of ICTs for MSMEs, awareness of the role of rural areas for development, and grounded work through project’s partnerships with local institutions.

113. Another source of evidence for sustained results was the survey responses on the ways in which beneficiaries use the knowledge or skills acquired through the project. Three main categories of knowledge/skills use emerged out of 71 comments:

● Use of knowledge or skills to improve MSME policy (22 comments): project beneficiaries reported that they had been raising MSMEs’ issues in new projects to respond to the COVID-19 crisis (e.g., introduction of entrepreneurial training, business counselling and access to credit), improving policies for youth employment and community development, incorporating MSMEs’ challenges in accessing markets and finance into work in competition law, mainstreaming sustainability principles in MSMEs’ development initiatives, supervising partnerships between MSMEs and larger companies, onboarding MSMEs to the e-commerce platform, easily assessing women’s problems in farms, establishing priority policies for women-owned, female-intensive businesses and social enterprises, using analytical tools (E-view, SPSS, GIS) to identify policy trends and priorities for MSMEs, sharing best practices within the organization and with other partners (e.g., the critical role of cooperatives in formalization). An unexpected impact was on increased exchanges between policymakers and the UN through presentations of the work they have done, and another was on changing cultural relationships with the environment:

The ocean economy is not very well explored in my area. The ethnic groups have always avoided water out of cultural and spiritual reverence. My development programmes never included the ocean economy until after this project. I have started an awareness initiative that presents the ocean as a possible and lucrative source of livelihood. I believe the initiative is making a breakthrough because locals (though still very few) are beginning to lodge applications for fishing licenses to the authorities.

● Use of knowledge or skills to strengthen MSME resilience (24 comments): project beneficiaries

reported that they have been applying their new knowledge in supporting and mentoring MSMEs in topics such as financial literacy, sustainability and ecological concerns, product exportation, and diversification of business opportunities. These consulting services have been assisting family enterprises too. MSMEs reported that they have been applying the new knowledge in their business growth plans and accounting systems, in designing and upscaling projects within the circular economy and the energy sector, creating an association to buy larger volumes from suppliers, applying entrepreneurship competencies to avoid fraud, diversifying production with facial masks for COVID-19, improving production by reducing pollution, implementing biosecurity protocols, and using digital technologies to reach out to prospective clients, do business marketing, expand collaboration with partners to participate in tenders and expand the business activities. Two unexpected impacts were a beneficiary who started an NGO to focus on climate change, and another who installed wi-fi internet in his village using solar panels to ensure that they could continue working remotely.

● Use of knowledge to take training forward (25 comments): beneficiaries reported that they are taking training forward either by taking further training (e.g., on green technologies, computer training) or by training others (e.g., teaching at vocational courses, designing new courses for MSMEs about internationalization and digitalization, and sustainable development, developing new training material for national counterparts, business development services’ offers). They had also been using what they learned to explain to work colleagues how to operate more effectively and efficiently, to do female entrepreneurship advocacy to other women entrepreneurs, and to protect their families from COVID- 19.

114. Challenges for beneficiaries to use their new knowledge or skills refer, mostly, to lack of financial resources, follow-up mentoring, and a favourable business environment. Fifty-three survey respondents indicated their difficulties in making use of the knowledge or skills acquired through the Surge project. The main difficulties were around unfavourable business environment for MSMEs in the country or weak business ecosystems, lack of local resources (policies, technologies, access to remote regions), financial constraints

64

(including barriers to access financial resources by entrepreneurs and governments), turnover of government officials, low political interest and commitment from senior management, limited technical skills and resources, lack of country data for effective policy analysis, lack of education for sustainable development, lack of entrepreneurial mindset in the community and among small business owners, lack of skilled workers, lack of post-training follow-up and mentoring, how to transfer knowledge to complex country situations, challenges to coordination and synergies between relevant agencies, and internet costs.

115. Beneficiaries’ suggestions for ways in which the UN could support MSMEs in their countries covered continuing to invest in capacity-building activities, facilitating access to finance, supporting inter-institutional coordination inside the country’s government bodies, and increasing interventions at the local level. Survey respondents suggested numerous ways in which the UN could support MSMEs in their countries (72 comments). They can be grouped into the following main categories: continue developing capacities of all actors (i.e., governments, MSMEs, other national partners, such as chambers of commerce and business development services) through training, mentoring and webinars, strengthen or contribute to facilitating access to finance for the MSMEs (including seed money, grants, access to credit, etc.), support improving the coordination across ministries of MSMEs-related policies, and increase interventions at local level (i.e., supporting NGOs and MSMEs outside of the main cities, supporting the organization of cooperatives, launching communication campaigns for information dissemination), conducting more regional and country-specific studies (e.g., policy papers, sectoral approaches), and provide tools to support internationalization of MSMES (e.g., network program). There is great emphasis, both in terms of number of comments as well as in relevance for inclusive and sustainable development, on the inclusion of women, youth, persons with disabilities and rural entrepreneurs across these suggestions.

5.5 Gender, Human Rights, and Leave No One Behind

116. This section explores the extent to which gender equality, human rights and leaving no one behind perspectives were considered in the design and implementation of the project.

To what extent were perspectives on gender equality, protection of human rights and reaching underserved groups integrated into design and implementation of the project? What results can be identified from these actions?

117. The Surge Project´s design was guided by two key UN documents that provided clear direction for the integration of gender, human rights, and LNOB: The UN Secretary-General’s Shared Responsibility, Global Solidarity report and the UN framework for the immediate socio-economic response to COVID-19. The Global Solidarity report 142 provided overall guidance by recognizing that the most vulnerable were the hardest hit by the pandemic, stating the UN’s commitment to help people and societies, especially the most vulnerable, and emphasizing the need for full respect of human rights, ensuring no one is left behind. Aiding in implementation of the Global Solidarity’s guidance, the Framework143 outlined detailed responses to the pandemic with specific deliverables and activities related to gender dimensions, human rights and Leave No One Behind (LNOB). Particularly, the relevant responses including the assessments were expected to be gender- responsive and involve a human rights-based approach to data collection with disaggregated data such as age, sex, migratory status, health status, socio-economic status, place of residence and other factors. The analysis of the human rights and gender impacts would then inform the design of policies that address these risks considering gender aspects and disability-inclusive responses. The Framework also provided a set of indicators to monitor and assess the human rights implications of the COVID-19 crisis, including socio- economic impacts and LNOB aspects. In addition, the Guidelines for the Preparation of Project Document for the Development Account144 required the applicants to outline how the project would contribute to gender equality and the enhancement of human rights, with particular emphasis on “leaving no one behind”, taking into consideration how the specific needs of groups such as youth, persons with disabilities, older people, refugees, migrants, the poor, and others would be addressed.

142 Shared Responsibility, Global Solidarity: Responding to the socio-economic impacts of COVID-19. March 2020. 143 A UN Framework for the immediate socio-economic response to COVID-19. April 2020. 144 https://www.un.org/development/desa/da/wp-content/uploads/sites/52/2022/07/T15-Project-Document- Guidelines-vf.docx

65

118. In line with the guiding UN documents, the Surge Project aimed at supporting the MSMEs that were considered the most vulnerable to COVID-19 within the private sector, highlighted the gender, HRBA and LNOB perspectives and fairly covered the gender aspects. Youth and other vulnerable groups were also covered, though to a lesser extent.145 Given the larger number of vulnerable groups working in the informal MSMEs- particularly women and youth - the project was developed to contribute to inclusive growth during the economic resurgence after the pandemic146. As highlighted in the project document, most informal MSMEs are led by the working poor, women, youth and other marginalized and vulnerable groups who need to be integrated into an inclusive economic resurgence after COVID-19 and who are at most risk of falling into poverty and being left further behind. The initial project document included a brief section on the assessment of vulnerable groups such as the women traders involved in cross-border trade, and initiatives benefiting the vulnerable groups, especially women and youth. The second phase proposed a coherent approach towards MSMEs, with the purpose of reaching out to the most affected target groups, including women and informal workers. A specific section on mainstreaming gender equality and human rights aspects including social protection was developed to outline the project approach, and intended to ensure a gender perspective, LNOB aspects and HRBA would be integrated into the activities of different clusters across preparedness, response, and recovery stages. The project document147 further provided a brief assessment of the gender issues and made reference to youth and vulnerable groups under each cluster and stated that the project interventions would address issues in an inclusive manner leaving no one behind. The 3rd phase project document also referred to the project approach of reaching out to the most affected target groups, including women and informal workers148. Specific activities integrating gender and youth were also covered including capacity- building workshops for national government officials and informal MSME entrepreneurs, especially youth and women, an entrepreneurship training workshop to assist entrepreneurs from vulnerable backgrounds and to deliver support to MSMEs in the region, placing its efforts on supporting the poorest, marginalized and women-led micro and SMEs. The result frameworks included specific outputs mainly benefiting women entrepreneurs such as training initiatives for cross-border traders (almost all female traders) and outcome- level indicators covering gender-disaggregated data as well as COVID-19 impact assessments. However, the project document covering the initial phase only mentions persons with disabilities in relation to the project’s links to the SDG targets 8.5 and does not specifically outline disability-inclusive responses. Furthermore, some of the outcome indicators include gender-disaggregated data but do not propose effective data collection methods to capture other vulnerable groups and vulnerabilities or gender dynamics beyond the limited quantitative data.

119. In addition to the key guidelines provided by the UN Secretariat, it was unclear to what extent internal procedures and programming guidance for each UN partner on gender mainstreaming, gender equality or LNOB played a role in the integration of all aspects in the design and implementation of the Surge project. Although the overall project was designed with a strong alignment to the UN guiding documents on HRBA and gender, the integration of these cross-cutting components in the implementation within each UN partner’s activities was less consistent. As an informant from one of the implementing UN entities stated, “gender mainstreaming and reducing inequality are part of their [the staff’s] DNA in all they do.” On the other hand, another claimed that due to many partners involved and the urgency of the situation, internal clearance processes regarding gender aspects for DA projects were not necessarily followed for the Surge project. Also, it was implied by yet another informant that with additional resources (i.e. human, financial, and time) then gender responsiveness, HRBA, and LNOB could have been better integrated into the implementation of the project components.

120. There was limited evidence available of systematic sharing of experiences, capacity, or lessons learned related to the integration of the cross-cutting themes, which might have enhanced leveraging synergies among the UN partner organizations or with other relevant agencies, such as UN Women. One example of a good practice was in the Republic of Moldova where UNECE worked closely with the UN Country team and UN Women contributed to the development of a gender-sensitive roadmap for the government based on a study on MSMEs conducted by the Surge project. One of the survey respondents further

145 UN. COVID-19 Response. July 2020. Global Initiative towards post-Covid-19 MSME sector: MSME Surge Project. Phase II Project Proposal. 146 UNCTAD. 2020. Concept note: Global Initiative towards post-Covid-19 MSME sector Phase I. 147 UN. COVID-19 Response. July 2020. Global Initiative towards post-Covid-19 MSME sector: MSME Surge Project. Phase II Project Proposal. 148 UN. COVID-19 Response. February 2021. Global Initiative towards post-Covid-19 MSME sector: MSME Surge Project. Phase III Project Proposal.

66

emphasized that the project was largely complementary to the work of a few agencies - such as UNDP, UNCTAD and UN Women - and helpful in the process of the CCA analysis and gender assessment. UN Women was also involved in supporting a regional conference on women in the Latin American region. Collaboration with Empretec enabled the development of a training programme for people with low literacy and the Farming-as-a-Business training, facilitating participation and empowerment of the vulnerable groups in the sector. Reflecting on the available documentation and evidence, the collaboration and sharing of experiences among project partners or other UN organizations regarding the integration of gender, HRBA and LNOB perspectives were limited to some cases as outlined above and not systematic across the project clusters.

121. The level of integration of these cross-cutting themes by the main counterparts depended on the counterpart’s understanding, willingness, and priorities. The UN partner organizations worked with the governments, including corresponding ministries and officials, as their main counterparts. One informant stated that as a DA initiative, their main counterparts were governments and they looked at how governments shifted their policies to support women in SMEs. Under cluster 5, one of the governments was not interested in women's studies, mentioning that they wanted everyone to benefit without discriminating against men or women-led enterprises. It was also challenging to receive inputs in places where there was no institutional structure dealing with gender issues or women’s empowerment such as in Cluster 5 or to ensure women’s participation, as it was reported that government counterparts were responsible for selecting participants for the Latin American and the Asia and the Pacific regions. The Bangladesh Start-up Ecosystem Assessment Report integrated gender aspects but the women participation at the follow-up panel was low. In Brazil, as reported by an informant, there was a presence of women in the webinars but no gender parity149. Even if there was a supportive government in Moldova where a roadmap for MSMEs covering gender aspects was developed, new priorities, the crisis in the region and limited funding were cited as reasons why there was very limited follow-up to the government roadmap or recommended actions in the project study supported by the Surge project.

122. In Gambia, however, the government considered youth and women as the main target beneficiaries, given that they made up the largest portion of the informal MSMEs sector in the country and looked at how to revitalize this sector and address their needs. As another example, in Malaysia, as a local counterpart, the National Association of Women Entrepreneurs of Malaysia (NAWEM) helped support women entrepreneurships and ensure women participation through engaging in the webinars. Furthermore, the Cross-border Trade Associations, together with the governments, were involved in selecting the participants, who were almost all women as the activities were targeting women as main stakeholders, for the training programs organized as part of the UNCTAD component on Women and Cross-border Trade, including in Malawi and Zambia. One informant suggested that a key lesson learned for the success of the overall cross- border trade and gender initiative was to collaborate closely with the government authorities and ensure their buy-in and support for the activities.

123. Several needs and impact assessments integrated gender and human rights impact of the pandemic among MSMEs to varying degrees. Under different clusters, several assessments of the impact of COVID-19 on MSMEs particularly focusing on gender aspects were conducted to inform design of project interventions including policy recommendations aiming at leaving no one behind. UN DESA conducted a study on the impact of COVID-19 among MSMEs in Kenya and their resilience mechanisms. The assessment included youth and women organizations, focused on the impact of the pandemic on women and youth-led entrepreneurs and provided specific recommendations for vulnerable groups like the elderly, widowed and orphans in Kenya. UN ESCAP Bangladesh Startup Ecosystem Assessment Report150 included a detailed gender analysis and highlighted the importance of integration of gender and inclusivity angles in the funding opportunities to support the ecosystem by bringing in women, people with disabilities, and people from underrepresented and marginalized groups. UN ECE rapid impact assessments on MSMEs were undertaken in the region with a view of informing gender-responsive trade policies. The assessment reports covered gender gaps, gender- disaggregated data, and strategic priorities to achieve women equality including gender-responsive trade policies. 151 UN ECA supported one region and 11 countries to assess the impacts of the recent crises on the MSMEs of its member states in Southern Africa. Almost all the reports cover gender and/or youth aspects

149 The participants list of the region does not have gender disaggregated data to confirm the informant´s statement. 150 Bangladesh startup ecosystem assessment report | ESCAP (unescap.org) 151 UNECE. The impact of COVID-19 on the trade and business development prospects of female-owned enterprises in the Republic in Armenia. 2021; and The impact of COVID-19 on the trade and business development prospects of female-owned enterprises in the Republic in Armenia. Available at: https://unece.org/trade/studies-regulatory-and- procedural-barriers-trade

67

while the study conducted in Eswatini only indicated that the gender details in terms of ownership were not provided by the state. Furthermore, ECLAC reviewed policies to mitigate the impact of the COVID-19 pandemic on SMEs in nine Latin American countries152 and identified government measures with special emphasis on employment opportunities for women, people with disabilities and youth. The global report of the UNCTAD on COVID-19 Pandemic Impact on Micro, Small and Medium Sized Enterprises: Market Access Challenges and Competition Policy identified gender inequalities, provided disaggregated data for gender, age, minorities, persons with disabilities, and recommendations for entrepreneurs and MSMEs from vulnerable backgrounds particularly affected by the digital divide. The national studies on analysis of the impact of COVID-19 on MSMEs in South Africa and Thailand covered gender issues and gender aspects and proposed recommendations for women, youth and vulnerable groups. The national study on competition and market access policies in the resurgence of MSMEs in post-pandemic in Brazil, however, has no reference to gender, human rights or LNOB aspects.

124. The majority of assessments were gender responsive and primarily focused on gender aspects and on the impact of the pandemic on women-led MSMEs. While several assessments included youth aspects, only a few studies and data analyses covered people with disabilities. When asked, two informants expressed an awareness that other vulnerable and marginalized groups were more rarely covered, though they did not explain why. Despite this limitation, the studies were well received and helped identify the needs of those vulnerable groups. When the survey respondents were asked if the project activities and the content reflected the perspectives of women, youth, people with disabilities or other vulnerable groups, the majority (68%) responded positively. The work dedicated to profiling women-led MSMEs was found insightful to develop legal amendments supportive of women employability, according to one survey respondent. Another respondent mentioned that the research on women’s entrepreneurship was considered helpful for policymakers and ministerial staff in Member States to improve their understanding of how support given to women in business pays off.

125. The degree of integration of gender, human rights and LNOB aspects and implementation of assessment recommendations varied across clusters. In a number of cases, documentation was lacking about how those assessments or policy documents were followed up and to what extent the issues pertaining to vulnerable groups were addressed. Even if the assessments were conducted to identify needs and actions were proposed to address those needs of the vulnerable, there were cases where the evidence showed implementation of the actions was lacking or limited. The Project readiness report on the development of a digital platform on technology and innovation by SMEs in Southern Africa required that users and partners consider women, youth and environmental issues. However, the output report of the digital platform did not cover engagement with women, persons with disabilities or other vulnerable groups or aspects. While the Bangladesh Startup Ecosystem Assessment Report153 included a detailed gender analysis, one of the informants stated that “there was a section on gender, that´s all.” There was no systematic follow-up on how the project-funded studies were utilized. Another informant stated that gender mainstreaming is often on paper and not in reality.

126. On the other hand, in Moldova, the recommendations based on the gender-responsive assessment were translated into a roadmap, in collaboration with the government and UNCT. The government stakeholders considered the study as a big input. However, as one informant described, only some of the recommendations were able to be followed up, such as cross-border transit physical capacity and e- commerce funded by USAID, due to the crisis in the region and limited funding opportunities, among other challenges. In Kenya, the rapid impact assessment findings on women and youth-led entrepreneurs informed the planning of another initiative integrating gender and youth aspects with funding from a project. The project team worked closely with the government and linked their funding streams to increase the overall impact. As informed, none of the proposed activities were stand-alone, but rather aligned with the results chain – cross- cutting vertical and horizontal. Entrepreneurship policy review informed the new MSMEs strategy in Uganda. For the first time, with the support from the Surge project and building on the existing partnership with UNCTAD, Uganda included migrants and refugees in the high-level strategic plan, in addition to women and youth as vulnerable groups, according to an informant. The strategy serves as a framework to mobilize resources to realize different pieces. Another informant described that recently the government launched a strategy to help refugee communities.

152 Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, Mexico, Panama, and Uruguay 153 Bangladesh startup ecosystem assessment report | ESCAP (unescap.org)

68

127. Building on the “Informal Cross-border Trade for the Empowerment of Women, Economic Development, and Regional Integration in Southern and Eastern Africa” project of UNCTAD’s Trade, Gender and Development Programme, the capacity-building activities were designed based on the existing gender responsive programme which was then tailored to address emerging needs of women traders during the pandemic in Botswana, Kenya, Malawi, the United Republic of Tanzania, and Zambia. Given the urgency and time limitation, an informant said this component was adapted and implemented very quickly by including COVID-19-related considerations. As highlighted by another informant, it was relatively easy because of the existing relationship with the stakeholders. The training content was revised to include new trade regulations and address needs of different vulnerable groups to enhance knowledge of their rights and obligations, regional trade rules, and on the formalization and registration of business. In partnership with Empretec, training contents were tailored both for participants with low literacy and high literacy levels, enabling engagement with the most vulnerable communities. Gender considerations specific to women traders including harassment were mainstreamed into the activities. Beyond the training activities, the initiative brought together the border officials and authorities dealing with immigration and customs to raise the issue of harassment against women traders. In addition to raising the gender issues, as highlighted above (para 96), the project could further enhance its support to government capacities in cross-border trade by involving government officials, including border officials, in the training courses, as trainees, or having a dedicated training session for them. The majority of the participants who responded to the workshop surveys found the workshops useful and relevant for their businesses. The policymakers and other stakeholders who attended the regional policy dialogue on women and cross-border trade considered the workshops successful at providing a forum to discuss the challenges faced by women informal and small-scale cross-border traders, and formulate practical solutions to improve their situation and enhance the contribution of cross-border trade to economic development and regional cooperation (Box 2).

Box 2: UNCTAD Cross-border Trade and Gender Initiative.

128. Another project initiative primarily focusing on gender aspects was the ESCWA´s Women Empowerment for Technology and Entrepreneurship (AWETE)154 programme which was launched through the DEPAR platform to address the national challenges hindering women empowerment in technology and

154 https://www.unescwa.org/news/boosting-women%E2%80%99s-empowerment-technology-and-entrepreneurship https://depar.unescwa.org/regional/awete

This initiative is a good example of integrating gender responsiveness in the Surge project interventions in extremely vulnerable communities facing additional challenges due to the impacts of the pandemic. Building on the pre-existing programming, the Trade and Gender initiative was adapted quickly to respond to urgent and emerging needs of the cross-border traders, primarily women.

The cross-border trade and gender component of the Surge project replicated the training activities for informal and small-scale cross-border traders of the previous UNCTAD initiative “Informal Cross-border Trade for the Empowerment of Women, Economic Development, and Regional Integration in Southern and Eastern Africa¨ which was implemented from 2016 to 2019. The training workshops were based on the tailored Empretec programme by adapting an existing methodology to meet emerging needs of women traders during the pandemic. The expected results of this initiative included increased awareness of trade rules and procedures, improved knowledge of their rights and obligations, enhanced use of official border posts and trade through formal routes, ability to adapt the businesses and developed strategies, and awareness of COVID-19 measures. Under this initiative:

✓ Almost 300 cross-border traders participated in the workshops, almost all were women.

✓ 9 in-person workshops (6 days each) with two components organized: training on cross-border trade rules and development of entrepreneurial skills.

✓ Botswana, Kenya, Malawi, the United Republic of Tanzania, and Zambia covered.

✓ Project partners included relevant ministries, trade authorities and Cross-border Trade Associations of the respective countries.

✓ Over 90% of the participants found the workshops useful and relevant.

✓ 95% of the participants of the regional policy dialogue considered the workshops successful.

✓ Five traders´ guides ¨Women in informal cross-border trade: A small-scale trader's guide to trade rules and procedures¨ tailored for Botswana, Kenya, Malawi, Zambia and Tanzania.

69

entrepreneurship across the Arab region. AWETE provides a community space for female entrepreneurs, innovators, and professionals to share knowledge and build new partnerships. Under this initiative, the project developed women empowerment ecosystem maps for 22 Arab counties155 and organized a series of five roundtable discussions in the region, bringing key experts and stakeholders advocating for women´s rights.

129. The majority of the survey respondents had the perception that the project activities were accessible for women, youth, people with disabilities, or other vulnerable groups, although the document review identified limited data in project reporting and other relevant sources referring both to cases with varying degree of gender parity and others with no data on gender or other aspects. The ability to assess comprehensively the participation of vulnerable groups is limited to available documentation that mostly covers gender and no other vulnerabilities. In some cases, the data indicated that the events successfully facilitated the engagement of women, ensuring gender parity. The online Empretec TOTs organized in French, English and Spanish for low-literate people reached out to around 100 people, 49% women. Whereas the workshops/courses/webinars on circular economy reached out to 600 participants, only one-third of them women. In other cases, the information on gender or other aspects is not available to make a comprehensive assessment covering all project interventions. However, the majority of survey respondents (70%) believed that the project activities were accessible for women, youth, people with disabilities, or vulnerable groups (Figure 20). The respondents also mentioned that the project included or supported women, youth, people with disabilities or other vulnerable groups through empowering vulnerable groups especially women and supporting their access to funds.

Figure 20: Perceived accessibility of project activities for women, youth, people with disabilities or other vulnerable groups (n=126 respondents).

Source: Independent evaluation survey, 2023.

130. Adapting to COVID-19 restrictions enabled the engagement of the vulnerable groups to some degree, particularly women entrepreneurs in the project activities while, in some cases, their participation was hindered due to the lack of access to technology, selection process of beneficiaries, location of events, or the pandemic impacts and restrictions that made it difficult to bring people together. Going virtual equalized those who were able to participate in project activities and built bridges across countries and regions. However, participation was limited to only those with physical and financial access to the necessary technology, and ensuring participation was challenging as there were too many online initiatives from a range of development partners. Given the pandemic restrictions, most of the interventions particularly in the early stages of the pandemic including workshops and training programs were organized online, and online portals enabled free access to everyone with access. As reported by one of the informants, NGOs could follow some of Empretec´s work online, particularly vulnerable groups and women on the ground. Empretec´s TOT workshops were

155 https://depar.unescwa.org/regional/awete/ecosystemmaps

33%

37%

21%

3% 2% 0%

4%

Project activities were accessible for women, youth, people with disabilities, or other vulnerable groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

70

revised, and short videos were developed to better serve the vulnerable groups including rural entrepreneurs, farmers and low-literacy entrepreneurs156. One informant believed that having everything online helped people from diverse populations to link across groups and locations in the country. The final report of the project also highlighted that the skills development was facilitated thanks to the online workshops increasing delivery efficiency and accessibility with specific attention to vulnerable groups and people with low literacy levels157.  In addition, the online regional policy dialogues on the role of competition policy in supporting MSME’s economic recovery in the post-COVID-19 crisis were new, in the sense that they were online and open to all stakeholders from governments to MSMEs covering several countries at once. Although the online platforms provided unique opportunities and, in some cases, the only way to reach out to the vulnerable communities, they hindered the engagement of those marginalized groups with no financial and technological access.

131. Additionally, cases were also found of implementation modalities limiting participation of some vulnerable groups. For example, selections of cross-border trade workshop participants, almost all women traders, was the responsibility of the governments in collaboration with the Cross-border Trade Associations. In the Latin American region, it was reported that there were challenges to ensuring the participation of women or other vulnerable groups since it was the governments that conducted the selection of participants. In addition to the selection processes, it was reported that criteria in a few training programmes limited the participation of the most vulnerable. . When translation was provided in another aspect of the Surge project, it was indicated that the translation was difficult to understand, which limited the effectiveness of reaching out to the most vulnerable. In most cases, there was no reporting on participants’ self-identifying with a range of vulnerable groups, such as people with disabilities, making it difficult to assess the extent of engagement in the Surge project by various vulnerable groups. It was indicated by an informant that they had no way to measure how many and with which vulnerable group the participants identified. The locations of some of the events, such as holding them only in the capital city, were also considered as inhibiting participation from diverse groups and regions.

132. A few impact assessments point towards positive outcomes particularly for women. Overall, 64% of the survey respondents felt that the project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups (Figure 21). The survey respondents revealed that the relevant areas on which the project has contributed to the resurgence of MSMEs, policy or economic improvements, or advancing sustainable development included improved inclusion of vulnerable groups with a special reference to women entrepreneurs, funding opportunities for women, addressing the needs and supporting women-led MSMEs. Capitalizing on the knowledge or skills acquired through the project, some survey respondents reported that they facilitated priority policies for women-owned and women-responsive businesses, development of female entrepreneurships, and access to credit for women entrepreneurs and women capacity-building.

Figure 21: Perceived contribution of the project to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups (n=105 respondents).

156 Under the Surge project, Empretec offered in-person, hybrid as well as online training activities to respond to the different needs and COVID-19 restrictions of various contexts. 157 UNCTAD. September 2022. Final Report: Global Initiative towards post-Covid-19 resurgence of the MSME sector.

71

Source: Independent evaluation survey, 2023.

133. The UNCTAD component on Women and Cross-border Trade delivered solid results in terms of enhancing cross-border traders’ capacity, mostly women. The impact assessment on the cross-border trade in the pre- and post-pandemic environment covering 2019-2021158 indicated that the training workshops that were attended by almost all women helped women entrepreneurs become more effective cross-border traders. The impact assessment further outlines how the initiative helped enhance the knowledge and abilities of the cross-border traders to benefit from greater business resilience (Please see the effectiveness section of this report for assessment findings). The initiative also contributed to the formalization of the businesses of the cross-border traders. As reported by an informant, female traders were informal but now most of the training participants have registered their businesses.

134. The impact assessment on the Empretec workshops (ETWs) which assessed the economic impact of the ETWs on the participant entrepreneurs in selected project countries indicated positive results for women entrepreneurs, only for those areas where gender disaggregated data was provided159. Based on the findings, the number of women in business after the ETW increased by 15% in Benin, 7% in Ghana and 9% in Nigeria. 160 The F/M TEA ratio (Female and male, Total Early Stage) also demonstrated positive outcomes in Benin (15% higher161), Cameroon (13% higher), Gambia (57% higher) and Zimbabwe (20% higher).162

The project was also effective in facilitating women and youth-led MSME registration and formalization. UNCTAD helped facilitate e-registration and formalization of MSMEs in El Salvador, Benin, Cameroon and Mali through an e-registration portal. In Benin, the portal increased business registration by 91%, with 84% increase in women-led MSMEs and 181% increase in youth-led MSMEs. A similar trend was observed in Mali where business registration increased by 40% including 49% increase in women-led MSMEs, and 110% in youth-led MSMEs.

135. In contrast to survey respondents’ perceptions, the available data on inclusion of vulnerable groups and vulnerabilities is limited and hinders the evaluation’s ability to assess the degree to which gender, HR and LNOB aspects were integrated into the Surge project. Of the data that is available, it is mostly for the gender-

158 UNCTAD (2022). Cross-border Trade in the Pre- and Post-Pandemic Environment Evidence from Malawi, the United Republic of Tanzania and Zambia. Geneva. 159 The assessment provided limited gender disaggregated data and did not cover any other cross-cutting dimensions. 160 UNCTAD. MSME Surge Project. Empretec: Impact Assessment Survey Report. 2023. 161 In Benin, for instance, the ratio 0,8/1 referring to 8 MSME businesses opened by women to every 10 MSME businesses started by men – at Empretec the ratio is 0,92:1 (15% larger). 162 The F/M TEA (Total Early Stage) (Female / Male TEA ratio, applying only to early-stage businesses) is an indicator of how Entrepreneurship may be used as a tool for equity and women empowerment, especially in traditional male dominated societies as usually verifiable in Africa.

32%

32%

28%

2%

3% 0%

3%

The Surge project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

72

disaggregated information for the project beneficiaries with no or limited information on age, persons with disabilities or other characteristics. As reported by one informant, the initiative focused on female youth but there was no record on how many female youth entrepreneurs were reached. In some cases, the gender- disaggregated data was only available for those who responded to the event´s survey and not for all the participants, making it difficult to assess the gender parity163. The data collection tools were mostly not designed to collect information necessary to assess the cross-cutting dimensions. For example, one of the workshop evaluation forms used under cluster 1 did not have any questions to identify issues related to gender, age, disabilities, and other vulnerabilities of specific groups. This is also observed across clusters covering various project initiatives.

136. Furthermore, the project results framework mainly captured quantitative gender-disaggregated data, but did not capture the degree of the project´s contributions to different vulnerable groups and vulnerabilities. There was also limited follow-up to assess the impact of the training or other initiatives from gender and human rights perspectives. As one informant observed, follow-up monitoring should have been included to assess the impact of the training programs and nobody was doing it.

137. One of the exceptions was the Impact Assessment on the Empretec Workshops (ETWs) covering 6 African countries between 2020 and 2022, namely Benin, Cameroon, Ghana, Nigeria, The Gambia and Zimbabwe. The assessment on ETWs did not capture all the cross-cutting aspects comprehensively but provided gender-disaggregated data for the number of women in business after ETWs. Another assessment was conducted for the capacity-building activities organized by the UNCTAD Trade and Women initiative in Malawi, the United Republic of Tanzania and Zambia aimed at small-scale and informal cross-border traders, mainly women. The impact assessment covered the capacity-building activities carried out from 2019-2021 at selected border posts, including previous analytical work carried out by UNCTAD´s Trade, Gender and Development Programme since 2016 under a 10th tranche DA project164. The compilation of success stories of women entrepreneurs who received Empretec training - such as the story of how four women in Malaysia built success during COVID-19165- also helped provide anecdotal evidence on the project’s contributions to women empowerment and inclusive growth.

163 The number of participants of the UNCTAD Webinar Series on SDG Reporting by companies, including MSME—The adaption of “Guidance on core indicators for entity reporting on contribution towards implementation of the Sustainable Development Goals (GCI)” in China was 150 while the gender-disaggregated data is only available for 26 survey respondents. 164 The Development Account project “Informal Cross-border Trade for the Empowerment of Women, Economic Development, and Regional Integration in Southern and Eastern Africa”, implemented during 2016-2019. 165 https://unctad.org/news/how-four-women-malaysia-built-success-during-covid-19

73

6. Conclusions

138. Based on an analysis of the findings, the evaluation formulates the following conclusions.

Conclusions

Relevance

The project responded to the needs of Governments and MSMEs. Interventions contributed to and were informed by needs and impact assessments. Recipients of project interventions were often involved in or contributed to the design and delivery of project activities and outputs. On various occasions, project outputs were tailored to the needs of target users and beneficiaries, being countries or vulnerable groups. The relevance of the interventions was somewhat mitigated by the mandates and capacities of the implementing UN entities. The latter are equipped for supporting medium- and long- term policy changes rather than immediate crisis responses. Other challenges included the restrictive Covid-19 measures and the short time frame for implementation of the project. Nonetheless, on the whole, the implementing UN entities exercised adaptive management to ensure high relevance of their interventions. The phased approach, in particular, provided the opportunity to introduce new activities not foreseen at an earlier stage.

Coherence

The project was anchored in the mandates and comparative advantages of the implementing UN entities. Selected interventions followed previous work and specific requests from member States and were often coordinated with or complementary to existing activities. At conception, the target of the project was to address specific constraints faced by MSMEs. It was designed around the Entrepreneurship Policy Framework to show the complementarities across interventions that each implementing UN entity would deliver in response to regional priorities and capacities. Some collaborations between implementing UN entities were pursued and complementarities established, but on a limited level. A few synergies in the form of joint outputs were integrated in the 3rd phase of the project. By design, technical task forces created through the project were expected to bridge the implementing UN entities. However, this proved difficult to launch and sustain, owing to several constraints, including a highly ambitious objective given the global crisis and competing priorities within each agency. Nevertheless, the coordination and complementarities established by the project among the implementing entities contributed to expanded dissemination and outreach of outputs, avoided overlaps between them, and allowed significant knowledge exchanges aligned with and supportive of a “One UN” system. On the other hand, collaboration of the implementing UN entities with other UN organisations, including UNCTs, was limited. Despite this shortcoming, at national level, target recipients of project outputs found the interventions complementary to those of other UN agencies.

Efficiency

Project coordination was complex and demanding, involving seven UN entities and the target delivery of around 100 outputs, of which some were composed of many activities. Nevertheless, the coordination of the project was found to be efficient overall given the circumstances and resources available. The coordination of the design phase of the response was actively supported by the DA-PMT. Coordination of project implementation was led by UNCTAD in collaboration with a Steering Committee that met on a regular basis, most often bi-monthly. UNCTAD’s Budget and Project Finance Section (BPFS) produced the project’s financial monitoring table every month, which included the expenditure data of UNCTAD as well as the other implementing entities. A monitoring dashboard was created by UNCTAD to facilitate the tracking of UNCTAD’s project delivery. Data from some of the other implementing UN entities was added, when provided, at two reporting points during the lifetime of the project.

The project would have benefited from the allocation of more resources towards the coordination and monitoring of overall implementation. Across the implementing UN entities, the project was considered complex and hard to follow outside of the interventions under their direct control. Some staff perceived that there were too many participants in the coordination meetings, that the project lacked a Theory of Change, and that it was primarily a compilation of activities rather than a cohesive and mutually reinforcing bundle of interventions supporting in depth any given country. Some staff in the UN Regional Commissions questioned the efficiency cost of a global or inter-regional response versus regional responses. The project was implemented in three phases, which were supportive of adaptive management and perceived by staff as the most rational approach considering the circumstances. However, it was also pointed out that clearer visibility

74

from the onset on the funding available for future phases would have facilitated planning and the search for synergies.

Effectiveness

The Development Account programme, as a funding modality, was quick to respond and adapt to the COVID- 19 crisis. The prompt availability of resources and rapid orientation toward a global response contributed to the rapid launch of a wide range of interventions. However, questions remain as to its adequacy for addressing the immediate needs of a crisis in comparison with other UN agencies and mechanisms with a mandate for humanitarian response. The governance of this project, with DA-PMT’s oversight, the management structure with UNCTAD in the lead coordinating role, and each implementing entity leading the operational implementation at the regional level, ensured the global and regional scope of project delivery. At the operational level, implementing entities relied more on partnerships with local and regional institutions rather than coordination within the larger project management structure. The project governance and management mechanisms evolved in tandem with the COVID-19 crisis and the project’s emerging needs. The result was a high delivery rate but with limited inter-agency collaboration in the process. Yet, despite limited collaboration, this independent evaluation identified that the project was effective in producing identifiable results at the outcome level.

Overall, the project delivered 85% of the planned outputs. Phase 3, particularly, delivered the highest number of outputs but had the lowest outcome effectiveness rate, mostly because of budget cuts and some outputs still in progress at the time of this evaluation. There is evidence of the effective improvement of national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship and MSME promotion (outcome 1.A) [such as the adoption of a revised entrepreneurship strategy by South Africa, Uganda and Seychelles]; improved resilience and competitiveness of MSMEs (outcome 1.B) [for example with 64.5% of participants opening a new business after attending Empretec-based training]; facilitated MSME registration and formalization through e-platforms (outcome 2) in El Salvador, Benin, Cameroon and Mali; improved access to finance, including upskilling in financial literacy (outcome 3) [for example with 97% of the MSMEs in Latin America that attended trainings reporting improved financial literacy in accounting and reporting and also improved capacity to manage financial resources]; increased MSME access to innovation and technology (outcome 4) [including through capacity development on green technologies for SMEs in Southern Africa]; and increased access to local, regional and international markets through digitalization and non-tariff measures (outcome 5) [for instance with case studies and on competition in Thailand, South Africa and Brazil, followed by webinars and the creation of an online course on SMEs and competition policy, primarily for government officials].

In relation to the project indicators, the assessment of their effectiveness was based on limited information, proxies, and expert judgement since they often lacked SMART criteria. It was possible to infer that 85% of them showed some level of progress, either through objective measures (e.g., sales increase) or proxies (e.g., post-training satisfaction assessments and survey responses). The high effectiveness results can be partly attributed to the conceptual framework provided by the EPF, as well as the cluster-based approach with each cluster addressing specific regional constraints and responding to demand from member States. In addition to clustering about 100 outputs from seven implementing entities into 5+1 outcomes, the EPF provided a coherent thread to relate outputs that cut across outcomes. Further analysis would be needed, however, to clearly identify which of those outputs are the most suitable or adaptable for a crisis response.

The project improved the capacity of policymakers in designing and implementing policies supportive of MSMEs, especially in terms of contributing to a country’s government responses, country-specific studies, technical assistance, training courses, and the development of digital tools and regional interventions. This improved capacity is reflected in beneficiaries’ reports of how they have been considering the needs of MSMEs, including those led by women and youth, in their daily work. These results could become more transformative and sustainable with more institutional support to networks of policymakers and communities of practice for mutual and regional learning on supportive policies for MSMEs, as well as by mainstreaming gender and Leaving No One Behind (LNOB) approaches in government officials' capacity-building.

Sustainability

The project developed approaches, tools and capacities that are being transferred at multiple levels. At the policy level, the addition of more functionalities to e-government tools, expanding training courses to more countries, and the establishment of agreements to implement the project recommendations are all strong

75

evidence of the sustainability of the project's achievements. At the behavioural level, beneficiaries are applying new knowledge and skills to their daily work, be it by including MSMEs' issues in policy making or by improving MSMEs' management, performance, outputs and, by extension, resilience. Challenges to this transfer of knowledge relate to the lack of financial resources, need for follow-up mentoring, and lack of a favourable business environment.

Gender, human rights, and disability

The overall project design was well aligned with the two key UN documents that provide clear direction for the integration of a human rights-based approach, which include addressing the aspect of gender equality and Leaving No One Behind. Although these aspects were well-articulated in the project documents, strong follow-through and documentation of their implementation was limited to a few project components. For those components that did include these aspects, gender was more commonly integrated and youth to a lesser degree. There was very little evidence of the outreach, participation, or impact of project activities for people with disabilities or other vulnerable groups. The UNCTAD component on Women and Cross-Border Trade targeting five countries in Africa was a good practice in how to design and implement this type of intervention with a strong reflection of the needs of some of the most vulnerable women across border areas.

76

7. Recommendations 132. Based on the evaluation findings and conclusions, the following recommendations could be considered by the implementing UN entities and the DA-PMT.

139. Recommendation 1: (i) UNCTAD should further leverage the experience gained through the project to map out how components of the Entrepreneurship Policy Framework relate to and can be best positioned to support MSMEs in their recovery from different types of crises and (ii) DESA and the Regional Commissions should add their analysis of how their work can contribute towards the objective. UNCTAD, DESA and the Regional Commissions could build on the project to map their areas of intervention in support of the MSME sector, as well as capacities and knowledge on the implementation of the EPF components, including in a crisis context, and opportunities for broadening the uptake of EPF components at the regional level based on national needs. UNCTAD, DESA and the Regional Commissions could further consider identifying areas of joint interventions that would trigger complementarities and synergies between the agencies. This could involve collaborating on the development of capacities of all national and sub-national actors, i.e. Governments, MSMEs, other partners (e.g. Chamber of Commerce, Business Incubators, etc.) in line with United Nations Sustainable Development Cooperation Frameworks (SDCF) in respective countries; strengthening or contributing to facilitate access to finance for the MSMEs (including seed money, grants, access to credit, etc.); improving the coordination of MSMEs related policies across ministries; increasing interventions at the local level, such as by supporting NGOs or MSMEs outside of the main cities. The application of a human rights-based approach, gender responsiveness and inclusion of other vulnerable groups (LNOB) should be ensured.

140. Recommendation 2: UNCTAD should continue building on the momentum generated by the project to continue fostering knowledge exchanges and promoting the Entrepreneurship Policy Framework. UNCTAD, DESA and the Regional Commissions should identify means to more meaningfully continue to share good practices and lessons learned on the demand-driven support they provide to the MSME sector, including on areas such as green/circular economy; innovation policies and ecosystems; MSME formalization and strengthening MSME capacities for the empowerment and leadership of women, youth and groups in vulnerable situations. UNCTAD and the Regional Commissions could also propose organizing regional events and/or a global conference to promote entrepreneurship policy and further advance the goals and impact of the Surge project. Furthermore, learnings from the experiences with the Surge about external partnerships should encourage UNCTAD to promote the EPF and entrepreneurship development to other UN agencies also engaged in this area (e.g. ILO, FAO, UN Women, etc.), to development banks, or to related initiatives such as the recent Global Accelerator on Jobs and Social Protection. UNCTAD could also consider joining and supporting events that promote the achievement of SDG 8.3 as an avenue to further promote the EPF.

141. Recommendation 3: The DA-PMT should develop a clear framework for assessing the costs and benefits of implementing a global or inter-regional project versus regional projects. As a global crisis can affect regions and countries differently, global or joint projects should be developed only when there are clear benefits of joint implementation. Thus, a global or inter-regional vs. regional response would not be a priori decision, but a result of a clear assessment. A specific guideline or framework should be developed for this. Some of the assessment criteria could include the need or demand from member States for an integrated response; the range of common versus entity-specific activities and added value of complementary interventions (i.e. in terms of geographical coverage, reach of the target population, capacity, and/or coverage of multiple reinforcing technical areas, or networks and partnerships); capability to execute joint interventions (e.g. in terms of time, resources, logistics, and flexibility to pivot interventions without bureaucratic procedures); coordination costs (which increase with the number of participating UN entities); project inception modalities and governance and knowledge management requirements to facilitate collaboration and knowledge sharing; scaling and sustainability plans; expected results of spreading resources versus concentrating on fewer countries; and so forth. The time taken to conduct such assessments will increase project coherence and effectiveness and facilitate the identification of the financial and human resources needed for project implementation. The network of DA Focal Points is a key existing asset for this assessment.

77

142. Recommendation 4: Implementing UN entities should ensure that they have a comprehensive Results Framework for the entire project as well as an adequate monitoring plan, with indicators that are designed to support the ongoing monitoring. The results framework should have one objective and ideally have one outcome per cluster/workstream. The objective should state the intended goal of the project, describe the overall achievement targeted by the project, involving a process of change aimed at meeting the needs of identified beneficiaries, and reflect the overall funding available to the project. Each objective should include reference to the project’s beneficiaries and its substantive focus. The objective should not attempt to explain the ways in which the project intends to achieve the objective (i.e. it should not include the word ‘through’ or describe the internal work of the UN using verbs such as ‘support’, ‘facilitate’ or ‘contribute’). The outcomes (OCs) should describe the changes that are expected to occur as a result of the completion of outputs. The OCs should be achievable within the project’s timeframe and budget, and should be specific enough to be measured by the associated indicators of achievement. The indicators of achievement (IAs) should provide measures for monitoring progress towards achieving the OCs and reporting on them after completion of the project. Every indicator needs to provide clearly defined baselines, units of measurement and targets, detailing the quantity, quality and timing of expected results. The monitoring, evaluation, accountability, and learning (MEAL) system should also be designed to capture HRBA, gender and LNOB aspects. In phased interventions or during project execution, any changes in the logframe if/when pivoting activities should be clearly explained to the wider team.

143. Recommendation 5: Implementing UN entities should ensure that sufficient resources are allocated to project coordination, technical collaboration, and partnership building. The absorption capacity of implementing entities can be challenged by crisis response projects which add to the planned programme of work. This is further compounded by projects that come with an extensive UN partnership and a global scope. Sufficient resources should be dedicated to global coordination and to building global partnerships with strategic stakeholders (e.g., UN organisations engaged in supporting the MSME sector; development banks). Capacities should also be directed to supporting technical collaboration and the staff implementing interventions, including towards synergy or liaison with the UNCTs. When designing the project, implementing UN entities should consider featuring coordination and partnership-building in the Theory of Change or logframe of the project. Tools to support continuous connections and knowledge exchange, and to ensure institutional memory should be part of the response package, such as a project website, SharePoint space for all team members, and a Yammer network or Teams channel. The integration of cross-cutting aspects (HRBA, gender responsiveness, LNOB) also requires expertise with sufficient and dedicated time and resources. Guidance could be development to project managers on how to do this.

78

Annexes

Annex 1: Project’s Theory of Change .....................................................................................................................................

Annex 2: Evaluation matrix.......................................................................................................................................................

Annex 3: Data collection instruments ...................................................................................................................................

Annex 4: List of documents reviewed ...................................................................................................................................

Annex 5: List of individuals interviewed ...............................................................................................................................

Annex 6: Evaluation surveys ....................................................................................................................................................

Annex 7: Evaluation Terms of Reference ..............................................................................................................................

79

Annex 1: Project’s Theory of Change

80

Annex 2: Evaluation matrix

Relevance

Key questions Suggested measures or evidence Suggested sources and methods

● To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID-19?

● Evidence of needs assessments (surveys, reports, minutes from consultations, workshop reports, etc.).

● Country contexts. ● Reference to the Surge project in country,

programmes, or COVID-focused UN evaluations. ● Reference and opinion of UN staff and partners. ● Opinion of staff from the implementing UN entities,

including Project Design Team, Project Steering Committee, and Task Forces.

● Opinion of external partners, including other UN organizations, Governments, other partners and stakeholders.

• Desk review: surveys, minutes from consultations, workshop reports, country assessments, sectoral analysis, SERPs.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

● To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g. COVID-19 Socio-Economic Response Plan)?

● Reference to SERPs in project progress reports and outputs.

● Reference to the Surge project in UN PRODOCS and in country, programmes, or UN/UNCT annual reports.

● Reference to the Surge project in country, programmes, or COVID-focused UN evaluations.

● Opinion of UN staff and external partners.

• Desk review: Progress reports, project outputs, SERPs.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

• Coherence

Key questions Suggested measures or evidence Suggested sources and methods

● To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

● Evidence of collaboration or joint outputs with other UN departments or programmes; evidence of other UN departments or programmes referring to or using MSMEs project’s outputs.

• Desk review: Strategies and workplans of participating UN

81

● Evidence of synergies between implementing UN entities and overlaps avoided; evidence of joint activities.

● Evidence of the project being mainstreamed in the workplans of other departments or programmes within implementing UN entities.

● Evidence of the project being reflected in the annual workplans and time commitments of staff from implementing UN entities.

● Opinion of staff from the implementing UN entities, including Project Design Team, Project Steering Committee, and Task Forces.

● Opinion of external partners, including other UN organizations, Governments, other partners and stakeholders.

● Reported instances of improvements in the coordination of the response to the COVID crisis by implementing UN entities.

● Evidence of adaptive management and adjustments in the course of project implementation.

entities, PRODOCs, progress reports, evaluations.

• Interviews: Staff from participating UN entities, UNCTs.

● To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID- 19 in delivering socio-economic support to Member States?

● Number and types of other UN agencies contributing to project implementation.

● Evidence of partnerships with other UN agencies; evidence of collaboration or joint outputs with other UN departments or programmes; evidence of other UN departments or programmes using outputs from the MSMEs project.

● Evidence of the project being mainstreamed in the workplans or programmes of other UN agencies.

● Opinion of staff from other UN agencies, including UNRCOs and UNCTs.

● Opinion of external partners, including Governments and other stakeholders.

● Reported instances of improvements in the coordination of the response to the COVID crisis by other UN agencies.

● Evidence of adaptive management and adjustments in the course of project implementation.

• Desk review: Strategies and workplans of participating UN entities, PRODOCs, progress reports, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

82

• Efficiency

Key questions Suggested measures or evidence Suggested sources and methods

● How well coordinated was the process for the response among the entities implementing the joint project?

● Evidence of project monitoring meetings; technical meetings of the project task Forces.

● Evidence of joint activities, synergies between participating UN entities, overlaps avoided; project mainstreaming in participating UN entities’ workplans.

● Opinion of UN staff, including Project Design Team, Project Steering Committee, Task Forces, UNRCOs and UNCTs.

● Opinion of external partners, including Governments and other partners and stakeholders.

● Reported instances of improvements in the coordination of the response to the COVID crisis by participating UN entities.

● Time taken to develop and deliver planned outputs; evidence of delays.

● Evidence of adaptive management and adjustments in the course of project implementation.

• Desk review: Meeting minutes, progress reports, workplans, PRODOCS.

• Interviews: Staff from participating UN entities (e.g. Project Design Team, Project Steering Committee, Task Forces), UNCTs, external partners.

• Case study: UNCT staff and partners.

● How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

● Reported impact according to project monitoring meetings; technical meetings of the project task Forces.

● Opinion of UN staff, including Project Design Team, Project Steering Committee, UNRCOs and UNCTs.

● Opinion of external partners, including Governments and other partners and stakeholders.

● Speed of implementation; time taken to develop and deliver planned outputs; evidence of delays.

● Evidence of adaptive management and adjustments in the course of project implementation.

• Desk review: Meeting minutes, progress reports, workplans, PRODOCS.

• Interviews: Staff from participating UN entities (e.g. Project Design Team, Project Steering Committee, Task Forces), UNCTs, external partners.

• Case study: UNCT staff and partners.

• Effectiveness

83

Key questions Suggested measures or evidence Suggested sources and methods

● To what extent did the programme (Development Account) and project governance and management structures and processes enable, or hinder, the effective implementation of the joint project and the achievement of its results?

● Evidence of project monitoring meetings; technical meetings of the project task Forces.

● Programmes and projects monitoring and evaluation reports.

● Opinion of SHS staff, partners, and stakeholders; ● Evaluators’ expert judgement drawing on all evidence

sources.

• Desk review: Meeting minutes, progress reports, workplans, PRODOCS.

• Interviews: Staff from participating UN entities (e.g. Project Design Team, Project Steering Committee, Task Forces), UNCTs, external partners.

● To what extent has the project contributed to the expected outcomes as enunciated in the project document?

● Evidence of project outputs and reference to project’s outputs in policies, SDG related reports, and other national publications on MSMEs and entrepreneurship (e.g., UN, private sector, CSOs/NGOs, etc.).

● Outcome indicators: All; level of achievement of planned outcomes.

● Programmes and projects monitoring and evaluation reports.

● Opinion of UN staff, partners, and stakeholders. ● Evaluators’ expert judgement drawing on all evidence

sources.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations, Empretec centers data and reports, data collected by UNCTAD via its e- regulation and e-registration portal.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

● How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

● Evidence of uptake, use, and influence of project outputs.

● New approaches and policies adopted and capacity and resources to sustain these.

● Reported country level uptake and institutionalisation of new approaches introduced as a result of ILO COVID response.

● Opinion of UN staff, Governments, and other partners and stakeholders.

● Evaluators’ expert judgement drawing on all evidence sources.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

84

● What innovative approaches or tools, if any, did the response use, and what were the outcomes and lessons learned from their application?

● Use made and outputs of innovations such as online or blended modalities at country level, gamification and other interactive techniques.

● Engagement in short term measures outside of normal activities.

● Outcome indicators: IA 4.1., IA 4.2. ● Opinion of UN staff, Governments, and other partners

and stakeholders. ● Evaluators’ expert judgement drawing on all evidence

sources.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

• Sustainability

Key questions Suggested measures or evidence Suggested sources and methods

● What measures were adopted to ensure that outcomes of the response would continue after the project ended?

● Reported contribution made through the project towards impact in each of the outcome areas for intended beneficiaries and in shaping national policies

● Governments and other beneficiaries’ perceptions of impact/likely impact

● Governments and other beneficiaries’ perception of strengthened capacity.

● New approaches and policies adopted and capacity and resources to sustain these.

● Reported country level uptake and institutionalisation of new approaches introduced as a result of ILO COVID response.

● Evidence of new arrangements, partnerships and funding being put in place to support the scaling up of work implemented or given new prominence as a result of the pandemic.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Gender, human rights and disability

Key questions Suggested measures or evidence Suggested sources and methods

85

● To what extent were perspectives on gender equality, protection of human rights and reaching underserved groups integrated into design and implementation of the project? What results can be identified from these actions?

● Proportion of vulnerabilities disaggregated surveys, assessments.

● Evidence of human rights conventions and strategies (e.g. CEDAW, CRC, CESCR, CRPD, etc.) referred in progress monitoring reports.

● Evidence of project outputs mainstreaming HRBA principles and LNOB.

● Extent to which partnership agreements and project activities include specific measures advancing gender equality, inclusion, human rights.

● Evidence of vulnerable groups or their representative organisations involved in project implementation and monitoring.

● Outcome indicators: IA 1.5, IA 1.6, IA 1.7, IA 1.8, IA 2.1, OP 3.1, IA 3.2, IA 5.3; level of achievement of planned outcomes.

● Opinion of UN staff, Governments, and other partners and stakeholders.

● Evaluators’ expert judgement drawing on all evidence sources.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

86

Annex 3: Data collection instruments

Relevance

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

1. To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID- 19?

● To what extent has the project been articulated with countries’ priorities to respond to the COVID- 19 crisis?

● How did you identify the needs for the project activities or products you delivered in target countries?

● Did you specifically assess the needs of women and vulnerable groups? If positive, how?

● Is there any evidence of such needs (e.g. demands from Member States, surveys, minutes from consultations, workshop reports, market assessments, past evaluations)?

● To what extent has the project been articulated with countries’ priorities to respond to the COVID- 19 crisis?

● How were the needs of national beneficiaries identified?

● Did you specifically assess the needs of women and vulnerable groups? If positive, how?

● Who was consulted and what evidence is there (e.g. surveys, minutes from consultations, workshop reports)?

● Did the Surge Project respond to a national or sectoral agenda or priority?

● Were your needs assessed and did they inform project activities? If positive, how?

● To what extent did the training activities, knowledge products, or technical support delivered by the Surge project respond to your learning needs?

● To what extent were the training activities, knowledge products, or technical support delivered by the Surge project relevant to your work?

● To what extent did the training activities, knowledge products, or technical support delivered by the Surge project provide you the opportunity to share your knowledge and skills with other participants?

● Were you overall satisfied with the training activities, knowledge products, or technical support

87

delivered by the Surge project?

2. To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g. COVID-19 Socio- Economic Response Plan)?

● How did the Surge project’s products and activities align with the SERPs?

● To what extent was the project aligned with UN priorities to LNOB by reaching the most vulnerable and the protection of human rights?

● How did the Surge project’s products and activities delivered in your country respond to the SERP?

● To what extent was the project aligned with UN priorities to LNOB by reaching the most vulnerable and the protection of human rights?

● To what extent was the project aligned with UN priorities to LNOB by reaching the most vulnerable and the protection of human rights?

Would be primarily assessed through a desk review of a sample of SERPs.

Coherence

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

3. To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

● What were the other key activities that your department or other departments in the organization carried out to support MSMEs during the period 2020-2022? Which departments implemented those activities? How did you synergize these activities with the Surge project?

● Is there any evidence of joint activities or outputs? Is there any evidence of

● Did you cooperate with different departments from the project UN entities? If positive, did these UN entities synergize their inter- departmental collaboration while working with you?

● What were the good practices? What could have been done differently?

● Did you receive support (from different departments of the implementing agencies or) from several implementing entities?

● If positive, is there any evidence of complementary activities designed to maximize synergies and avoid overlaps? What were the good practices?

● What could have been done differently?

88

complementary activities designed to maximize synergies and avoid overlaps?

4. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non- Secretariat) to COVID- 19 in delivering socio- economic support to Member States?

● Were there any other UN organisations outside of the core project partners that carried out activities to support MSMEs during the Covid crisis (2020- 2022)? Which ones and how did you synergize these activities with the Surge project?

● To what extent did the project support other efforts by UN entities to support Member States intending to mitigate socio-economic impacts of COVID-19?

● Is there any evidence of joint activities or outputs? Is there any evidence of complementary activities designed to maximize synergies and avoid overlaps? What were the good practices?

● What could have been done differently? Are there any UN partners with which complementarities could have been strengthened (e.g. ILO, UNDP)?

● To what extent did the project support other efforts by UN entities to support Member States intending to mitigate socio-economic impacts of COVID-19?

● Among the range of activities that your organization carried out in the country to support MSMEs during the period 2020-2022, did you synergize any of these interventions with the Surge project?

● Is there any evidence of joint activities or outputs? Is there any evidence of complementary activities designed to maximize synergies and avoid overlaps? What were the good practices?

● What could have been done differently?

● Did you receive support from other UN organisations? If positive, is there any evidence of complementary activities designed to maximize synergies and avoid overlaps with the project’s implementing entities? What were the good practices?

● What could have been done differently?

● In your opinion, how do you assess the level of synergies and complementarity of the project with the interventions of other UN organisations in the country? (Likert: 6 levels)

● In your opinion, were there any UN organisations with which the Surge project could have increased synergies or avoided overlaps at country level (if positive, please specify which UN organisations): (Open ended).

89

Efficiency

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

5. How well coordinated was the process for the response among the entities implementing the joint project?

● How did you coordinate project activities with the other entities? Were there any opportunities for any joint work contributing to reducing implementation costs or scale benefits?

● To what extent did the project benefit from resources (structure, personnel, partnerships) made available by ongoing projects?

● What were the good practices in terms of coordinating the Surge project?

● What were the challenges? What could have been done differently?

● Did the UN entities coordinate adequately project implementation?

● What were the good practices? What could have been done differently?

● Did you receive support from several implementing entities?

● If positive, is there any evidence of complementary activities designed to maximize synergies and avoid overlaps? What were the good practices?

● What could have been done differently?

6. How did the three- phase budgeting and programming approaches impact the efficient delivery of the project?

● Did the phased approach come with a sufficient level of predictability to ensure synergies with the work of your agency? Did the phased approach prevent the project from formulating a long-term vision? Did the phased approach facilitate the formulation of

● Were the activities of the UN entities sufficiently predictable to ensure proper synergies with the work of your agency at country level?

● Did the phased approach facilitate any adaptive management? What were the good practices?

● Were project activities sufficiently predictable to ensure proper synergies with your organisation’s agenda of work? Could you properly plan the collaboration and joint work with the UN entities?

● Did you have a clear vision about what the UN

90

interventions across a pathway from crisis response, to recovery, and development?

● Did the phased approach facilitate any adaptive management? What were the good practices?

● What could have been done differently? What lessons learned from each of the three phases supported the continuation and continuous improvement of the project?

● What could have been done differently?

entities intended to deliver and the expected effects of their interventions?

Effectiveness

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

7. To what extent did the programme (Development Account) and project governance and management structures and processes enable, or hinder, the effective implementation of the joint project and the achievement of its results?

● Was the programme modality (Development Account) adequate to respond to a crisis? What were the strengths and weaknesses of the programme modality (Development Account) in a crisis context?

● What were the challenges faced by the project’s governance and management structures and processes?

● How do you assess the project governance and management structures and processes?

● What worked well? Are there any good practices that could be systematized?

● What could have been improved/what would you propose doing differently next time?

● How do you assess the project governance and management structures and processes?

● What worked well? Are there any good practices that could be systematized?

● What could have been improved/what would you propose doing differently next time?

91

● What were the good practices and lessons learned from project implementation?

8. To what extent has the project contributed to the expected outcomes as enunciated in the project document?

● What would you highlight as the most significant outcomes of the project in the following areas (as relevant)? (i) Improving national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship MSME promotion in post-COVID- 19 resurgence; (ii) Improving resilience and competitiveness of MSMEs in post-COVID-19 resurgence; (iii) Facilitating MSMEs registration and formalization; (iv) Improving MSMEs access to finance; (v) Increasing MSMEs access to innovation and technology; (vi) Enhancing MSMEs access to markets.

● What examples or evidence would demonstrate such outcomes?

● Who were the primary beneficiaries of the project?

● What would you highlight as the most significant outcomes of the project in the following areas (as relevant)? (i) Improving national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship MSME promotion in post-COVID- 19 resurgence; (ii) Improving resilience and competitiveness of MSMEs in post-COVID-19 resurgence; (iii) Facilitating MSMEs registration and formalization; (iv) Improving MSMEs access to finance; (v) Increasing MSMEs access to innovation and technology; (vi) Enhancing MSMEs access to markets.

● What examples or evidence would demonstrate such outcomes?

● Who were the primary beneficiaries of the project?

● What would you highlight as the most significant outcomes of the project in the following areas (as relevant)? (i) Improving national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship MSME promotion in post-COVID- 19 resurgence; (ii) Improving resilience and competitiveness of MSMEs in post-COVID-19 resurgence; (iii) Facilitating MSMEs registration and formalization; (iv) Improving MSMEs access to finance; (v) Increasing MSMEs access to innovation and technology; (vi) Enhancing MSMEs access to markets.

● What examples or evidence would demonstrate such outcomes?

● Who were the primary beneficiaries of the project?

● In what ways did the training activities, knowledge products, or technical support delivered by the Surge project influence your perspectives?

● To what extent did the training activities, knowledge products, or technical support delivered by the Surge project enhance your skills and/or knowledge of the subject matter?

● To what extent did the training activities, knowledge products, or technical support delivered by the Surge project enlarge your professional network?

● Were you able to use/apply in your work the knowledge or skills acquired through the training activities, knowledge products, or technical support delivered by the Surge project?

● Were there measurable changes in your activities and performance when you got back to your workplace that you can attribute to the training

92

activities, knowledge products, or technical support delivered by the Surge project?

● To what extent have the changes in your performance and new level of knowledge or skills sustained over time?

9. How did the response contribute to the participating country Governments’ responses to COVID- 19, especially in the area of MSME resurgence?

● What examples would you highlight showing that the project contributed to any of the following achievements (as relevant): (i) MSMEs sustained and growing their businesses, including through exporting and integration in value chains; (ii) Businesses newly established, newly registered/formalized MSMEs; (iii) Jobs sustained and created; including with regard to women and youth entrepreneurs; (iv) Policies and measures developed and implemented by governments to facilitate the MSME resurgence in the immediate term and to enhance their shock resilience and competitiveness in a longer term.

● What were the unintended positive and

● What examples would you highlight showing that the project contributed to any of the following achievements (as relevant): (i) MSMEs sustained and growing their businesses, including through exporting and integration in value chains; (ii) Businesses newly established, newly registered/formalized MSMEs; (iii) Jobs sustained and created; including with regard to women and youth entrepreneurs; (iv) Policies and measures developed and implemented by governments to facilitate the MSME resurgence in the immediate term and to enhance their shock resilience and competitiveness in a longer term.

● What were the unintended positive and

● What examples would you highlight showing that the project contributed to any of the following achievements (as relevant): (i) MSMEs sustained and growing their businesses, including through exporting and integration in value chains; (ii) Businesses newly established, newly registered/formalized MSMEs; (iii) Jobs sustained and created; including with regard to women and youth entrepreneurs; (iv) Policies and measures developed and implemented by governments to facilitate the MSME resurgence in the immediate term and to enhance their shock resilience and competitiveness in a longer term.

● What were the unintended positive and

● To what extent have the knowledge and skills you acquired though the training activities, knowledge products, or technical support of the Surge project contributed to improving the performance or results of your organization?

● To what extent did the knowledge and skills you acquired though the training activities, knowledge products, or technical support of the Surge project contribute to your ability to advance sustainable development.

● In what ways have the training activities, knowledge products, or technical support delivered by the Surge project contributed to: (i) Improving national capacities on formulating and implementing enabling

93

negative effects of the project?

negative effects of the project?

negative effects of the project?

policies on green, resilient and inclusive entrepreneurship MSME promotion in post-COVID- 19 resurgence? (ii) Improving resilience and competitiveness of MSMEs in post- COVID 19 resurgence? (iii) Facilitating MSMEs registration and formalization? (iv) Improving MSMEs’ access to finance? (v) Increasing MSMEs’ access to innovation and technology? (vi) Enhancing MSMEs’ access to markets? (vii) Engaging and sustaining women and youth entrepreneurs and other vulnerable and marginalized groups?

● Please, share concrete

examples of the types of policy or economic improvements, contribution to advancing sustainable development, or other entrepreneurship impacts that resulted from the skills and knowledge you acquired though the training activities, knowledge products, or technical support

94

delivered by the Surge project

10. What innovative approaches or tools, if any, did the response use, and what were the outcomes and lessons learned from their application?

● In which areas was the Surge project innovative?

● What were the most promising practices that participating UN entities should consider institutionalizing or replicating?

● In which areas was the Surge project innovative?

● What were the most promising practices that participating UN entities should consider institutionalizing or replicating?

● Was there anything innovative or unique with the Surge project that participating UN entities should consider institutionalizing or replicating?

● Was there anything innovative or unique with the Surge project that participating UN entities should consider replicating or scaling? (open ended)

Sustainability

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

11. What measures were adopted to ensure that outcomes of the response would continue after the project ended?

● Has the project advanced partnerships amongst project participants, national institutions and the MSMEs in support of sustainable results?

● How did the project contribute to leveraging funding and financing of government and other resource partners (multilateral, bilateral, etc.)?

● What were the enabling factors that contributed to making the project transformative?

● Has the project advanced partnerships amongst project participants, national institutions and the MSMEs in support of sustainable results?

● How did the project contribute to leveraging funding and financing of government and other resource partners (multilateral, bilateral, etc.)?

● What were the enabling factors that contributed to making the project transformative?

● Has the project advanced partnerships amongst project participants, national institutions and the MSMEs in support of sustainable results?

● How did the project contribute to leverage funding and financing of government and other resource partners (multilateral, bilateral, etc.)?

● Do you find that UN entities managed adequately the need for immediate response and the search for longer- term effects?

● To what extent did the Surge project contribute to promoting better preparedness for future crisis? (Likert scale)

95

● What were the enabling factors that contributed to make the project transformative?

● To what extent did the project contribute to promote better preparedness for future crisis?

Gender, human rights, and disability

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

12. To what extent were perspectives on gender equality, protection of human rights and reaching underserved groups integrated into the design and implementation of the project? What results can be identified from these actions?

● To what extent have women and other vulnerable and marginalized groups engaged, including outreach and selection process, types of leadership roles, decision-making, among others?

● To what extent have project activities been designed and implemented with particular attention to the needs and contexts of women and other vulnerable and marginalized groups?

● To what extent have Member States engaged and integrated human

● To what extent have women and other vulnerable and marginalized groups engaged, including outreach and selection process, types of leadership roles, decision-making, among others?

● To what extent have project activities been designed and implemented with particular attention to the needs and contexts of women and other vulnerable and marginalized groups?

● To what extent have national partners engaged and integrated

● To what extent have women and other vulnerable and marginalized groups engaged, including outreach and selection process, types of leadership roles, decision-making, among others?

● In what ways have the activities affected women and other vulnerable and marginalized groups in their lives?

● In what ways were women and other vulnerable and marginalized groups affected by COVID-19 restrictions and how was

● To what extent have women and other vulnerable and marginalized groups benefited from the training activities, knowledge products, or technical support delivered by the Surge project? (Likert: 6 levels)

● Any comments? (open ended)

● Disaggregated analysis of survey data

96

rights protections in their project activities?

human rights protections in their project activities?

that addressed in the project’s efforts?

● To what extent have national partners engaged and integrated human rights protections in their project activities?

97

Annex 4: List of documents reviewed

● Callo-Müller M. V. 2020. Micro, small and medium enterprises (MSMEs) and the digital economy. ESCAP. Bangkok.

● ECLAC. 2020. Sectors and businesses facing COVID-19: Emergency and reactivation, Special Report n.4. Santiago, Chile.

● ESCAP. 2021. Rethinking MSME Finance in Asia and the Pacific: A Post-Crisis Policy Agenda. Bangkok. ● ESCAP. 2021. The Role of Competition Policy in Strengthening the Business Environment for MSMEs in the ASEAN

Region. 11 February 2021. Bangkok. ● ESCAP. 2022. Bangladesh Startup Ecosystem Assessment Report. Bangkok. ● ESCAP. 2022. MSME Access to Finance: The Role of Digital Payments, MSME Financing Series No.7. Bangkok. ● ESCAP. 2022. Policy guidebook for MSME development in Asia and the Pacific, 2nd edition. Bangkok. ● ESCAP. 2022. Small and Medium-Sized Enterprises Lending Approaches: The Role of Banks in Asia. MSME

Financing Series No. 6. Bangkok. ● ILO. 2020. COVID-19 and the impact on agriculture and food security. Geneva. ● ILO. 2018. Women and Men in the Informal Economy: A Statistical Picture, Third Edition, Geneva. ● ILO. 2020. Contagion or starvation, the dilemma facing informal workers during the COVID-19 pandemic.

Geneva. ● ILO. 2021. ILO Monitor: COVID-19 and the world of work. Seventh edition. 25 January 2021. Geneva. ● ILO. 2021. ILO Monitor: COVID-19 and the world of work. Seventh edition. 25 January 2021. Geneva. ● ILO. 2022. Independent High-Level Evaluation of ILO’s COVID-19 response 2020-22. Geneva. ● IMF. 2020. World Economic Outlook Update: A Crisis Like No Other, An Uncertain Recovery. Washington. ● IMF. 2021. World Economic Outlook Update. January 2021. Washington. ● OECD. 2020. Global Outlook on Financing for Sustainable Development 2021. Paris. ● UN OCHA. 2020. Global Humanitarian Response Plan for COVID-19. Geneva. ● UN Women. 2020. Guidance Note for Action: Supporting SMEs to Ensure the Economic COVID-19 Recovery is

Gender-Responsive and Inclusive. UN Women Regional Office for Asia and the Pacific. Bangkok. ● UNCTAD. 2020. Concept note: Global Initiative towards post-Covid-19 MSME sector Phase I. April 2020.

Geneva. ● UNCTAD. 2020. COVID-19 Response. Global Initiative towards post-Covid-19 MSME sector: MSME Surge

Project. Phase II Project Proposal. July 2020. Geneva. ● UNCTAD. 2020. COVID-19: Firmer Action Needed to Better Protect Consumers. Geneva. ● UNCTAD. 2021. COVID-19 Response. Global Initiative towards post-Covid-19 MSME sector: MSME Surge

Project. Phase III Project Proposal. February 2021. Geneva. ● UNCTAD. 2021. Global Initiative towards post-Covid-19 resurgence of the MSME sector. MSME surge project -

Addendum Phase 3 Project Proposal. Geneva. ● UNCTAD. 2021. How COVID-19 affects MSME access to markets and competition: A review of key issues and

recommendations for future action. Geneva. ● UNCTAD. 2021. Impact of the COVID-19 pandemic on trade and development: Recovering, but unevenly -

Situation as at 31 March 2021. Geneva. ● UNCTAD. 2022. Cross-border Trade in the Pre- and Post-Pandemic Environment Evidence from Malawi, the

United Republic of Tanzania and Zambia. Geneva. ● UNCTAD. 2022. Entrepreneurship & innovation in the new health economy. Geneva. ● UNCTAD. 2022. Final Report: Global Initiative towards post-Covid-19 resurgence of the MSME sector.

September 2022. Geneva. ● UNCTAD. 2022. Guidance for action: Supporting MSMEs’ recovery from the COVID-19 pandemic. Prepared for

the trade competition commission of Thailand. Geneva. ● UNCTAD. 2022. The COVID-19 pandemic impact on micro, small and medium sized enterprises: Market access

challenges and competition policy. Geneva. ● UNCTAD. 2023. MSME Surge Project. Empretec: Impact Assessment Survey Report. Geneva. ● UNDESA. 2018. Micro-, Small and Medium-sized Enterprises (MSMEs) and their role in achieving the Sustainable

Development Goals. New York. ● UNDP. 2020. UNDP Briefing Note Gender COVID-19. New York. ● UNECA. 2022. The role of digitalisation in strengthening capacities of Micro, Small and Medium-Size Enterprises

(MSMEs) in Southern Africa to take advantage of the AfCFTA. ECA Sub-Regional Office for Southern Africa (SRO- SA) & ECA Digital Centre of Excellence. Addis Ababa.

● UNECE. 2020. Guidelines and best practices for micro-, small and medium enterprises in delivering energy- efficient products and in providing renewable energy equipment. Geneva.

98

● UNECE. 2020. Guidelines and Best Practices for MSMEs to assure resiliency and progress towards a circular economy in sustainable resource management and critical raw material supply chain solutions. August 2020. Geneva.

● UNECE. 2020. The Impact of COVID-19 on trade and structural transformation in Georgia. Geneva. ● UNEG. 2008. Code of Conduct for Evaluation in the UN system. New York. ● UNEG. 2020. Ethical Guidelines for Evaluation. New York. ● UNIDO. 2020. COVID-19, Implications and Response : Digital Transformation and Industrial Recovery. Vienna. ● United Nations Moldova. 2020. COVID-19 Socio-economic Response and Recovery Plan. June 2020. Chisinau. ● United Nations. 2017. Resolution adopted by the General Assembly on 21 December 2016. 71/221.

Entrepreneurship for sustainable development. A/RES/71/221. General Assembly. New York. ● United Nations. 2019. Resolution adopted by the General Assembly on 20 December 2018. 73/225.

Entrepreneurship for sustainable development. A/RES/73/225. General Assembly. New York. ● United Nations. 2020. A UN framework for the immediate socio-economic response to COVID-19. April 2020.

New York. ● United Nations. 2020. Resolution adopted by the General Assembly on 2 April 2020. A/RES/74/270. General

Assembly. New York. ● United Nations. 2020. Shared Responsibility, Global solidarity: Responding to the socio-economic impacts of

COVID-19. March 2020. New York.

99

Annex 5: List of individuals interviewed

Redacted for confidentiality purposes.

100

Annex 6: Evaluation surveys

The evaluation carried out six external surveys to inform the assessment of the relevance, coherence, and effectiveness of the project and its contribution to outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups. The surveys targeted participants to project activities and beneficiaries. The lists of contacts were provided by UNCTAD, DESA, ESCAP, ECLAC, UNECA, and UNECE. The surveys were composed, for some part, of questions that were common to all six questionnaires, and for other parts, of questions specific to each of the 5+1 project outcomes. The questionnaires were made available in English. They were translated in Spanish for outcomes 3 and 5 as target recipients were primarily in the Latin America and Caribbean region. The surveys were anonymous and remained open for 2 weeks, from Thursday 20 July to Friday 4 August. Two reminder messages were sent to increase the response rate. The surveys were launched to a combined list of 1454 persons, with 89 messages bouncing back. Altogether, the surveys compiled feedback from 133 respondents. All survey questions were optional. Questionnaires partially completed were kept in the batch of results when they contained information that was judged relevant, credible and meaningful. Statistics were calculated on the basis of the number of valid responses per question and not on the basis of the overall number of respondents to the surveys. The overall response rate to the email surveys is circa 9.7%. The surveys present the opinion of those who responded but not of the entire list of recipients of the questionnaires nor of all the beneficiaries of the project.

Outcome Email addresses Bounced back Valid email Valid responses Percentage

1A 574 38 536 52 9.70% 1B 258 24 234 40 17.09% 2 21 0 21 4 19.05% 3 219 4 215 11 5.12% 4 98 1 97 13 13.40% 5 284 22 262 13 4.96%

Total 1454 89 1365 133 9.74% Relevance of the project 1. To what extent do you agree with the following statements?

Strongly agree

Agree Somewhat agree

Somewhat disagree

Disagree Strongly disagree

Do not know

Total

The project activities responded to my priorities and were relevant to my work

54 58 15 1 2 1 2 133

The project complemented interventions by other international organizations working in my country/region

36 56 23 4 3 1 7 130

101

2. Did the project increase synergies or avoid overlaps with other agencies/entities working in this area?

Please explain.

● A lot of organizations were conducting projects regarding COVID, however this one was the only one providing actual information on practical ways to mitigate some challenges arising from the pandemic.

● As every and each project in Armenia there are overlaps

● Auemntaron las sinergias, al publicarse reformasa las leyes laborales y fiscles.

● Aumento mis criterios previos al proyecto

● Aumentou a sinergia com outras entidades à medida que deixou bem clara a importância das parcerias.

41%

44%

11%

1% 1%

1% 1%

The project activities responded to my priorities and were relevant to my work

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

28%

43%

18%

3% 2% 1%

5%

The project complemented interventions by other international organizations working in my country/region

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

102

● Have no detail information

● Here, it is more of overlaps with other agencies.

● I am not aware that there was good mutual communication between the job holders during the execution of the project.

● I believe the project will increase synergies and complement other initiatives in place

● I can say that any project makes an effort to enhance synergies and do away duplications if there is to optimize the resources and maximize the positive impact.

● I did not see avoiding of overlaps with other agencies working in this area especially at country level.

● I think the project definitely increased synergies with other domestic agencies / entities working in each specific area.

● I think this project increase synergies with other project

● IDEP; PNUD HUB REGIONAL DAKAR

● Increased synergies

● It created synergy as Seda provided a number of virtual trainings which is still continuing to date.

● It focused on areas that affected our common clients. However, there is need to make a follow up and implement the recommendations.

● It increase synergy because from my view point I think the manpower strength of this organisation will stand out if the y are innovative ,

● It increased synergies

● It increased synergies and avoided overlaps

● It increased synergies in the sense that it adequately complemented what had been on the ground by the activities of other agencies.

● It increases synergies

● It is still in progress.

● It really increase synergy fill in gaps in the others

● It was helpful for avoiding overlaps

● Like womens ministery

● more strategic alliances

● Neither yes or no because I have seen it or positive impact on me

● No

● No awareness

● No comment

● No it did not

● No.

● Nos proporcionó elementos para poder considerar el comportamiento en éste tipo de eventualidades temporales

● Not sure. Communication between agencies is not always the best.

● Not yet because the understanding among the companies and leaders is not concrete to work hand in hand however strengthen the relationship and strong collaboration among the companies is still under consideration

● Our country SME office

● Partnerships and cooperation within the different parties involved was evident. This includes Donors, Investors, Grant Managers, Incubators, Accelerators, Chambers of Commerce, Hubs, the UN and the Governments of the various countries involved.

● Permitió por sobre todo tomar conocimiento de las realidades regionales de las pymes, y sus dificultades u oportunidades para la mirada exportadora y las herramientas y brechas existentes

103

como barreras a superar y que en la mayoría de los casos es posible extender los casos de éxitos o superaciones de un país a la región

● Project increase synergies, gives hope especially with the financial support. It was really helpful. Covid -19 severely damaged the whole life emotionally, physiologically, financially. It was really good that this project was there during the Covid-19 difficult times.

● project increased synergies with and complemented internal activities in own company

● Project never visible in the communities

● Se dieron conexiones y nuevas sinergias que resultaron muy positivas

● Se evitó solapamientos, ya que se cuidó esta parte en la capacitación.

● si porque nos ayuda a tomar decisiones para formular proyectos para las Mypes en nuestro pais

● Strong synergies were evident, probably due to strong collaboration between the Agencies.

● The inter-institutional cooperation is important.

● The project activities can not overlap even if having similar set of objectives since it is a needed necessity for developing countries.

● The project avoids overlaps with other agencies.

● The project did create synergies and was generally applicable to many situations

● The project encouraged synergies with other agencies including the ILO in supporting and promoting formalization as well as the UNDP in supporting private sector development and SMEs resilience building.

● the project has increase synergies with other agencies

● The project has increased synergies rather than overlapping other activities in this area.

● The project increase the synergies of the efforts with other agencies and entities by focusing areas of intervention and rationalizing resources

● The project increased synergies

● The project increased synergies with between Government Agencies and the Private Sector whereby innovative ideas were explored and cutting edge solutions were developed.

● The project increased working relationships with other agencies. The project also covered some of the areas which Government might have resources to reach.

● The project looked at all the problems of the MSMEs and contributed to the summary of information in one place.

● The project was avoid overlaps with other agencies.

● The project was not only important source of information for the Moldova's Government, but also it was largely complementary to the work of few agencies, such as UNDP, UNCTAD, UN Women etc and was helpful in the process of CCA analysis, Gender assessment etc. Particularly, the project recommendations were presented during the Socio-Economic task force chaired by UNDP , the Economic Council under Prime-Minister and development partners coordination platform co-chaired by the UN and World Bank Group in Moldova. some of the findings were further adopted as activities under the UNCTAD and UNDP projects (notably regarding customs procedures and SME support).

● the project was used to make recommendations as part of a study on the MSME business environment in Madagascar

● The project was very important session

● This programme increased the synergies amongst the relevant ministries/institutions that works on MSMEs

● To build capacities to MSM"s

● To considerable extend did avoid overlap, what is important is to bring MSME center stage especially in developing economies and that unique role has been played by this attempt

● we had synergies and networking

● yes

104

● yes

● yes great synergies avoiding overlapping between UN RECs and DESA in NYC created new partnerships within and outside UN system

● yes it did

● Yes it did. Our collaboration with Regional Economic Commissions of the UN was very helpful, UNESWA and UNECLAC were already incorporating some aspects of competition work and SMEs and this project enabled us to collaborate more and share our expediencies like never before. We also closely collaborated with UNESCAP in their SMEs and also to organize the DA project closing event in Bangkok.

● yes it increased synergies with other agencies working in that area

● Yes it increases synergies

● Yes it was complimentary

● Yes right

● Yes the project enhanced synergies with other agencies through collaborations and partnerships to provide relevant support to MSMEs

● Yes the project increases the synergies of Empretec Centre Benin and chamber of commerce and Industry in Benin

● Yes,

● Yes, it did and help a lot

● Yes, it did increase synergies. Especially with NGOS who are impacting on skills acquisition, they also made impact but UNTACD surpassed them all

● Yes, it did. Information on the issues of access to markets and disruption of supply chains supported competition agencies in enforcing the law to support MSME sustainability.

● Yes, it increased synergies since several agencies were involved to implement the project.

● Yes, it was structured in a close coordination with other implementing agencies and key national counterparts

● Yes, MSME is the most hit by COVID-19 and need special focus

● Yes, proper planning of involved actors boosted synergies and avoided overlaps.

● Yes, since it was implemented in close collaboration with the UN Regional Economic Commissions and with the member States counterparts fully involved.

● Yes, the project increased synergies and the multi partner approach worked well.

● Yes, the project increased synergies. It created a platform where women who make up the majority MSMEs in developing countries are able to access technology at an affordable cost to enhance their businesses.

● Yes, the timing of the project was excellent, while businesses and other agencies were cautiously distanced due to Covid-19 lockdown, the project was empowering us as development practitioners to prepare for a come back to assist us in supporting our clients recover from Covid-19.

● Yes, it's created prioritisation, definitions and specificities toward gaps and overlaps.

● Yes. Maximize the use of resources in the area.

● Yes. Because the entrepreneurship is an issue that relates to all the agencies

● Yes. It led to knowing how one could use the current computer advancement in pushing business including customers participation

● Yes. The project complemented other studies conducted at national level for specific target group.

● Yes. The project was specific to the needs of women entrepreneurs

● Yes. In some ways.

● Yes… increased synergies and helped to strengthen actions.

105

Usefulness of the project 3. To what extent do you agree with the following statements?

Outcome 1A Strongly agree

Agree Somewhat agree

Somewhat disagree

Disagree Strongly disagree

Do not know

Total

The project provided access to information and best practices on government support measures for post COVID-19 recovery

35% 39% 18% 2% 4% 2% 0% 51

The project improved my knowledge on entrepreneurship/MSMEs policies design and implementation in the context of post COVID-19 recovery

39% 39% 12% 4% 6% 0% 0% 51

The project contributed to identifying new/improved policy measures for MSME promotion in post COVID-19 resurgence

43% 33% 14% 6% 4% 0% 0% 51

Outcome 1B Strongly agree

Agree Somewhat agree

Somewhat disagree

Disagree Strongly disagree

Do not know

Total

The project provided information useful for starting a new business or expanding a business venture

36% 41% 8% 3% 8% 0% 5% 39

The project improved my knowledge on entrepreneurship and business development

32% 45% 11% 3% 5% 0% 5% 38

The project contributed to identifying new/improved measures to increase business sales

42% 26% 18% 3% 5% 0% 5% 38

Outcome 2 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project provided useful information on MSMEs formalization/registration

36% 41% 8% 3% 8% 0% 5% 3

The project improved my knowledge of how to increase the number of businesses registering

32% 45% 11% 3% 5% 0% 5% 3

The project contributed to identifying new/improved measures to decrease the administrative costs of starting a business

42% 26% 18% 3% 5% 0% 5% 4

106

Outcome 3 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project provided useful information on financial inclusion and MSMEs access to finance

30% 50% 20% 0% 0% 0% 0% 10

The project improved my knowledge on accounting and reporting, including on the SDG reporting, and improved my capacity to manage financial resources

50% 20% 20% 10% 0% 0% 0% 10

The project contributed to identifying new/improved measures for financial inclusion and MSMEs access to finance

40% 40% 10% 10% 0% 0% 0% 10

Outcome 4 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project provided useful information on MSMEs access to innovation and technology

75% 25% 0% 0% 0% 0% 0% 12

The project improved my knowledge on how to increase MSMEs access to innovation and technology

75% 25% 0% 0% 0% 0% 0% 12

The project contributed to identifying new/improved measures to increase MSMEs access to technology

50% 33% 17% 0% 0% 0% 0% 12

Outcome 5 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project activities and publications provided useful information on MSMEs access to markets

38% 54% 8% 0% 0% 0% 0% 13

The project improved my knowledge on how to increase access to markets, including export and integration into value chains

54% 31% 15% 0% 0% 0% 0% 13

The project activities contributed to identifying new/improved measures for consumer protection and competition

46% 46% 8% 0% 0% 0% 0% 13

The project activities contributed to identifying new/improved measures on

33% 33% 25% 0% 0% 0% 8% 12

107

agricultural quality and food loss reduction

3-B. To what extent do you agree with the following statements?

Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project activities were accessible for women, youth, people with disabilities, or other vulnerable groups

33% 37% 21% 3% 2% 0% 4% 126

The project activities and content reflected the perspectives of women, youth, people with disabilities, or other vulnerable groups

33% 35% 26% 2% 1% 0% 2% 126

33%

37%

21%

3% 2% 0%

4%

The project activities were accessible for women, youth, people with disabilities, or other vulnerable groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

108

4. What (if anything) would you describe as innovative or unique from the project that the UN should consider

replicating or upscaling? Why?

● A simplified micro lending business method, such as funding a local small businesses with a day to day transaction of lend to collect daily loan offer with affordable interest rate.

● All.

● Ampliar programas de apoyo a zonas de vulnerabilidad a terceros países, o buscar cooperación es que llegue a los más necesitados e ir dándoles un acompañamiento

● As women are involved with this project, so UN should consider replicating.

● Business that clearly accommodate those with disabilities

● Capacitance communities to engage in entrepreneurship

● Considero que debe darle continuidad a las capacitaciones para que los usuarios hagamos mejor uso de la información financiera

● Continue supporting the local women led and youth led initiative to increase accessibility of services by women and girls in hard to reach communities.

● Direct training to entrepreneurs on appropriate technology application and innovation

● Diversification of business opportunity innovations and mentoring of Entrepreneurs

● During project designation, the target beneficiaries were not fully involved hence still some gaps which need more attention. some of the areas are…; Sustainability component is not there and exit strategy

● E-Commerce, promoting and upscaling global entrepreneurship especially for marginalized and vulnerable groups

● Everything is good

● excellent actions in the SDGs

● Finding, after training participants, small fund should be made available to start practicing what was learnt in the class

● Good governance in private sector to make engine of growth

● Government should already implement what has been proposed

● I don't know

33%

35%

26%

3% 1% 0%

2%

The project activities and content reflected the perspectives of women, youth, people with disabilities, or other vulnerable groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

109

● I live in a rural and the poorest province in my country. I was impressed by the reach of the project because while my country (the capital city) is hosting many UN regional offices, UN programmes are unheard of in these parts, though I suspect they are visible elsewhere in the country and region.

● Implementation of capacity-building training manpower , creativity to this various group,

● improving capacity-building

● Improving EE/RES secondary legislation for all countries

● Innovation simply means new methodology in doing things but on the aspect of activities of international organization in a particular country, the people that benefit almost everything are people at high, not people at lower, and whereas people at the lower class is most needed in the programme.

● innovations from women

● Innovative digital finance solutions and regulatory frameworks to support both financial inclusion and growth of SMEs, especially those owned and led by women. It is a new perspective for private sector.

● Innovative or unique from project that UN should consider they must teach also at local high school to give light to our future leaders

● it is an innovation project as it has increased women participation in business activities post-covid 19 era.

● It is necessary to give a valid and uniform assessment of the data (through the UNFC and UNRMS system), which is necessary for planning and understanding the issues in the preparation of strategic documents. Therefore, the application of the UNFC and UMRMS system for all resources, at the national level.

● la digitalizacion en el mundo globalizado

● La inclusión de jóvenes, para el fortalecimiento de sus habilidades y destrezas con una temporalidad más temprana.

● Large scale multifaceted project allowing for rapid intervention

● Las recomendaciones para tratar económicamente este tipo de eventualidades

● Let's the poor BREATHE

● Los emprendimientos liderados por mujeres, con criterios de triple impacto: Económicamente rentable, socialmente inclusivo y medioambientalmente amigable.

● More online activities

● N/A

● Need to upscale countrywide best practices to encourage or handhold MSME as an information brochure

● No comment

● Nothing at all, the project was very satisfying

● Nothing to say

● Opportunity or skills sharing from neighbouring countries. Forging business relationships and Empretec relationships.

● organize face-to-face courses for policymakers to support small and medium enterprises

● Our company benefited from Covid-19 by contracting testing & certification of medical Covid facial masks

● Platform or some sort of reimbursement for under privileged individuals who want to be part of this initiative

● policy dialogues hybrid dialogue inclusion of private sector

● Practically of the topics that were discussed

● Reducing border barriers, increasing non-refundable investment support for small and medium-sized enterprises in developing and underdeveloped countries, and improving cooperation between countries

110

● Should highly concentrate on youth and women empowerment especially on agricultural technology since the unemployment rate in the country is still high and is one of the sector which can help to reduce poverty yet still considering the best practices to reduce pollution of the environment

● Still figuring out on that

● Supporting small business for access to grants and Entrepreneurship training.

● Targeting people with disabilities and the vulnerable groups

● The best practices of project outcomes delivered in other countries should be supported and encouraged for adoption in other countries.

● The combination of independent study, YouTube videos, exercises, moderated online participation and tests facilitated greater engagement with the material, learning from the facilitator, and peer learning

● The cost of tools and others types for calculate

● The focus on MSMES and the practical approach undertaken allowed for impactful outcomes, which is not always the case of other technical cooperation projects.

● The global initiatives towards post covid 19 resurgence...its unique in tackling the novelty covid 19 and should be improved upon for future resilience and R n D by UN and lessons learn should be noted

● The number of times the trainings are done

● The online courses, webinars, and information dissemination.

● The opportunity for the usage of online tools for the dissemination of EMPRETEC programme to impact large participants

● The project, I interacted with was on formalization of MSMEs through the Cooperative business model and to small extent topics on Enterprise development were touched.

● The project's focus on capacity-building and skill development for MSMEs is vital for their sustainable growth, emphasis on sustainable and inclusive development, and the project's efforts to leverage technology and digital solutions for MSMEs are forward-thinking.

● The role competition policy in the economic recovery of MSMEs. Reason: MSMEs continue to face challenges related to their interaction with bigger counterparts. For instance, suppliers of agricultural produce to bigger multinational buyers. Traditional competition law rules do not present effective solutions. Consider issues of abuse of superior bargaining power and abuse of buyer power.

● The studies might contain innovative and/or unique conclusions and recommendations, so the UN should consider the best ones for replicating or upscaling.

● The training exercises to improve knowledge

● The uniqueness of this project, time span and evolution was specially in terms of allowing us as project managers to bring in new ideas and try them out. I have done projects before and also tried to implement new ideas, but this project was a clean slate and a lot of room for innovation. it was challenging but very enlightening. Such room to introduce ideas should be considered in projects as opposed to straight jackets. In the spirit of the new normal, projects should evolve and fit with the times.

● The whole approach was innovative, enabling one stop shop for mechanisms and best practices towards post-Covid-19 resurgence of the MSME sector.

● the work dedicated to the analysis of structural barriers to trade was very relevant and helped to identify measures necessary for facilitation of exports. Specifically, conducting flash studies to maintain evidence-based feedback from MSMEs on barriers to trade and access to external markets is crucial, especially in the context of current supply chain deterioration context in ECA region caused by the war in Ukraine. Furthermore, the work dedicated to profiling of women-led MSMEs was insightful and helped to develop legal amendments supportive to women employability.

● UN Projects

111

● UN should consider upscaling the Empretec behavioural approach to entrepreneurship skills development due to its high impact on motivation and confidence of participants and success of their business ventures.

● Undertaking a needs analysis and developing programs based on the needs of the MSMEs

● Upscaling new starts ups and contribute to identifying new/improved measures to increase businesses

● Upscaling the development of investment attraction mechanisms for energy system

● Use of online platforms. It helps people in MSME sector to even participate whilst near work.

● Women entrepreneurship can be upscaled to more people living in rural areas.

● yes

● YES ; SDGs is my occupation professional

Since completion of the project 5. Please share examples on how you were able to use the knowledge or skills acquired through the project in

your work.

● Access to information in one place was important to me.

● Acquired the best practices to the use and reduction of pollution while maintaining high level of Production

● An important project that I worked on, I highlighted key issues that are vital for the MSMEs whereby we provided entrepreneurial training, business counselling support and access to credit facility which helped the business to survive during and past the COVID pandemic.

● As a hoúsewife and a farmer, The acquired knowledge is helping me

● As an import-export retail business that I'd like to develop, I haven't yet applied the acquired knowledge toward my business.

● as part of my consultancy activities and with my students, including public administration officials in vocational training

● Asesorias a pequñas empresas y personas fisicas empresarias, par dar cvumplimiento a sus obligaciones fiscales y de información financiera para la obtenión de financiamienmtos. Cumplimiento en disposiciones de actividades conm enfoque a la sustentabilidad y cuidados ecoplogicos.

● Aun no he realizado un proyecto percé que haya sustentado las capacitaciones pero si comparto los conocimientos

● Better appreciation of challenges of MSMEs in accessing markets and finance which I incorporated into my analysis of competition law cases.

● By explaining deeply for the people at work how opportunity is being ma(d)e use of effectively and efficiently.

● coming up with easy women’s problems in farms

● COMPLETION AND CAPACITY KNOWLEDGE GREEN TECHNOLOGIES

● Conducting surveys and interviews with entrepreneurs, especially in rural areas, gave a deeper understanding of the existing problems of small businesses.

● De várias formas, uma delas seria a busca de parcerias com outras órgãos e entidades para assistir as micro e pequenas empresas no processo de exportação

● Desde la academia, socializo los temas tratados y como se puede adoptar soluciones de otras latitudes, previa adaptación a nuestro entorno o país. En el ámbito de las consultorías damos a conocer a clientes que en su gran mayoría no solamente son pymes sino también empresas familiares, respecto a los desafíos, puntos de nivelación necesarias y oportunidades para la

112

exportación de sus productos. Esto de de vital importancia habida cuanta que estamos en una region tri nacional, conocida como las tres fronteras. Ciudad del Este, Paraguay; Foz de Iguazú, Brasil y Puerto Iguazú Argentina.

● Design and implementation of new courses for MSME about internationalization and digitalization in my country

● Diversification of business opportunity innovations

● education for sustainable development

● En mi trabajo mejoro mis criterios de aplicación de la normativa

● En mis proyectos de crecimiento empresarial.

● Establish priority policies for women-owned and female-intensive businesses and social enterprises

● For future errands

● Giving charities to the needy people

● Goal setting was one of my best Pec, after the training l set meaningful , reasonable and achievable goals, . My friend invited me to join and play an online that will pay double in 30 days but I remembered being taught how to take calculated risk, l turn it down only for the scheme to fail after two weeks, my business money remained intact because l apply what l learn.

● Health and safety should be our own responsibility

● I am developed with accounting method

● I attended training & workshops

● I have attended the training and applied through my independent business development consultancy assignments by providing further trainings, information, and advice

● I have been able to design andupscale my projects within the circular economy space

● I have learned to utilize the E-view and SPSS analytical tool in identifying policy trends and priorities in the MSME sector in The Gambia

● I have not.

● I incorporated the skills into my organisation policies to advance youth meaningful employment and community development

● I just established an NGO which focuses on Climate change

● I learnt on how to reach out to prospective clients through the use of technology such as phone and internet to advertise my small business using whatApps.

● I managed to access GIS software and intend to use it in agribusiness interventions.

● I managed to understand the key problems that arise in the supply chain of critical mineral raw materials during crisis situations (pandemics, etc.), especially in conditions when there is no uniformity in their available data (in terms of quantities, availability, import-export, needs, etc.).

● I see solutions in any problems or situations. And it has improved my working abilities

● I was able to protect my self from covid and help my family gain from my skill too

● Implementar protocolos de bioseguridad en las asociaciones o empresas

● Important to use skill acquired especially for women entrepreneurs and employees how easily converging can enable to get them credit , skill up gradation and special needs yet being in competitive world.

● Improved my appreciation of synergies and partnerships in development.

● Increased the level of communication using internet. More active collaboration with partners to create consortium and participate in new tenders in area we didn't participate before. Expand area of activities.

● Information sharing om border formalities

● Innovation and best practices

● Installed wifi in my village home using solar to ensure I continued working in spite of remote location and lack of electric grid

113

● Introducing net metering for PV Autoproducers

● It enabled me to go for computer training

● Like Business Development Service (BDS) Centre, all knowledge acquired through the project will continuously impact the delivery in terms of training for the capacity-building of MSME in Benin

● Looking for innovation to solve emerging society issues is a tool for success

● Material generated through the project was of sufficient interest to Member States that it resulted in multiple invitations to present the work at various forums, and this enhanced links between policymakers and the UN.

● Mothers in headhouse

● My contribution has been in Energy Sector ( SME)

● N/A

● No capital to invest on the knowledge

● None

● online course

● Only in small entities that need prepare financial statements to share to Banks.

● Our company benefited from testing and certification of Covid-19 related facial masks

● Por mi actividad profesional, revisamos información de varias actividades económicas y las divulgaciones de los efectos del covid no pueden ser generales, porque a unas actividades económicas la pandemia hizo crecer sus operaciones comerciales

● Programme development and management - applied the knowledge and skills gained in initiatives crafted for MSME development for nationwide implementation Mainstreaming sustainability principle - sustainability principles were embedded in several MSME development initiatives of the office, ensuring that environmental, social, and economic considerations are integrated into our frameworks.

● Propagation

● Report findings were publicly disseminated amongst Development Partners (via different modalities and platforms) and were subsequently incorporated into their project design. The key findings were included in the CCA while certain policy recommendations utilized to provide inputs for EU annual report.

● Roll-out new training material developed during the project and disseminated through ad-hoc online sessions to national counterparts

● Several included mechanisms were subsequently utilized on certain level in my work environment and daily activities, for example, available state support for the MSMEs.

● Shared as a best practice to other partners that we are working with and rallied the critical role of Cooperatives in formalization

● sharing my knowledge with women entrepreneurs to uplift their businesses..

● Support decision-makers in the country in the elaboration of initiatives, and measures for the development of female entrepreneurship.

● The analytical study generated policy recommendations, business operations, communication, and financing options that might be used in future work.

● The examples of businesses, how they adjusted to the situation gave me new ways of thinking how to improve during challenging times.

● The knowledge gain was highly considered helpful in the areas health emergencies and livelihood survival greatly & the should shared through the programme help migat

● The knowledge has been of great help in sourcing, grouping and keeping up with marketing. And secondly, it has greatly improved my view on record keeping.

● The ocean economy is not very well explored in my area. The ethnic groups have always avoided water out of cultural and spiritual reverence. My development programmes never included the ocean economy until after this project. I have started an awareness initiative that presents the ocean as a

114

possible and lucrative source of livelihood. I believe the initiative is making a breakthrough because locals (though still very few) are beginning to lodge application for fishing licenses to the authorities.

● The online training course was a good example to highlight . It was possible to gather participants in an area which has not been covered before., the link between competition and SME policies.

● The products developed correspond to useful tools when supporting MSMEs in specific areas, such as competition law and policy.

● The project provided references to us in supervising the partnership between MSME and large/medium companies.

● The study helped us refocus on the new needs of MSMEs

● Through training.

● To be able to conduct trainings online, has been my best rewarding experience.

● To teach students

● We are able to understand and tackle similar challenges in future as we are better informed. We are more resilient and adaptable than before.

● We have organized workshops in relation to the UNCTAD MSME project in Thailand both national and regional agencies.

● We organizing all of training and expos by online and conducting IT technology for our operations

● We used the skills learned in undertaking needs analysis to undertake a needs analysis for our capacity-building programs

● Women capacity-building is important to acquire skills through knowledge.

● Working with FAO and UNDP in developing National Horticulture and Agro Processing Strategies

● Yes onboarding some companies to ecommerce platform. This initiative was lead by my self through skilled gained in the training

6. What are the challenges, if any, in applying the knowledge or skills acquired through the project to your

work?

● Lack of support from the body at the end of the training. So the knowledge is not practicable in the wider market

● Some of the policy and technology that was discussed cannot be accessed locally

● Access to full process of the project

● Access to new suppliers was difficult, as they require higher quantities to be requested. Creation of association facilitated to solve the issue.

● Accessibility to the most under developed rural areas.

● ampliar el conocimiento y plasmarlo en proyectos factibles de acuerdo a la realidad de cada pais

● Application is always not easy at first but I pull through

● beyond an online evaluation like what is currently done here, set up an on-site follow-up programme after the training

● Budget constraints in implementing initiatives

● Business environment in my country with limitations for MSME

● Capacity-building, adequate financial support and mentoring

● Changing of focal points in government counterparts and partners

● Coming back to business after a long vacation is terable

● education for sustainable development

● Falta de conocimiento de terceros u a la vez creen que no sería necesario

● Few institutions/enterprises/decision-makers/experts contribute and facilitate sharing of good practices.

115

● Financial access

● finding buyers of the product

● For me about the job description in every profession

● For Thai MSMEs, it is required a number of knowledge and skills to confront the obstacles of doing business in the post Covid.

● Funding is the biggest challenge. I am unable to go as far as I wish to because even pitching a request for funding is time consuming with too many technical requirements.

● Get the community mindset on board

● Getting gadget to use especially on disabilities peoples

● Governments and professional colleagues are not interested.

● GREEN ECOMIE GREEN INDUSTRIAL PROCESS ...

● Healthy is our daily routine for every I our capacity-building

● If skill training will not be available.

● Information availability, financing models

● Insufficient guidance and strategic planning by senior management.

● It was a unique situation, people were in finding difficult to normal the new social behaviour as results of covid nigh

● It's still premature to outline any challenges.

● lack of funds to be able to reach out to all of the entrepreneurs.

● Lack of infrastructure & unskilled people merged to work with me slow down the job

● Limited country data availability to conduct analysis for effective policy decision-making process

● Limited resource envelope and lack of practical skills.

● Limited resource in undertaking effective surveys that will be used to guide a programme design

● Limited resources

● Limited resources and skills to implement strategies to the full capacity

● Logistics

● Los cambios bruscos que se tienen que efectuar para atender oportunamente este tipo de eventualidades de acuerdo con las necesidades de las empresas

● Los retos es cambiar la manera de hacer el trabajo aplicando los lineamientos de la niif para pymes

● Low competencies of SMMEs in adopting new technologies for attaining quick recovery from the effects of the Pandemic. Many SMMEs still look up to Government for support.

● Mainly the biggest challenge is the financial aspect of new startups.

● Master the tools

● N/A

● Need of repository of country-wise concessions or hand holding for MSMEs at one place

● No challenges

● No difficulties

● no problem

● Non

● None

● None so far.

● None.

● Not accessed

● not really

● Not yet

116

● O maior desafio seria a criação de um ambiente colaborativo para oa implementação de políticas públicas voltadas ao incentivo das exportações para micro e pequenas empresas

● One challenge was to promptly adjust the lab equipment and train staff for testing medical facial masks

● Opportunity seeking and taking initiative, l have seen an opportunity l would have taken the initiative but financial challenge caused a set back.

● Our governments intervention about policies

● project didn’t finish

● Que sea útil en la toma de decisiones.

● Resources to reach more traders on how they can do business the time of COVID -19.

● Societies of womens

● Some business shuttered when the COVID hit the world.

● Subsidies process

● Technical and financial capacity limitations

● Technical facilities

● The biggest challenge was persuading respondents to provide factual data and answer the questionnaire.

● The challenge is we raised high expectations from our member states to do activities together and with no funds for the continuation of the work, it is frustrating both ways.

● The challenge was to classify critical raw materials in a uniform and recognizable way through the realization of the project and make an assessment regarding their availability and needs for the health sector, which was realized most often in conditions of insufficient available data.

● The challenges are coordination and synergies amongst relevant agencies/entities.

● The cost of internet charges is currently high in my home country. high internet intermittent or rather bad internet signal.

● The main challenge concerns the establishment of strong partnership for the funding to support the delivery of EMPRETEC programme in developing countries

● The only challenge derives from the very specific focus of the work programme I coordinate, which doesn’t necessarily allow for extensively addressing general and broad MSMEs challenges.

● The only challenge is funds.

● The policy recommendations were not costed that creates difficulty in promoting their implementation amongst UN agencies or other development partners.

● Their uselessness

● Total y plenamente posible desde el ámbito académico estoy proponiendo temas que hacen a la necesidad de la potenciación de las pymes, desde sus digitalización, formalización y actualización en temas claves para fomentar la exportación. Y desde el ámbito de la consultoría y desde la cámara de empresarios de Ciudad del Este y Alto Paraná estamos promoviendo un Diplomado en Comercio Exterior dirigido a empresarios pymes y empresas familiares.

● Training Ministries, Agencies and Regulatory bodies

● Trasmitir adecuadamente los conceptos de sustentabilidad.

● We have been able to apply the knowledge.

7. Please share examples, if any, of how the project included or supported women, youth, people with disabilities or other vulnerable groups.

● Accounting and budges

117

● All projects that will be implemented has to include Gender and Development (GAD) and must be inclusive.

● All socially and economically advantaged groupings such as the women have encouraged to form cooperatives and access funds constituency development funds to alleviation and improve their welfare.

● By spreading the information to people in need of it.

● CIRCULAR ECONOMIE OF RECYCLE WASTE SOLID

● Community project on understanding the need for non pharmaceutical adherence practice.., proper information about the viral nature, the strains ,transmission of covid and also fight disinformation due to lack of information.

● Como complemento a la mirada exportadora y desarrollo digital de las pymes, también impulsamos y socializamos los criterios de sostenibilidad con triple impacto: económico, social y ambiental, como política de las empresas privadas y dentro del mismo se enfatiza la oportunidad inclusiva para mujeres, jóvenes, y personas con discapacidad, u otros grupos vulnerables.

● conscientization in education for sustainable development (SDG5 , SDG10)

● Differently able were able to seek required skills and seek for sustainable employment.

● Diversification of business opportunity innovations

● Door to door activities, service was brough close to people and hence elder and disable women and girls and men were able to access it

● Empowerment

● Enhanced business models and resilience

● gender equality in panel discussions inclusion of gender dimension in report

● Have no detailed information

● Humanity and responsibility

● I don't know because I can see any positive update on us

● Iam vice chairperson and cofounder of Autism Rwanda so here I tried to educate women's and men's in this organization how to use devices for people with disabilities

● I'm looking forward to get disabled volunteers and include the in creating a clean society

● In Benin the selection of the beneficiaries includes creterias which has eased the inclusion of women, youth and other vulnerable groups

● In rural surveys and surveys, more than 50% of the building materials trade is run by women

● It increased the level of knowledge

● It was important to invite women-in-bussness in our activities . A PhD student used our output to finalize a thesis of women in finance, in Moldova.

● knowledge and technical skills

● La invitación fue generalizada, sin condicionar condiciones.

● N/A

● NA

● New tools to support to MSME owned by women

● No aplica

● No los hubo.

● No tengo conocimiento cuantas mujeres incluyo

● None

● Not in my cooperative sector

● not really

● One of key targets of the project was to ensure at least 30% of women participation.

● Our testing lab for testing and certification of medical facial masks included female staff

118

● Over 50 per cent of participants in activities supported by this project are women

● Participation

● Personally as a woman, l was really empowered, l saw myself as a redefined entrepreneur, l had become more efficient in my work

● Por la pandemia y la falta de movilización libre, propicio innovar estos medios de divulgación que facilitó la divulgación de esté tipo de información

● Priority will be given to support first and the level of support will be higher than that of a regular business

● Project participants got starter packs and were able to uplift their livelihoods.

● Project supported women, youth and vulnerable families by promoting EE/RES

● Proyectos privados con PERU LNG en Peru se viene ejecutando productos productivos y con nuevas propuestas para mejorar la comercializacion y articulacion

● Research on women's entrepreneurship enabled policymakers and ministerial staff in Member States to have a better understanding of how support given to women in business pays off.

● Still doing researches on that

● The innovative aspects of doing things was the best part of the training, with Data your market will sell.

● the necessary trainings given to them are useful to them

● The project conducted various capacity and skills training activities were beneficiaries mainly targeted women, youth and vulnerable groups.

● The project had a gender component.

● The project included an analysis of women's participation in "Small and Medium Enterprises" projects, which were represented both in the field of critical raw materials and the supply chain during Covid-19, and which are expected in the post-Covid -19 period. This analysis was evaluated very positively.

● The project included and looked at those target groups and can be a starting point for some more detailed activities

● the project produced analytical piece on the impact of Covid-19 on trade and business development prospects of women-led companies which fed into subsequent policy action adopted by the Government in relation to childcare as a part of women empowerment and employability.

● The project was supported for the women and youth through capacity-building training

● The project would also cover on areas of " COVID-19 Seed recovery for women and Youth"

● The said group had been able to get access to entrepreneurship training and access to credit facility.

● The survey sampled enterprises from women, youth and PWDs

● There was a deliberate effort to include women and youth among the participants

● There were Covid Relief funds disbursed to women and youth MSMEs to cushion them during the crisis. In Kenya for example the group WomenWork in partnership with Kenya Chamber of Commerce and 4G capital ensured we got the funds directly to our phones through mobile money.

● There were specific trainings targeting women entrepreneurs of developing countries in East Africa, which understood that these MSMEs also needed to shift from informal trading to formal businesses.

● these people are highlighted in the recommendations made but the implementation is up to the government

● Through knowledge sharing seminars and workshops

● Training of women entrepreneurs in Malaysia/ Empretec training in the context of Cross-border trade activities/Training on Farming as business/E-Regulation platforms allowing for formalisation of informal sector.

● Upskilling of youth and women businesses

● Use of sign language

119

● We do have a success story from a woman from Zimbabwe who participated in the online course: see her story. link here. https://unctad.org/news/how-make-small-businesses-developing-countries- more-competitive

● we have not any special project

● While my farming project supports mostly women and youth, the new initiative is more popular among men.

● Women Empowerment

● Women, youth, people with disabilities are we support them

Results of the project 8. To what extent do you agree with the following statements?

Outcome 1A Strongly agree

Agree Somewhat

agree Somewhat

disagree Disagree

Strongly disagree

Do not know

Total

The project contributed to formulating and implement enabling policies on green, resilient and inclusive entrepreneurship for MSME promotion in post COVID-19 resurgence

32% 39% 15% 10% 2% 0% 2% 41

Outcome 1B Strongly agree

Agree Somewhat

agree Somewhat

disagree Disagree

Strongly disagree

Do not know

Total

The project activities contributed to expanding my business venture or to starting a new business

15% 42% 18% 0% 12% 0% 12% 33

The project supported my business by increasing sales

15% 27% 24% 3% 12% 3% 15% 33

The project activities contributed to the creation of new jobs

18% 39% 21% 3% 12% 0% 6% 33

Outcome 2 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project contributed to facilitating MSME registration and formalization

100% 0% 0% 0% 0% 0% 0% 3

The project activities contributed to decreasing the administrative costs of starting a business

0% 100% 0% 0% 0% 0% 0% 2

The project contributed to increasing the number of businesses registering

50% 0% 50% 0% 0% 0% 0% 2

120

Outcome 3 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project contributed to improving financial inclusion and MSMEs access to finance

11% 44% 33% 11% 0% 0% 0% 9

Outcome 4 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project activities contributed to increasing MSMEs access to innovation and technology

27% 45% 27% 0% 0% 0% 0% 11

Outcome 5 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project contributed to new/improved measures on consumer protection and competition

11% 33% 56% 0% 0% 0% 0% 9

The project contributed towards new/improved measures on agricultural quality and food loss reduction

13% 25% 50% 0% 0% 0% 13% 8

The project contributed to increase MSMEs access to markets, including export and integration into value chains

13% 63% 25% 0% 0% 0% 0% 8

8-B. To what extent do you agree with the following statements?

Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project contributed to improving the resilience and competitiveness of MSMEs in post-COVID-19 resurgence

38% 35% 16% 4% 3% 2% 3% 106

The project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups

32% 32% 28% 2% 3% 0% 3% 105

121

9. Please share any examples of how the project has contributed to the resurgence of MSMEs, policy or

economic improvements, or advancing sustainable development in your country.

● A través del sistema de registro, importante para conocer cómo se desarrollan económicamente las

empresas.

● Access to finance and creation of programs so that MSMEs pivoted to continue with their operations.

● After that , Most policies at work has aspects protection and awareness

● As always in Armenia nothing ever helps.

● By applying the knowledge acquired

37%

35%

16%

4% 3% 2%

3%

The project contributed to improve the resilience and competitiveness of MSMEs in post COVID-19 resurgence

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

32%

32%

28%

2%

3% 0%

3%

The project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

122

● By developing good guidelines for MSMEs to assure resilience and progress towards a circular economy in sustainable resources management and critical raw material supply chain solutions.

● By touching the concern people.

● City of Harare developed a MSMEs policy which now recognize the importance of informal sector and has included use of technology as one of the building blocks.

● Con éste tipo de proyectos apoyados por la ONU, desarrollados por especialistas son de mucha ayuda para los que quieran y estén interesados para aplicarlos en beneficio de las actividades económicas de mi pais

● conscientization in education sustainable development in companies

● Diversification of business opportunity innovations especially into Renewable energy

● Economic improvements through Income generating activities

● El proyecto desde mi percepción, lo que ha logrado es aportar herramientas de singular importancia para el fortalecimiento de las empresas que han sobrevivido a la pandemia, a entender mejor los dolores del mercado, y las oportunidades y barreras para la exportación.

● Empowerment

● En mi trabajo no tengo acceso para poder dar un ejemplo

● GREEN JOBS. GREEN TECHOLOGIES

● I don't know

● I joined the project as a freelance development practitioner, and was not commissioned by my government. I don't think any representative from my country joined the project. However, I do engage in multi stakeholder forums, workshops and conferences where I make a point of raising issues of SMME resilience for inclusive development.

● In my country, I see a lot of businesses are moving into trading online, thus increasing their market.

● influenced Thai SME policy and national strategic plan

● it contributed with easier access to supporting mechanisms, as well as tailored guidelines by country for the MSMEs in order to overcome all challenges from the new environment caused by the pandemic.

● It's really improved the small through new methods of engagement using available technology and innovations to drive market ...also in the areas of supply chain innovation to deliver goods and services

● logistic service is more developed

● Maybe for those who are already in business

● mejorar los ingresos economicos de las familias, contar con capital, mejorar los costos de produccion

● More recently in my country Kenya, various funding opportunities are now available to MSMEs through the convenience of mobile phone. This includes the Women Enterprise Fund, the Hustler Fund, and Group loans. Grant Managers such as the KCIC Group have ensured sustainable development in our country by promoting and funding sustainable projects throughout the country.

● MSME is going to be back bone for developing economies. Focus on policies to encourage them especially guide them to explore eCommerce advantages

● MSMEs started using e-commerce and this lead the development of digitalization and consequently facilitate business operations

● Muchos mipymes se reinventarse y a la vez entendieron la importancia de que hay que estar preparados ante cualquier eventualidad, no sobrevive el más sabio u cauteloso, sino quien tenga una idea innovadora , la desarrolle y sepa como llegar al consumidor

● My appreciation of MSME challenges in markets contributed to effective enforcement. We were able to progress complaints faster and also require large buyers to review oppressive contracts to support sustainability of their MSME suppliers.

● N/A

123

● na

● New skills were mastered in the conditions of the pandemic, and certain types of risks of natural disasters and pandemics began to be considered in a valid way, which also influenced the introduction of new jobs for developing and solving this problem.

● No comment

● Not reached affected communities

● Not yet

● Only in cases the entity has the obligation to prepare financial statements to acces to loan in a local bank.

● Our government poor credit methods

● our project was a good example for resurgence MSME, after that, we will prepare for the future the sustainable strategy for vulnerable groups.

● Project allowed for distribution of fiscal incentives through e-Regulation platform El Salvador. EPF strategies developed for beneficiary countries and recommendations on implementations included measures specifically oriented towards MSME rsurgence

● Promoting olive biomass pomace for heating and hot water energy demand

● Relevant

● Revamping the Waste management system through collaboration with government in the sanitation of the environment in Calabar, Nigeria.

● Se logro la reorentacion operativa de PYME al establecer politicas decontratación depersonal

● Several online events were organized to share success stories of Empretec participants during COVID 19 restrictions which raised awareness of others on business opportunities during the pandemic.

● sharing knowledge

● so far there is no significant contribution however sensitization to the communities and to companies that have great impact on the green technology were the targets for training how they can contribute towards the climate emission

● Some traders they have knowledge on how to control resources for business and looking ways on how to find start-up Capital

● Still doing research on that

● successful project to put up

● The country shifted from somewhat backward thinking and started to focus on ideas that will positively impact the society more especially in the rural areas

● The focus of the project was to support MSMEs to overcome the constraints enhanced by the pandemic, so enhancing resilience in a sustainable manner was a constant goal.

● The Ministry of Agriculture is coordinating National strategies meant to empower farmers towards sustainable production and sustainable Value Chain Development. Women, Youth and People with disability are being supported to ensure NO ONE IS LEFT BEHIND

● The outcomes of the conducted studies helped to develop policy recommendations provided to the government in support of economic improvements and advancing sustainable development of the country.

● The partnership involved the Microsoft Small Enterprise Authority of Kenya and hence got practical feedback from the study

● The project enabled better access to information and contributed to the increase in the number of MSMEs

● The project had a positive impact on MSMEs, there were a lot of job creations reported.

● The project has trained entrepreneurs to be more efficient and confident in their work thereby contributing to economic development

● The project jobs The project activities contributed to the creation of new jobs Strongly agree The project activities contributed to the creation of new jobs Agree The project activities contributed to

124

the creation of new jobs Somewhat agree The project activities contributed to the creation of new jobs Somewhat disagree The project activities contributed to the creation of new jobs Disagree The project activities contributed to the creation of new jobs Strongly disagree The project activities contributed to the creation of new jobs also the project contributed to improve the resilience and competitiveness of MSMEs in post COVID-19 resurgence here in Rwanda

● The project revealed challenges faced by women-led MSMEs which subsequently were complemented by development of gender-based assessment jointly with the WBG. Furthermore, challenges faced by women in terms of access to ECD facilities (which became more acute during the pandemic lockdown) were addressed by the GoM policy on regulating the ECD facilities in private sector that contribute to women's economic empowerment and employability through regulation of establishment of creches and kindergartens under private companies employing young women with children, as well as facilitating expansion of private and public ECD facilities as one of the measures supportive of women labour participation.

● The project strengthens youth and women's employment opportunities, therefore, improving economic and community development.

● The project supported a Policy Implementation Forum: Supporting Micro Small and Medium Size Enterprises (MSMEs) in the Post COVID-19 Pandemic Era - Promoting MSME Formalization This event was jointly organised by DSDG/DESA and the Ministry of Trade, Industry, Regional Integration and Employment (MoTIE) of The Gambia and the UNDP. It brought together high-level policymakers and development partners to review challenges and identify solutions supporting the growth of the MSME sector in The Gambia in the post-COVID-19 pandemic era, taking into account opportunities offered by the African Continental Free Trade Agreement (AfCFTA).

● The results were widely distributed to National stakeholders, some concrete actions were implemented by ODIMM Moldova by launching new support schemes for MSME..

● The SDG’s are simply seen as fancy paperwork that yields funds, without any concrete plans on how to implement them…. Nigeria

● The training opened the eyes of the participants to the potential of MSMEs especially through leveraging ICTs

● there is a lot of development in my country as many businesses were in the verge of closing up during covid but thanks to the project, they survived.

● Through this project the government of the Republic of Zambia under the ministry of small and medium enterprises has greatly improved women participation in business activities through various coorporatives

● to support women with disabilities, students are mobilizing with companies and individuals to collect small equipment such as a sewing machine, knitting needles and balls of wool, raffia etc so that these women can start producing and selling fashion or decorative clothing and accessories

● we establish comprehensive business policies that prioritize women-owned businesses, female- intensive businesses and social enterprises

● Womens inclusive in polca, and governement

Looking forward 10. Are there other ways in which you think the UN could support the MSMEs sector in your country?

● I have a startup business in agricultural exports.

● Physical training and accelerator programme would be good

● 1. conducting relevant trainings on the implementation of MSME sector in the country. 2. development and sending of press releases on the report to government agencies.

● Access to finance for MSME, increase to financing resource

125

● Access to finance that may serve as post-training support for MSMEs

● be closer to local reality beyond best practices because there is no one best way but all depends

● Building the capability of BSOs, especially where the function of trade has devolved to regions, provinces and even lower forums. The Federal bodies in Pakistan are no longer representative of the MSMEs, especially no support for sectoral specialization.

● By establishing a zero-rated network programme that connects entrepreneurs with their immediate market/community, linked to other up to the international community

● By Funding Start up organizations and continue to provide classes for quality results.

● By provide some seed money

● By providing various training on how to manage businesses

● By sending technological tools

● By sharing knowledge and capacity development

● Collaborate with national governments to promote the implementation of best practices, in particular, to disseminate best practices in the development of the MSME sector.

● Conducting specific trainings in the field of financing and access to the market

● Considero que si dándole seguimiento con mas capacitaciones para ampliar esos criterios

● Continue supporting and mentoring MSMEs

● Design and develop policies for businesses in specific industries

● Developing BDS services/Facilitating business through e-tools/allowing for smooth implementation of new sustainability reporting standards introduced by ISSB.

● Direct engagement with private sector and capacity-building of private in promoting good governance

● Ejemplificando mas la información tomando como base la actividad económica para que sea comparable la información y poder medir el impacto en nuestro pais

● Enlightenment and also empowerment

● Grassroots propagation

● Ground -top approach which will break and limits the barriers of state bureaucracy and other means of engaging the people ..who will be directly affected

● I don't know.

● I hope to continue the indicative

● implementation of pilot projects with training and financial support for rural businesses

● In addition to training, the project could invite MSMEs to submit proposals for financial support so that there is a more practical element to the training

● In collaboration with UNCT to develop an agile policy paper that would highlight short- and long term measures necessary to stimulate economic growth (and improvement of livelihoods) amidst regional war-related crises and trade disruptions.

● In Moldova, a majority of micro, small, and medium enterprises (MSMEs) owned and led by women operate without access to resources and benefits. Assistance could be provided to the ministries and, agencies and to the Secretariat of the Prime Minister on the development of policies and initiatives in support of entrepreneurship, especially for women, youth, and the elderly, of digital finance solutions, transition to sustainable trade and a circular economy.

● Increase support to improve the business environment as well as to reduce the cost of doing business, including: A. Access to finance/capital/credit B. Support the establishment of the Single Window Business Registration (e-services) for countrywide coverage C. Improve the quality of infrastructure for product development and trade D. Support the development and improvement of business environment-related infrastructures etc.

● Information Dissemination

● more financing for the conscientization in the education for sustainable development

126

● More funding opportunities are required to meet the demand especially with the economic crises being witnessed. Do not tire in doing good.

● More targeted activities or studies, specified to certain issues/challenges on MSMEs will be helpful.

● MSMEs need continuous training to enhance their knowledge and skills and to face new challenges: there should be annual courses on a wide range of issues for their update.

● n/a

● na

● New webinars or courses about the key issues for MSME

● Not specific to UN, but the MSMEs in my country are one sector that needs support in many areas. For example, in my view, they need support on energy planning and management - eg more awareness to energy audits, awareness of the possibilities to improve, utilisation of RES, how the energy influence the price of their service/product etc.

● Project Funding for the ongoing UNDP Supplier Development Programme in Botswana. More MSMEs to be capacitated to be part of the Government and Private sector Supply Chain.

● Provide direct capacity-building support to MSME entrepreneurs, with priorities given to women, youth and groups in vulnerable situations; focusing more on the effective implementation of MSME supporting policies in line with demands of MSME entrepreneurs, instead of supporting the formulation of policies themselves

● Provide support in developing MSME friendly policies, complement government resources through financial and technical support

● Publicizing the work of our entrepreneurs internationally

● purchasing women innovations

● Si

● Si por supuesto. Es una figura de autoridad y relevancia, por lo que genera interés en participar.

● Si, compartiendo información de PYMES que requieran asesorias, en caso de contar con alguna estadistica relacionada.

● si, con apoyo de ONG, instituciones privadas y otros

● Sim.

● soft loans and more accessible training publicity

● Support in establishing technology and innovation hubs, centers, and incubations for youth, women and disabled persons

● The UN would support the women and youth on Start-up capital, cross border traders lost thier capitals during the COVID-19 because of movement restriction and there was no business at all.

● There is a little support

● Through capacity-building

● Through capacity-building and mentoring projects.

● Through direct training of personnel on concrete examples, but including personnel in government sectors where, in accordance with procedures, the relevant issues are dealt with and where there are "fully trained and qualified professional workforce", in order to improve the effectiveness and efficiency of their work and fully realize the principles of sustainable development in the shortest period.

● Through organised registered cooperatives

● To open more international trading opportunities (business linkages)

● To provide grants and access to credit facility. To provide vocational training and value chain training.

● To support the underserved people MSMEs and marginalized communities' economic growth, the UN should collaborate with grassroots NGOs that promote employment and community development. The local NGOs are closer to the people and understand their challenges.

● Totalmente, a más de los programas que ya tienen desarrollado para aplicar, considero que la vía de llamados a proyectos de co creación, con amplia participación de los actores de cada región del

127

pais, es de fundamental importancia. Pues solo con el involucramiento participativo de los actores de cada región geográfica es posible generar confianza, luego adherencia al proyecto y finalmente sentirse orgullosos de los logros del trabajo en equipo, participativo, que no solo busca el resurgimiento de las pymes, sino que además siembre la cultura de sostenibilidad en línea con los ODS, y la cultura de la legalidad mitigando los casos de corrupción e impunidad, a través de las buenas prácticas empresariales y personales como ciudadanos. Me suscribo para emprender y caminar este sendero de la mano de ustedes. Considero que el llamado a presentar proyectos de co creación es el camino correcto e infalible.

● UN can do more on business development services, including as a follow and complementary to behavioural methodology of the Empretec programme. More efforts could be done to bring together and coordinate MSME related policies and activities of different agencies at a national level.

● UN should develop core team to discuss with developing economies countries the best practices across glove for MS ME and help them to have online real time repository of MSMEs, their issues and resolution mechanisms. UN can support this activity.

● work more with chambers of commerce

● yeah UN should allow local national and INGO to acquire small funding instead and support accept taking risk organization if the country wants move together localization agenda.

● Yes

● Yes

● Yes This is because the COVID-19 affected a lot of MSMEs especially we at the northern part of Ghana

● Yes by presenting international benchmarks for EE/RES through workshops and supporting countries for secondary legislation for EE/RES

● Yes keep training members of MSMES

● yes there are many 1.funding 2.couching after trainings will be useful. 3.capacity-building 4. study tours to learn from other countries.

● Yes through the trainings and expertise availability for the better and proper enforcement and participation of the policies

● Yes through training in entrepreneurship.

● Yes,

● Yes, because most of us were train without finding so we plead that next time small Grant be given for a start up

● Yes, in the universities

● Yes, Kenya. A review to establish how public policy and competition policies support or create challenges to MSME competitiveness and sustainability.

● Yes, most beneficial of this coordinators don't lodge the beneficiary account. I am on opinion that individual account should be credited direct. Because I didn't benefit from the programme too

● Yes, pick up from the recommendations of the project and and start engagements with stakeholders that promote and develop the MSE sector such as the Micro and Small Enterprise Authority in Kenya

● Yes, targeted resources for capacity-building and training on how such models as cooperatives can be suitable vehicles for formalization.

● Yes. Huge amount of options. Starting with eliminating useless jobs that we have in local UN. Some projects has admin costs that never have given any result.

● Yes. It's necessary to give stability to the incubators. And may be that UN can promote and support that initiative face the government

● Yes. Using technologies requires not only capacity. Resources are needed to purchase the necessary gadgets. On top of this, piloting a project in the country can increase replication potential if results are positive. Learnings and practice can help shape policy at national level. If UN can fund a pilot project where learnings are generated, that is ideal in Zimbabwe.

128

Final questions 11. In what type of organization do you work?

Type of organisation Total

Government 28

Micro, Small or Medium Enterprise (MSME) 17

Other private sector, Business or industry association, Commercial Registry Office 20

Other (please specify) 43

Other (please specify)

● Academe

● Academe

● Academic

● Association

● CAD Internazionale ● Centro de Estudios Tecnología e Investigación: Es una consultora privada pero vinculada a temas

de investigación con la Universidad Católica

● Consulting Company

● CONTADOR PUBLICO INDEPENDIENTE

● Cooperative Development

● Cooperatives

● EMPRETEC CENTRE

● Farmer ● Former civil servant of the Ministry and now retired and independent consultant in the Mining

sector

● Former Head of Branch at UNCTAD responsible for this project until 1 May 2021

● H

● Have worked in sector now advisory role

● I am a Business Advisor

● Independent business development consultant

● INGO

● International NGO working in Sustainable Development

● International Organisation

● International organisation

● International Organization

● International Organization

● International organization

● ITC

● NGO

● NGO /Expert

● None

● Nongovernmental Organization

● Public University

● Research institution

● SOCIOLOGIST in OR at FRANCE

129

● Think tank

● UN

● Un

● UN

● UN

● UN agencies

● UNDP

● United Nations

● United Nations entities

● University 12. What country do you represent?

Country Total

Albania 2

Armenia 3

Bangladesh 1

Bhutan 1

Botswana 1

Brazil 2

Burkina Faso 1

Cameroon 1

Colombia 1

Dominican Republic 1

El Salvador 2

Ethiopia 2

France 1

Gambia 3

Germany 1

Ghana 1

Guatemala 2

Honduras 1

India 2

Indonesia 1

Italy 1

Kenya 7

Kyrgyzstan 1

Lesotho 3

Macedonia 2

Madagascar 2

Malawi 1

Mexico 2

Moldova 3

Mongolia 1

130

Namibia 1

Nigeria 10

Pakistan 1

Paraguay 1

Peru 2

Philippines 5

Rwanda 2

Senegal 1

Serbia 2

Somalia 1

South Africa 3

South Sudan 1

Swaziland 1

Sweden 1

Tajikistan 1

Thailand 3

Togo 1

Uruguay 1

USA 2

Venezuela 1

Vietnam 1

Zambia 3

Zimbabwe 3

Total 101 13. What is your gender?

Gender Total

Female 42

Male 62

Other/Prefer not to say 1

Total 105 14. Do you identify as part of any of the following groups?

Type of vulnerable group Total

Youth 30

Indigenous 17

LGBTQ+ 0

Person with disabilities 2

Survivor of violence (domestic, gender-based, other) 2

Other vulnerable group (please specify) 24

131

Other vulnerable group (please specify)

● Above youth 1

● Academic 1

● Adilts 1

● Adulto mayor 1

● Adulto maytor 1

● Cross Border Traders 1

● Estado con alto índice de marginalidad económica y tecnológica. Oaxaca, México. 1

● Executive 1

● Grassroots women 1

● N/A 1

● NA 1

● No 4

● Non of the above 1

● None 2

● Pashtun Ethnic Minority 1

● People living with facial palsy 1

● Previously disadvantaged individual 1

● SIMPLE CITIZEN 1

● Single Mother 1

● Smallholder farmer 1 15. Are there any final comments that you would like to share about the project to help inform this

evaluation?

● A well curated and well executed project. We hope to have more opportunities like this.

● Agradecer al proyecto por habernos incluido

● Any

● Continue the good work.

● Does it matter? Have you ever considered anything said? Nope? What a surprise.

● en Uruguay para acceder a financiamientos se requiere información contable de acuerdo a NIC. La iniciativa es muy buena para que el microempresario cuantifique resultados u ordene sus finanzas internamente , pero a nivel país debe regirse por régimen reglamentado gracias por la generosa oportunidad que me brindaron y el excelente trabajo desarrollado

● Great programme it was

● I am hosting regular talks and training workshops and would be delighted to have a speaker from your office, and hopefully even a collaboration

● I enjoyed managing the project.

● I really need a financial support to support my MSMEs in the northern part of Ghana. Majority of their businesses have collapsed and it is making cost of living very difficult.

● I think everything is good

● In my opinion, in the implementation of UN projects, the majority of experts who could be engaged or active are not too interested in directly participating (lack of general knowledge, language, personal - direct engagement, somewhat weaker departmental communication in the state and private sector, etc.), which ultimately (in my opinion) affects the level of quality and the final design of the project.

● Is a good information for me

132

● It would be helpful for us if we are able to make partnership with UN Women or any UN agency.

● Keeping in touch with MSMEs

● Let's the poor BREATHE

● Many thanks and keep up the good work....while improving lives

● Many thanks for supporting for this work

● Me encantó la experiencia.

● Me ha gustado la capacitación, tal vez en otras similares se podría trabajar también con espacios de ILab, aplicando Productos Mínimos Viables, a través de interacciones de los participantes de diferentes países. Otra sugerencia es que puedan dar espacios para fortalecer los temas aprendidos en los 3 cursos con becas de CEPAL, generando llamados a proyectos de co creación en donde los participantes de los cursos puedan conseguir mayor permeabilidad respecto a los temas aprendidos, y así generar la sinergia necesaria con recursos y presencia de marca de ustedes. Todo lo aprendido puede complementarse con el uso de herramientas AGILE, y aplicando tecnologías de bajo costo y alto impacto para permear en forma sinérgica al campo o regiones de toda la riqueza aprendida en los 3 cursos.

● more successes

● More training The evaluation should not take a longtime to come evaluators to avoid the risk of forgetting the content

● N/A

● NA

● No

● None

● None

● Not really

● Not sure

● Personalmente agradezco por tomarme en cuenta para esta escuesta

● possibility of collaboration

● Share the project

● Thank you for the opportunity to participate in the training, and for the follow up

● Thanks for considering sharing the survey to get my feedback

● Thanks for the opportunity

● Thanks for the survey.

● Thanks for this program

● That all

● The project and upcoming successors should be of long-term with possible long-term financing to support infrastructure development, Capital funding support and long-term Capacity-building activities.

● the project helped to reveal real problems for small businesses and develop recommendations for overcoming them

● The project was a great opportunity to promote and support the MSME sector and Empretec network during COVID 19. Wish there were more projects like this one.

● The project was pro-poor and appropriate.

● The project would also cover on areas of sustainability, institutional support for continuety of the project, supporting material and equipment.

● The projection should initiate inclusive capacity-building's courses / sessions within regions and/or grassroot communities.

● The UN should incorporate NGOs into the project for fruitful outcomes.

133

● There are lots of information to share with you but it is confidential, that is one on one.

● This is a wonderful initiative, please continue with it as it sparks the slumbering creativity to some of us. Also, it shaping the communities to be better

● This project should deal with the individual groups not there heads by so doing you are dealing with this group of people directly

● Those trainings should be conducted in-person or hybrid inorder to make them more interactive, focused and effective

● training of smses

● Unicamente agradecer el apoyo que brindan

134

Annex 7: Evaluation Terms of Reference

Evaluation of the United Nations Development Account 12th tranche “Global Initiative towards post-Covid-19 resurgence of the MSME sector” (2023W)

TERMS OF REFERENCE

A. BACKGROUND A1. ABOUT THE DEVELOPMENT ACCOUNT

The Development Account (DA) is a mechanism to fund capacity development projects of the 10 economic and social entities of the United Nations Secretariat, namely: the Department of Economic and Social Affairs (DESA), the Economic Commission for Africa (ECA), the Economic Commission for Europe (ECE), the Economic and Social Commission for Asia and the Pacific (ESCAP), the Economic and Social Commission for Western Asia (ESCWA), the Economic Commission for Latin America and the Caribbean (ECLAC), the United Nations Conference on Trade and Development (UNCTAD), the United Nations Environment Project (UNEP), the United Nations Human Settlements Project (UN-Habitat) and the United Nations Office on Drugs and Crime (UNODC).

The DA provides capacity development support to developing countries in their implementation of the 2030 Agenda for Sustainable Development, as well as recommendations and decisions made in intergovernmental processes and relevant governing bodies. The DA-funded projects build on the mandates, individual technical capacities and comparative advantages of the respective implementing entities, while providing those mostly non-resident entities with the ability to operationalize their knowledge and know-how to deliver capacity development support at regional, sub-regional and country levels.

The Under-Secretary-General (USG) for Economic and Social Affairs is designated as the Project Manager of the Development Account with responsibility for overall coordination, programming, monitoring and evaluation, as well as for reporting to the intergovernmental bodies. The Project Manager is supported by the DA Steering Committee, who advises him/her on strategic policy and project-support matters.166 The Project Manager is also supported by the DA Project Management Team (DA-PMT) located within the Capacity Development Programme Management Office (CDPMO) of DESA, which assists with all aspects of the management of the DA, in particular with regard to programming, monitoring, evaluation and reporting. DA-PMT also liaises with the DA Focal Points in the implementing entities, who are most often the head of the entity’s unit responsible for project planning, project management, capacity development or technical cooperation, on all aspects of the management of DA-funded projects.

In response to the COVID-19 crisis, the Development Account has funded five short-term joint projects to help developing countries alleviate the socio-economic impact of the pandemic, including the project on Global initiative towards post-COVID-19 resurgence of the MSME sector (2023W).

A2. ABOUT THE PROJECT

166 The DA Steering Committee is composed of five members with one member representing each of the following implementing entities and key stakeholders: 1. DESA; 2. the regional commissions; 3. UNCTAD; 4. UNEP, UN-Habitat and UNODC (on a rotational basis); and 5. the Programme Planning and Budget Division (PPBD) of the Office of Programme Planning, Finance and Budget of the Department of Management Strategy, Policy and Compliance.

135

The COVID-19 crisis is plunging the global economy into a deep recession and micro, small and medium enterprises – which play a major role in emerging economies – are amongst the hardest hit. Trapped in economic stagnation due to large-scale lockdowns, millions of MSMEs have become the most vulnerable to COVID-19 within the private sector. Compared with large firms, small businesses have fewer resources and lower capacities to cope with the abrupt economic shocks economies are currently facing. With more than two-thirds of the global population employed by MSMEs, the unprecedented outbreak of the pandemic has vividly shown how tightly their activities are woven into the economic and social fabric of the world, as well as their critical role in social and economic resurgence.

The objective of the project is to develop and implement capacity-building tools for governments and micro, small and medium enterprises (MSMEs) to facilitate the resurgence and strengthen the resilience of MSMEs in developing countries and economies in transition. These capacity-building tools will seek to mitigate the economic and social impact of the global COVID-19 crisis and to facilitate the contribution of MSMEs to the SDGs implementation.

The project was designed based on the request for assistance for MSMEs from more than 50 Member States, including countries in Latin America and the Caribbean, Asia and the Pacific, Africa, Europe and the Arab regions, as well as intergovernmental demands and resolutions on COVID-19.

The project is jointly implemented by UNCTAD, DESA, ECA, ECE, ECLAC, ESCAP and ESCWA, and builds upon the comparative advantages of the participating agencies to provide immediate advice, capacity-building and support to governments and MSMEs during the ongoing global pandemic. The project is structured in five clusters that address the most critical areas of the MSME recovery. Broadly, the roles and lead entities for each of the clusters and workstreams are as presented in the following table:

Table 1. Project clusters and leads

Project cluster/workstream Lead agency Overall coordination UNCTAD Project cluster/workstream 1 Entrepreneurship and business skills promotion UNCTAD

Project cluster/workstream 2 Business facilitation/formalization

UNCTAD/DESA

Project cluster/workstream 3 Access to finance/financial literacy ESCAP

Project cluster/workstream 4 Access to technology and innovation UNECA

Project cluster/workstream 5 Access to markets UNECE

The beneficiary countries cover different geographical regions, as shown in Annex 2. The expected outcomes, indicators of achievement, and outputs are presented in the project results framework (Annex 3). The project was developed and implemented under three phases. A new set of outputs was designed or added at each of the three phases of the project.

Under the three-phase approach, the project budget was approved by phase. In 2021, when the phase 3 budget was discussed, the Development Account faced a funding gap. To bridge the gap, in November of the same year, the five joint projects were requested to reduce their proposed phase 3 budget by 1 million USD, which led to the curtailment of certain planned activities. For this project, the budget was reduced by $310,000.

Overall, a total of $4,490,500 was allocated under this project. Concretely, UNCTAD received $2,671,000, ECLAC received $134,000, ESCAP received $240,000, ECA received $467,000, ECE received $448,500, ESCWA received $370,000 and DESA received $160,000.

136

The project started its implementation in May 2020 and was scheduled to conclude on 31 March 2022, but received approval in February 2022 for an extension until 30 June 2022.

B. EVALUATION OBJECTIVES AND SCOPE

B1. EVALUATION PURPOSE, OBJECTIVES AND SCOPE The present evaluation will constitute a terminal evaluation of the Project. Terminal evaluations are mandatory for all DA-funded projects with a value above $1 million. The evaluation will be largely guided by the UN Development Account Project Evaluation Guidelines, issued in October 2019 and the evaluation policies of the implementing entities, in particular, UNCTAD, which leads the evaluation.

The main purpose of the evaluation will be to support accountability for results, and to enable learning.

This terminal evaluation of the project has the following specific objectives:

- Assess the results and establish the link between achievements and activities of the intervention;

- Assess the response delivery and external coordination167, including the extent of gender, human rights and disability mainstreaming; and

- Identify good practices and lessons learned from the project that could feed into and enhance the implementation of related interventions.

The primary intended users of the assessment are the management of the implementing entities. The evaluation will also provide accountability to project beneficiaries and member States. Furthermore, the evaluation will form a key input to the programme-level evaluation of the DA’s response to COVID-19 to be initiated by the CDPMO/DESA. The programme-level evaluation will entail: a synthesis of the terminal evaluations of five COVID-19 joint DA projects, including this project; a review of relevant 10th and 11th tranche DA projects; and a programme-level assessment. The primary audiences of the programme-level evaluation will include the DA Steering Committee, the DA-Programme Management Team (DA-PMT), and the management of the implementing entities. The results of the programme-level evaluation will also be presented to the General Assembly, through the biennial progress report on the implementation of the DA.

The evaluation will cover the duration of the project from May 2020 to 30 June 2022, covering all phases, clusters and activities.

B2. EVALUATION CRITERIA AND QUESTIONS The evaluation will assess the Project’s performance against the main criteria of relevance, efficiency, effectiveness, coherence, sustainability, gender, human rights and disability. In particular, the evaluation is expected to address a number of questions under the following criteria168:

Table 2: Evaluation criteria and tentative questions

167 The OIOS COVID-19 response evaluation protocol identifies the following three cross-cutting focus areas: 1) response delivery; 2) external coordination (or “Delivering as one”); and 3) business continuity. “Response delivery” is further defined as consisting of delivery of: 1) the existing mandate needed to implement previously mandated activities in the new environment created by the pandemic; and 2) the COVID-19-specific response (health and non-health) needed to address the pandemic specifically. See OIOS (October 2020), “COVID-19 Response Evaluation Protocol”, para 3-4. 168 The evaluation questions were developed as part of the “Proposed approach, scope and questions to the Evaluation of the United Nations Development Account’s Response to COVID-19”, which was jointly developed by the DA-PMT and select implementing entities in the spring of 2020. The document is designed to guide both the terminal evaluations of the five COVID-19 joint projects and the programme-level evaluation of the DA’s response to COVID-19, and is expected to be updated later in 2022 to reflect the confirmed approaches and timelines for the terminal evaluations of the five projects.

137

Relevance 1. To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID-19?

Relevance 2. To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g. COVID-19 Socio-Economic Response Plan)?

Efficiency 3. How well coordinated was the response among the entities implementing the joint project?

Efficiency 4. How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

Effectiveness 5. To what extent did the programme (Development Account) and project governance and management structures and processes enabled, or hindered, the effective implementation of the joint project and the achievement of its results?

Effectiveness 6. To what extent has the project contributed to the expected outcomes as enunciated in the project document?

Effectiveness 7. How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

Effectiveness 8. What innovative approach or tool, if any, did the response use, and what were the outcomes and lessons learned from its application?

Sustainability 9. What measures were adopted to ensure that the outcomes of the response would continue after the project ended?

Coherence 10. To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

Coherence 11. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID-19 in delivering socio-economic support to Member States?

Gender, human rights and disability

12. To what extent were gender, human rights and disability perspectives integrated into the design and implementation of the project? What results can be identified from these actions?

C. EVALUATION APPROACH AND METHODOLOGY The evaluation will be a transparent and participatory process involving the Project’s implementing entities and key stakeholders. It will be conducted based on gender and human rights principles and adhere to the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation.

The evaluation will apply a mixed-method design, including a combination of quantitative and qualitative analysis to inform findings.It is anticipated that travel of the evaluation team may take place in support of elaborating case study/ies, as well as to meet key project stakeholders in Geneva (UNCTAD and ECE). The selection of potential case study/ies and travel requirements will be developed as part of the inception report.

Following a preliminary documentation review and a limited number of inception meetings with the core project team, the Evaluation Team will develop an inception report for the evaluation, which will include the finalized overall scope and focus of the evaluation, evaluation questions and methodology, including

138

information on data sources and collection, sampling, key indicators, stakeholder mapping, selection of case study/ies, survey design, and the evaluation timeline.

The tentative methodology for the evaluation is presented in Table 4.

Table 3: Tentative methodology for the assessment

a) A desk review of Project documents, including documents/data related to:

o Project-level planning, implementation and results achievement, including but not limited to:

▪ Concept note, Phase 2 project proposal, and Phase 3 budget and outputs

▪ Progress report for Phases 1 and 2 (both financial and substantive/narrative report)

▪ Final report (both financial and substantive/narrative report)

▪ Meeting minutes, including the minutes of the bi-weekly/monthly DA network meetings

▪ Monitoring reports

▪ Information on non-DA resources, financial and in-kind, brought in by the participating entities

▪ Information on resources, financial and in-kind, contributed by partners/donors (including

information requested under the “supplementary funding” section in the progress reports,

which is often incomplete)

▪ Beneficiary/user feedback collected, including, but not limited to, workshop survey results,

user feedback on publications, advisory services, guidelines, methodology documents, etc.

▪ Requests for assistance/services received

▪ List of activities completed and details about each activity, including but not limited to:

- Agenda, participant lists (name, title, division/unit, organization, country, gender,

email address), report and any outcomes document, for each workshop/meeting

- Description of each advisory service, beneficiaries (including contact details of the

contact persons) and any outputs/deliverables produced

- List and description of tool(s), research papers, policy briefs, studies published and

information on how each product was disseminated and/or used, list of

recipients/users of the product (e.g., dissemination lists)

▪ Documentation related to broader projects or sub-projects of the participating entities of

which the Project or its component(s) has constituted an integral part or which are linked to

and/or build upon/succeed the work undertaken as part of the Project

▪ Documents and literature related to the Project context

▪ Relevant web and social media metrics related to the outputs of the project;

o Project strategic documents, including but not limited to:

▪ General Assembly's Resolution on Global Solidarity to fight the coronavirus disease 2019

(COVID-19) (A/RES/74/270);

▪ Secretary General's report on "Shared responsibility, global solidarity: Responding to the socio-

economic impacts of COVID-19";

▪ UN Framework for the immediate socio-economic response to COVID-19;

▪ 2021 Programme budget and mandate of implementing entities;

▪ COVID-19 Socio-Economic Response Plan of participating countries.

b) Questionnaires/surveys (in appropriate languages in addition to English) to relevant stakeholders in countries participating in a sample of project activities;

c) Telephone, online or in-person interviews with key stakeholders, including but not limited to:

139

o Project Coordination Team and project focal points of implementing entities

o DA-PMT

o DA focal points in participating entities

o Sample of UN Resident Coordinators/Country Teams, as appropriate

o Sample of key global partners

o Sample of country-level stakeholders (mainly stakeholders from key beneficiary countries)

d) Case Study/ies, which may include for example a detailed examination of a particular intervention, or of project activities at a regional or national level.

In addition to assessing the mainstreaming of gender, human rights and disability perspectives in the design, implementation and monitoring of the Project (evaluation question 12), the evaluation will integrate these perspectives in the management of the evaluation, data collection and analysis, as well as the development of the evaluation report. Gender balance will be given full consideration in the composition of the Evaluation Reference Group, elaborated in Section D1 (Evaluation management), and the Evaluation Team. Data collected and analyzed in the course of the evaluation will be disaggregated by gender to the extent possible and whenever appropriate, and the evaluation findings, conclusions and recommendations to be presented in the evaluation report will reflect a gender analysis.

The evaluation will be carried out according to the UNEG ethical principles and standards.169 The evaluators should demonstrate behavioural independence, impartiality, credibility, honesty, integrity and accountability in conducting the evaluation/assessment to avoid biasing the findings. The evaluators must also address in the design and conduct of the evaluation procedures to safeguard the rights and confidentiality of information providers. The Evaluation Manager will be responsible for ensuring that the Evaluation Team conducts the work assignments without any undue interference from those who were responsible for the implementation of the Project.

D. ORGANIZATION OF THE EVALUATION

D1. EVALUATION MANAGEMENT The independent final project evaluation will be managed/coordinated by UNCTAD’s Independent Evaluation Unit, with the support of an Evaluation Advisory Committee (EAC) that comprises a representative each of the evaluation units of the partner entities (DESA, ECA, ECE, ECLAC, ESCAP, ESCWA) and the Evaluation Officer with the CDPMO/DESA. The EAC primarily serves a quality assurance function and facilitates support to the Evaluation Team as necessary.

An Evaluation Reference Group (ERG) consisting of a representative from each UN partner entity (e.g., the DA Focal Point for each entity) and the DA-PMT will review and contribute inputs to key steps in this evaluation such as the TOR and draft final report.

Both the EAC and the ERG commit to submitting substantive comments on a timely basis, and comments will be invited on a ‘non-objection’ basis (no response = agree) so that the process is not delayed for an unnecessarily long time.

An independent Evaluation Team will be convoked to undertake this assignment. The Evaluation Team (ET) is responsible for conducting the evaluation, applying the methodology as appropriate and for producing the evaluation report. All team members, including the Team Leader, will participate in briefing and debriefing meetings, discussions, and will contribute to the evaluation with written inputs for the draft and final report. A selected number of the evaluation team members will participate in the mission travel(s) if applicable. The ET and the Evaluation Manager will agree on the outline of the report, in consultation with the EAC early in the evaluation process. The ET will develop its own evaluation tools and framework, within

169 UNEG (2020), Ethical Guidelines for Evaluation; UNEG (2008), Code of Conduct for Evaluation in the UN system

140

the available timeframe and resources. The team is fully responsible for its report, which may not reflect the views of any of the implementing entities of the project. The evaluation report is subject to quality control by the Evaluation Advisory Committee and clearance by the Evaluation Manager, as set out above.

The Team Leader guides and coordinates the team member(s) in their specific work, discusses their findings, conclusions and recommendations and prepares the draft and the final report, consolidating the inputs from the team member(s) with his/her own. The members of the evaluation team should possess a mix of evaluation skills and technical or sectoral/thematic knowledge relevant to the evaluation. In putting together the team, adequate linguistic, geographic and gender representation will also be key considerations.

The Evaluation Team will be provided full access to all project reports, documentation, and stakeholder lists and contact information. The Project Coordination Team are required to submit to the evaluation manager project documentation, including data and information residing with the other participating entities, in the last month of the project if possible, if not, immediately following the completion of the project, as well as support the evaluation process, including through facilitating the evaluators’ access to the project’s beneficiaries and other key stakeholders.

The roles and responsibilities in the evaluation process are described below:

Evaluation Manager (UNCTAD) will: ● Prepare the draft evaluation TOR and revise/finalize based on inputs received

● Prepare the TOR for each member of the Evaluation Team (Team Leader, Team Member and Expert(s))

● Recruit and manage the Evaluation Team

● Backstop the evaluation process, including supporting the development and administration of surveys,

support outreach of the evaluation team to project stakeholders, and access to secondary data listed in

Table 3.

● Oversee/provide quality assurance to the evaluation and the development of the evaluation report

● Facilitate the work of the Evaluation Advisory Committee and the Evaluation Reference Group

● Be responsible for clearance of the evaluation report

● Support the development of a management response to the evaluation report, including an implementation

plan

● Organise a virtual workshop on evaluation findings and lessons learned.

Evaluation Advisory Committee comprises a representative each of the evaluation units of the partner entities (ESCWA, ECA, ECE, ECLAC, ESCAP, DESA) and the Evaluation Officer with the CDPMO/DESA. The EAC primarily serves a support and quality assurance function. Specific responsibilities of the EAC include: ● Review and approve the evaluation TOR;

● Advise on the selection of the evaluation consultant(s) to ensure that the selection is based on the required

skills and qualifications;

● Support facilitating access from their respective entities to relevant project documentation and

stakeholders;

● Review and comment on the inception and evaluation reports; and

● Monitor and conduct periodic follow-ups on the implementation of evaluation recommendations

addressed to the parties within their entities.

Evaluation Reference Group, consisting of a representative from each UN partner entity (e.g., the DA Focal Point for each entity) and the DA-PMT,170 will review and contribute inputs to key steps in this

170 While multiple representatives of DA-PMT may attend meetings of the Global Reference Group, reflecting different roles held by each staff in relation to the Programme, DA-PMT will provide one consolidated written input as part of the review of the draft inception report and the draft evaluation report.

141

evaluation such as the TOR and draft final report. The ERG’s key function is to enhance the relevance, credibility and transparency of the evaluation process. Specific responsibilities include: ● To review the draft evaluation ToR and provide substantive feedback;

● To facilitate access from their respective entities to relevant project documentation and stakeholders;

● To review the draft evaluation report and provide substantive feedback, including coordinating feedback

from other sections, units and offices from headquarters and from the field to ensure quality and

completeness;

● To participate in the validation meeting of the final evaluation report;

● To play a key role in disseminating the findings of the evaluation and implementation of the management

response.

Project Coordination Team will: ● Facilitate the Evaluation Team’s access to relevant Project documentation and stakeholders, including

through:

o Collecting and compiling requested data and information from the participating entities, as

requested by the Evaluation Manager

o Providing an updated list of stakeholders, and facilitating access to the sample of stakeholders that

the Evaluation Team may wish to interview

o Facilitating the administration of questionnaires to workshop participants in the participating

countries

o Ensure the cooperation and contribution of the relevant staff of the implementing entities to the

evaluation process, as requested

● Lead the preparation of a response to the recommendations directed to the participating entities, including

an implementation plan

DA-PMT will: ● Participate in the Evaluation Reference Group

● Provide guidance on the allocation of the evaluation budget

● Organize a virtual meeting with DA focal points to discuss the key lessons from this evaluation as well as from other COVID-19 joint project evaluations and how to incorporate them in future programming, implementation, monitoring and evaluation of DA-funded projects.

D2. EVALUATION TIME FRAME The evaluation will be conducted from December 2022 to August 2023.

The evaluation process will involve five phases with the tentative timelines as below in Table 5 (the timelines may be adjusted should any exigencies arise):

Table 5: Evaluation phases and tentative timelines

Phase Timelines

1. Preparation

August 2022 – December 2022 ● Preparation and finalization of evaluation TOR ● Establishment of the Evaluation Reference Group ● Recruitment of the Evaluation Team ● Prepare package of documents required by the

Evaluation Team

2. Inception

December 2022 -March 2023

142

● Preliminary documentation review and preparation of inception report by the Evaluation Team, including development of data collection instruments (questionnaires/ surveys, interview guides)

● Brief visit to Geneva (3 days) to meet with UNCTAD Evaluation Unit and key project stakeholders (UNCTAD and ECE)

● Draft inception report due: 10 February 2023 ● Evaluation Manager review and inception report

revision by Evaluation Team: 13 - 22 February 2023 ● Reviews by Project Coordination Team, project focal

points of implementing entities, and Evaluation Advisory Committee (in parallel): 23 February – 7 March 2023

● Draft final inception report due: 14 March 2023 ● Final inception report approved: 20 March 2023

3. Data collection and analysis March- May 2023

● Desk review of remaining Project documents, including requesting additional documentation

● Online surveys of stakeholders ● Interviews with stakeholders ● Data analysis and triangulation

4. Report preparation and reviews May – July 2023

● Data analysis and triangulation ● First draft evaluation report due: 16 June 2023 ● Evaluation Manager review and report revision by

the Evaluation Team: 19 – 28 June 2023 ● Reviews by Project Coordination Team, project focal

points of implementing entities, Evaluation Advisory Committee and Evaluation Reference Group (in parallel): 29 June – 11 July 2023

● Revised draft evaluation report due: 18 July 2023 ● Final evaluation report with annexes: 25 July 2023

5. Dissemination and follow-up August 2023 and onwards

● Presentation to the Project Coordination Team, project teams of implementing entities and development and approval of a management response, including an implementation plan for recommendations

● Virtual workshop on evaluation findings, lessons learned and follow-up with the DA Focal Points: April 2023

D3. EVALUATION TEAM DELIVERABLES The Evaluation Team will be composed of a team of three consultants (evaluators), namely Team Leader, Team Member and a Gender and Human Rights (HRGE) Expert who also plays the role of Team Member. The two Team Members will report functionally to the Team Leader. The Team Leader will report to the Evaluation Manager. Each of the Evaluation Team has a set of deliverables as described below:

143

Deliverables for Team Leader and Team Member

o Initial review of key Project documents (preliminary document review)

o Preparation of an inception report with a finalized evaluation scope and focus, evaluation questions and methodology, including information on data sources, sampling and key indicators, stakeholder mapping/analysis, selection of case study/ies, as well as survey design

o Desk review of remaining Project documents

o Data collection and analysis based on the finalized methodology

o Preparation of an evidence matrix presenting a summary of evidence collected through each data collection method by evaluation question

o Development of a draft evaluation report, based on the template presented in Annex 1, for review by the Evaluation Manager, Project Coordination Team, project focal points of implementing entities, the EAC and the ERG

o Revision/finalization of the evaluation report, including all annexes, based on comments received

o Preparation of a 3-page summary of the evaluation report and a presentation (PPT) on key findings, conclusions and recommendations

o Presentation of evaluation report and discussions with relevant stakeholders such as Project Coordination Team, project teams of implementing entities, DA focal points of participating entities and DA-PMT.

Deliverables for Gender and Human Rights Expert/Team Member

o Initial review of key Project documents (preliminary document review), including identifying gender equality, human rights and disability inclusion dimensions and issues for consideration;

o Preparation of an inception report with a finalized evaluation scope and focus, evaluation questions and methodology, including information on data sources, sampling and key indicators, stakeholder mapping/analysis, selection of case study/ies, as well as survey design. Where applicable, gender equality, human rights and disability inclusion considerations will be integrated in the evaluation scope of analysis; evaluation criteria and questions design; methods and tools, and data analysis techniques;

o Desk review of remaining Project documents;

o Data collection and analysis based on the finalized methodology which would be gender sensitive;

o Preparation of an evidence matrix presenting a summary of evidence collected through each data collection method by evaluation question;

o Development of a draft evaluation report, based on the template presented in Annex 1 of the Terms of Reference for the evaluation, for review by the Evaluation Manager, Project Coordination Team, project focal points of implementation entities, the EAC and the ERG. The analysis of gender equality, human rights and disability inclusion dimensions and issues should be integrated in the report as an independent section and to the extent possible, these issues should be mainstreamed throughout the report, including in the evaluation findings, conclusions and recommendations.

o Revision/finalization of the evaluation report, including all annexes, based on comments received;

o Preparation of a 3-page summary of the evaluation report and a presentation (PPT) on key findings, conclusions and recommendations.

o Presentation of evaluation report and discussions with relevant stakeholders such as Project Coordination Team, project teams of implementing entities, DA focal points of participating entities and DA-PMT.

144

E. COMMUNICATIONS AND DISSEMINATION PLAN: The results from the evaluation including key lessons learned, best practices and recommendations will be shared widely with participating entities, partners and stakeholders, and member States. In particular, the following modes of communication could be used:

e) A workshop with all relevant stakeholders to present the key findings, recommendations and lessons learned. The evaluation report will be presented at a workshop attended by the implementing entities, the DA-PMT and other relevant stakeholders for discussion and validation. The implementing entities will be given the opportunity to present their management response, including an implementation plan for the recommendations;

f) A separate virtual meeting will be held with the DA focal points to discuss the key lessons from the evaluation as well as from other COVID-19 joint project evaluations and how to incorporate them in future programming, implementation, monitoring and evaluation of DA-funded projects and projects.

g) A copy of the final evaluation report will be published on UNCTAD’s website and the websites of the partner implementing entities, as appropriate; and

h) The key findings from the evaluation report will also form a key input to the programme-level evaluation of the DA’s response to COVID-19 to be initiated by the CDPMO/DESA.

i) Other communication briefs and products will be produced as appropriate.

  • Lists of figures, tables, and boxes
  • List of Acronyms and Abbreviations
  • Executive summary
  • 1. Introduction
  • 2. Description of the Project
    • 2.1 Background
    • 2.2 Project objectives and expected accomplishments
    • 2.3 Project strategies and key activities
    • 2.4 Beneficiaries and target countries
    • 2.5 Key partners and other key stakeholders
    • 2.6 Resources
    • 2.7 Link to the Sustainable Development Goals (SDGs)
    • 2.8 Innovative elements
  • 3. Evaluation objectives, scope, and questions
    • 3.1 Purpose and objectives
    • 3.2 Evaluation scope, criteria, and questions
  • 4. Methodology
  • 5. Findings
    • 5.1 Relevance
    • 5.2 Coherence
    • 5.3 Efficiency
    • 5.4 Effectiveness
    • 5.4 Sustainability
    • 5.5 Gender, Human Rights, and Leave No One Behind
  • 6. Conclusions
  • 7. Recommendations
  • Annexes
    • Annex 1: Project’s Theory of Change
    • Annex 2: Evaluation matrix
    • Annex 3: Data collection instruments
    • Annex 4: List of documents reviewed
    • Annex 5: List of individuals interviewed
    • Annex 6: Evaluation surveys
    • Annex 7: Evaluation Terms of Reference

Presentation

Languages and translations
English

Classification of international migration in Russia based on administrative data

Olga Chudinovskikh Lomonosov Moscow State University, Faculty of Economics

.

1

Economic Commission for Europe Conference of European Statisticians Group of Experts on Migration Statistics

Geneva, Switzerland, 7−8 May 2024

In world practice, the description of immigration channels to a particular country is often based on statistics of residence permits, which are already sorted by a certain class at the stage of issuance.

2

The traditional approach to classifying classes of admission is to highlight migration: through family reunification, economic (labour) migration, students, refugees and some other categories

Understanding what legal “gates” for migration exist and which ones migrants choose most often allows us to assess the nature of migration policy (although this does not always reflect the individual’s reasons for move).

https://www.americanactionforum.org/series/building-a-pro- growth-legal-immigration-system/

Considerable gaps in understanding classes of admission of foreigners to residence and naturalization in Russia and possible solutions

• Russia is one of the main migrants receiving countries of the world

• Stock of international lifetime migrants in accordance with the UN DESA estimates amounted to 11,6 million in 2020 (United Nations Department of Economic and Social Affairs, Population Division (2020). International Migrant Stock 2020.)

• From 2010 to 2023, according to the Ministry of Internal Affairs of Russia, 5.6 million temporary and permanent residence permits (2.4 million and 3.2 million respectively) were issued in Russia, 4.4 million people were granted citizenship through the Ministry of Internal Affairs.

• However we still have a very vague understanding of what formal grounds are most often used when the authorities make decisions on granting foreigners a particular status.

• There are no aggregated statistics of this kind in Russia, although there is information that allows us to shed light on this important issue.

• The choice is the administrative data of the Russian Ministry of Internal Affairs on temporary residence permits, residence permits and citizenship.

• Despite the differences in approaches in different countries, these data have much in common and can be considered comparable. If there is a distribution by admission class, they provide an idea of ​​through which “gates” migrants are allowed to reside in the country.

• One more advantage: these are understandable and comparable statuses and data, they are relatively easy to interpret

3

The issue of definitions and comparability of data is acute in migration statistics. Unlike Rosstat data, statistics on residence permits and citizenship are relatively comparable in an international context

Access of foreigners to the right to reside and acquire citizenship in Russia: different populations receive temporary and permanent residence permit, as well as citizenship (only partial overlapping)

4

Temporary residence permit

Quota based

Quota-free (several categories of foreigners)

Participants of State program of compatriots resettlement

Citizenship acquisition Permanent residence permit

Based on temporary residence permit

Foreigners with permanent residence permit

Special order (the foreigners are not

migrants)

4

Access to citizenship without any residence permit

Permanent RP without temporary one

Citizenship without permanent residence permit

Permanent residence permit without the temporary one (many categories of foreigners)

Two basic laws define the statuses that give a foreigner the right to reside and acquire citizenship:

The Law on the Legal Status of Foreign Citizens (FZ-115) (to be amended) and

the Law on Citizenship of the Russian Federation (FZ-62 and from 26.10.2023 - FZ-138). (* Law on migrants’ registration … № 109)

• The law does not distinguish admission or compliance classes. And the positions are not in a "thematic" sequence.

• But one can see the commonalities in them and try to group them

Статья 8. Постоянное проживание иностранных граждан в Российской Федерации

ГАРАНТ:

См. комментарии к статье 8 настоящего Федерального закона

1. Вид на жительство может быть выдан иностранному гражданину, прожившему в

Российской Федерации не менее одного года на основании разрешения на временное проживание.

Информация об изменениях:

Статья 8 дополнена пунктом 1.1 с 1 января 2023 г. - Федеральный закон от 14 июля 2022 г. N 357-

ФЗ

1.1. Вид на жительство может быть выдан иностранному гражданину, получившему

разрешение на временное проживание в целях получения образования, после завершения обучения

в государственной образовательной или государственной научной организации по программе

бакалавриата, программе специалитета, программе магистратуры, программе ординатуры,

программе ассистентуры-стажировки, имеющим государственную аккредитацию, или по программе

подготовки научных и научно-педагогических кадров в аспирантуре (адъюнктуре).

2. Без получения разрешения на временное проживание вид на жительство выдается:

1) иностранному гражданину, родившемуся на территории РСФСР и состоявшему в прошлом

в гражданстве СССР;

2) иностранному гражданину, который не достиг возраста восемнадцати лет и родитель

(усыновитель, опекун, попечитель) которого является иностранным гражданином и постоянно

проживает в Российской Федерации;

3) иностранному гражданину, не достигшему возраста восемнадцати лет, получающему вид

на жительство совместно с родителем (усыновителем, опекуном, попечителем) - иностранным

гражданином;

4) иностранному гражданину, имеющему родителя (усыновителя, опекуна, попечителя), сына

или дочь, состоящих в гражданстве Российской Федерации и постоянно проживающих в Российской

Федерации;

5) иностранному гражданину, достигшему возраста восемнадцати лет, в соответствии с

законодательством иностранного государства признанному недееспособным либо ограниченным в

дееспособности, получающему вид на жительство совместно с родителем (усыновителем, опекуном,

попечителем) - иностранным гражданином;

6) иностранному гражданину, который достиг возраста восемнадцати лет, в соответствии с

законодательством иностранного государства признан недееспособным либо ограниченным в

дееспособности и родитель (усыновитель, опекун, попечитель) которого является иностранным

гражданином и постоянно проживает в Российской Федерации;

7) иностранному гражданину, признанному носителем русского языка в соответствии со

статьей 33.1 Федерального закона от 31 мая 2002 года N 62-ФЗ "О гражданстве Российской

Федерации";

8) иностранному гражданину, который сам либо родственник по прямой восходящей линии

Categories of foreigners eligible to get the status

5

Data description

• Statistical reports of the Ministry of Internal Affairs of Russia (formerly the FMS) in the form 2-RD

• "Registration and visa work" – based on the Law “On the legal status of foreigners ...”

• "Citizenship"– based on the Law on Citizenship; • There is also a report "External labor migration", not considered in this work • Microsoft excel file with two sheets - on one the data is distributed by

citizenship of migrants, on the second - by regions of Russia. • Distribution of procedures by individual positions of the law. • Although admission to citizenship is not actually migration, it can be

considered as its final phase - in the sense of changing the status of a migrant.

6

An example of the statistical report 2-RD with data distributed by individual provisions of the law “On Legal status of foreigners in the RF”

Abbreviations denote certain positions of the law "On the Legal status of Foreign Citizens": for example, paragraphs 3 and 5 of Part 3 of Article 6 of Federal Law No. 115 are minor children and adults who are incapacitated and receive an ETA together with a parent. Only similar items are grouped.

7

ФОРМА "2-РД" из них

за январь - декабрь 2021 года

РОССИЯ КОНТРОЛЬНОЕ ЗНАЧЕНИЕ 1-РД => 190 691 18 990 167 596 236 259

Код строки по 1-РД в3.20.1 в3.21.1 в3.22.1 в3.52.1

Код графы по 2-РД РВР15 РВР16 РВР17 РВР18 РВР19 РВР53 РВР54 РВР27 РВР23 РВР55 РВР24 РВР25 РВР56 РВР57 РВР29 РВР30

ВСЕГО c1 190 691 18 990 167 596 15 106 7 352 3 081 841 3 749 56 844 5 13 46 505 6 934 26 192 397 236 259

Абхазия 895 c2 69 7 85 9 7 1 1 4 61 0 0 0 2 0 0 205

Австралия 036 c3 22 5 18 5 0 0 0 1 12 0 0 0 0 0 0 24

Австрия 040 c4 18 0 16 4 2 0 0 0 10 0 0 0 0 0 0 40

Азербайджан 031 c5 11 357 1 120 9 990 937 407 193 50 174 7 161 0 1 617 401 0 0 23 255

Албания 008 c8 2 0 1 0 0 0 0 0 1 0 0 0 0 0 0 10

Алжир 012 c9 97 8 76 9 1 1 0 0 62 0 0 0 1 0 0 102

Американское Самоа 016 c10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Ангилья 660 c11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Ангола 024 c12 6 1 6 1 0 0 0 0 5 0 0 0 0 0 0 12

Антарктида 010 с259 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Андорра 020 c13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

подпункт

10

подпункт

11

подпункт

12

без учета квоты в соответствии с п. 3 ст. 6 Федерального закона от 25 июля 2002 г. № 115-ФЗ «О правовом положении иностранных граждан в Российской Федерации»

текущего

года

предыдущ

его года

подпункт

1

подпункт

2 и

подпункт

6

подпункт

3 и

подпункт

5

подпункт

4

подпункт

7

подпункт

8

подпункт

9

Р А З Р Е Ш Е Н И Я на В Р Е М Е Н Н О Е П Р О Ж И В А Н И Е

Раздел 1. Сведения о

результатах

разрешительно-визовой

работы

К о

д п

о О

К С

М

К о

д с

т р

о к и

Принято

заявлений

о выдаче

ИГ и ЛБГ

разрешени

й на

временное

проживани

е

Принято

решений о

выдаче

разрешени

й на

временное

проживани

е

в том числе Число ИГ

и ЛБГ,

имеющих

действите

льные

разрешени

я на

временное

проживани

е на конец

отчетного

в

пределах

установле

нной

квоты

текущего

года

в пределах установленной квоты

Paragraph 9 of Part 3 of Article 6 of Federal Law No. 115 - participants in the State Program of Assistance to Voluntary Resettlement of Compatriots (repatriates)

Temporary residence permits

Data limitations

• No detail in the statistics of temporary residence permits (TRP) until 2014, and residence permits until 2021 (after the transition of many categories of “TRP outside the quota” to “Permanent residence permit without temporary residence permit”

• 2019-2023 many changes in legislation, rapid expansion of the list of grounds for obtaining a residence permit, bypassing a temporary residence permit, over the past three years (issues of comparability and aggregation)

• The modernization of the form lags behind changes in legislation. There is a problem of large uncategorized balances; feedback from the data producer is not easy.

When legislation changes (in recent years - intensively), a regrouping and redistribution of migrants occurs through other formal channels, When a migrant chooses the “gate” of admission to status, considerations of ease of achieving the goal dominate, rather than the accuracy of “hitting” the relevant clause of the law.

Classification results

9

Class of admission Categories included Peculiarities or limitations

Family reunification Children, parents, spouses, persons under

guardianship

No (not counting family members included in other classes – with the main applicants, but this is inevitable) There are questions about "marital" migration, but there is no evidence…

Human capital Graduates, Highly skilled specialists, holders of professions in demand, IT specialists, talents -

Ethno-cultural ties with Russia State program of compatriots resettlement and Russian language native speakers

The participants of the State Program in fact represent two other classes – skilled workers and family members. In addition, in the mid–2010 s and in 2022, many participants in the State Program were the people who received temporary protection status

Preferences by country of birth, citizenship or residence

Several categories, including "for humanitarian purposes" when applying for citizenship

Citizenship: the majority of applicants in 2019-2022, apparently, were not migrants. Eligibility to get citizenship for humanitarian purposes actually implies residence in a country with unstable political processes.

Status regularization Regularization, recognition, restoration (in

citizenship) -

Students – (since 2023 and only for temporary residence permits)

Students of tertiary system of education

The category is not filled so far, it reflects only a small part of the total number of international students, the interest is selective and depends on the country of student’s citizenship.

The class of refugees (or persons receiving protection) is not defined. Оnly a few refugees acquired citizenship in recent years (42 persons in 2021-2023) (included in the "other“ category). "Temporary protection" – applicants receive statuses already as participants of the State Program of compatriots reselltement

On the charts: distribution of received temporary and permanent residence permits by classes of admission 2021-2023, percent. Based on the data of the Ministry of Internal Affairs of Russia

10

Family reunification class prevails. Temporary residence permits often issued to migrants from “Ethno-cultural ties” class (State Program of compatriots

resettlement)

Main results in figures

13.4 6.6 7.8 9.4

36.66

35.8 39.9 37.0

27.75

24.7

29.6 26.9

17.70

28.9 4.4

19.5

4.14 3.1

6.5

4.2

0.00 0.0

11.3

2.4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2021 2022 2023 2021-2023

Other unspecified categories outside the quota

Students (from 2023)

Human capital

Preferences by country of citizenship, birth or residence

Ethno-cultural ties with Russia

Outside the quota, including: Family reunification

Within the established quota

Temporary residence permits

31.3

20.7 16.7

23

51.9 65.1

52.3

56.9

3.1 1.9

1.2

2.1

11.6 7.1

23.9

13.7

1.8 2.4 2.6 2.3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2 021 2 022 2 023 2021-2023

Other unspecified

Status regularization

Human capital

Preference for country of citizenship or country of birth and historical justice

Ethno-cultural proximity to Russia (Russian language speakers)

Family reunification

Residence permit on the basis of a residence permit (with unspecified class of receipt)

Permanent residence permits

Citizenship acquisition. The structure of recent years has been influenced by the mass admission of Ukrainian citizens (a shift in the structure of classes of admission to citizenship in favor of preferences by country of citizenship, birth or residence). In 2023, the proportion between classes became more “traditional”

11

16.0

36.2

55.9

43.215.2

14.3

16.3

15.0

50.1

48.7

26.5

40.8

17.8

0.0 0.0 0.0

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2 021 2 022 2 023 2022-2023

Distribution of persons granted citizenship of the Russian Federation, by class of admission, 2021-2023, percent

Family reunification Ethno-cultural ties with Russia Preferences by country of citizenship, birth or residence Human capital Status regularizationdata of the Ministry of Internal Affairs of Russia

Main finding: majority of foreigners in Russia get residence permit and acquire citizenship via family reunification class of admission

18.2

26.5

3.7

0.6

48.7

55.9

9.4

16.3

3.3

0

16.6

0

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Total temporary and permanent residence permits issued in 2023

Citizenship acquisition in 2023

Preference based on country of citizenship. Birth or residence

Human capital

Family reunion

Ethnocultural ties

Students

Regularization of status

Other

Distribution of persons granted Russian citizenship and

of decisions on the issuance of temporary residence permits and residence permits (total) in 2023 by class of admission, percent

data of the Ministry of Internal Affairs of Russia

Potential of the data used

• Possibility to analyze individual classes of migration and naturalization, their composition and changes in indicators

• Analysis of differences in the choice of channels (and admission classes) of migration and naturalization by citizens of different states

• Analysis of law enforcement practice in the field of issuance of a residence permit, residence permit and admission to citizenship at the level of individual regions of Russia

• Availability of data for previous years – a retrospective analysis of the situation, its dynamics by country and by region

13

14

The distribution of persons admitted to citizenship of the Russian Federation by enlarged classes showed large differences between citizens of different countries in the choice of formal channels of naturalization (more pronounced than with a detailed approach)

An example of country- specific composition of naturalization classes

86.0 85.4 80.4 78.6 71.8 69.6 67.3

56.4 55.9 48.8

41.3

25.0

10.0 9.1 6.3 2.1

12.9 12.6 16.1 19.0

22.8 28.8

18.9 37.6

16.3 28.0 53.9

10.4

8.2

39.9

0.3 0.4

0.2 0.0 1.0 1.6 4.1 0.6

8.0 5.0

26.5 13.7

3.5

5.1

81.8

50.8

92.3 97.0

0.0 0.0 2.4 0.4 0.1 0.1 1.2 0.1 0.7 9.1

0.4

59.3

0.0 0.2 1.0 0.5

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Other

Status regularization

Human capital

Territory based preference (country of birth, citiztnship, residence)

Ethno cultural ties with Russia

Family reunion

Distribution of citizens of selected countries by class of Russian citizenship acquisition. 2023 , percent. Based on the Ministry of Internal Affairs of Russia data

Assessment of the scale of labor, educational and forced migration

• Residence permits and admission to citizenship statistics cannot be a good basis for such immigration measurement.

• The majority of labor and educational migrants, apparently - within the framework of current legislation – can legally stay in Russia on the basis of other documents and having the status of a temporary resident / Moreover, with regard to labor migrants, this reflects the position of the state to maintain their temporary status. Apparently. qualified migrants receive a PRTR. Residence permit and citizenship through other channels, including through the State Program.

• Forced migration is quickly "absorbed" by migration in the class of "Ethnocultural ties with the Russian Federation" – due to the origin of the majority of forced migrants and their compliance with the definition of "compatriot"

15

Legislation is changing ….. New provisions, new categories of migrants eligible for residence permit issuance or citizenship acquisition

• Since October 2023, a new law on citizenship has been in force

• Starting this year, the adoption of the federal law "On the conditions of entry (exit) and stay (residence) in the Russian Federation of foreign citizens and stateless persons" is expected - the institution of temporary residence permits will be excluded. There will be a "long-term stay regime" differentiated by purpose of stay. But it is not yet clear how this will be reflected in the statistics.

• 10 categories of foreigners will be eligible to get residence permit in a simplified manner. They can also be combined into classes.

• It is expected that the form 2-RD will be changed. Comparability of indicators for individual categories should be good (because the formulations are inherited from previous laws)

16

Concluding remarks

• The administrative data used made it possible (at least partially) to close the gap in understanding the structure of long-term migration flows by formal classes of admission for residence and naturalization

• The conducted consolidation of categories and their classification simplifies the analysis and gives an idea of the organization of formal channels for permanent residence in Russia and admission to Russian citizenship.

• Using this approach, it is possible to objectively observe the predominance of grounds related to family reunification, which indicates that the nature of long-term migration in the Russian Federation corresponds to the most common situations in migrant-receiving countries and the compliance of the migration policy of the Russian Federation with the task of simplifying family reunification (although this aspect is not mentioned among the stated tasks of migration policy)

• There is also an imbalance towards formal (declared) preferences based on criteria unrelated to the availability of human capital, but they can mask the true characteristics of migrants.

• The proposed approach showed that Russia's policy on skilled labor migration is aimed at maintaining the temporary status of foreign workers.

• Formal channels of long-term migration and naturalization in no way reflect the volume of migration of individual species. It is necessary to use special types of statistics to estimate real volumes of these categories of migrants

• The continuation of the analysis of administrative data will be possible only if there is access to the data…

17

•Thank you for your attention!

18

  • Slide 1: Classification of international migration in Russia based on administrative data
  • Slide 2: In world practice, the description of immigration channels to a particular country is often based on statistics of residence permits, which are already sorted by a certain class at the stage of issuance.
  • Slide 3: Considerable gaps in understanding classes of admission of foreigners to residence and naturalization in Russia and possible solutions
  • Slide 4: Access of foreigners to the right to reside and acquire citizenship in Russia: different populations receive temporary and permanent residence permit, as well as citizenship (only partial overlapping)
  • Slide 5: Two basic laws define the statuses that give a foreigner the right to reside and acquire citizenship: The Law on the Legal Status of Foreign Citizens (FZ-115) (to be amended) and the Law on Citizenship of the Russian Federation (FZ-62 and f
  • Slide 6: Data description
  • Slide 7: An example of the statistical report 2-RD with data distributed by individual provisions of the law “On Legal status of foreigners in the RF”
  • Slide 8: Data limitations
  • Slide 9: Classification results
  • Slide 10: On the charts: distribution of received temporary and permanent residence permits by classes of admission 2021-2023, percent. Based on the data of the Ministry of Internal Affairs of Russia
  • Slide 11: Citizenship acquisition. The structure of recent years has been influenced by the mass admission of Ukrainian citizens (a shift in the structure of classes of admission to citizenship in favor of preferences by country of citizenship, birth or r
  • Slide 12: Main finding: majority of foreigners in Russia get residence permit and acquire citizenship via family reunification class of admission
  • Slide 13: Potential of the data used
  • Slide 14
  • Slide 15: Assessment of the scale of labor, educational and forced migration
  • Slide 16: Legislation is changing ….. New provisions, new categories of migrants eligible for residence permit issuance or citizenship acquisition
  • Slide 17: Concluding remarks
  • Slide 18
Russian

Классификация международной миграции в России на основании административных данных

Ольга Чудиновских Экономический факультет МГУ имени М.В. Ломоносова

.

1

Европейская экономическая комиссия Конференция европейских статистиков Группа экспертов по статистике миграции Женева, Швейцария, 7-8 мая 2024 г.

Мировая практика: описание каналов иммиграции в принимающую мигрантов страну часто базируется на статистике видов на жительство, которые уже на этапе выдачи относятся к определенному классу.

2

Традиционный подход к классификации видов на жительство – выделение миграции: ✓ по линии воссоединения семей, ✓ экономической (трудовой) миграции, ✓ студентов, ✓ беженцев и других категорий иммигрантов

Понимание того, какие легальные «ворота» для миграции существуют, какие из них чаще выбирают мигранты позволяет оценить характер миграционной политики (хотя и не всегда отражает мотивы переезда индивидов). https://www.americanactionforum.org/series/building-a-pro-

growth-legal-immigration-system/

Наличие заметного пробела в знаниях о формальных каналах миграции и натурализации в России и возможные решения

• Россия является одной из основных стран, принимающих мигрантов в мире

• Численность международных пожизненных мигрантов в соответствии с оценками ДЭСВ ООН в 2020 году составила 11,6 млн человек (United Nations Department of Economic and Social Affairs, Population Division (2020). International Migrant Stock 2020.)

• С 2010 по 2023 год, по данным Министерства внутренних дел России, в России было выдано 5,6 млн разрешений на временное и постоянное проживание (2,4 млн и 3,2 млн соответственно), 4,4 млн человек получили гражданство по линии Министерства внутренних дел.

• Однако у нас по-прежнему очень смутное представление о том, какие формальные основания чаще всего используются властями при принятии решений о предоставлении иностранцам того или иного статуса.

• В России нет обобщенной статистики такого рода, хотя есть информация, позволяющая пролить свет на этот важный вопрос.

• На выбор предлагаются административные данные Министерства внутренних дел России о разрешениях на временное проживание, видах на жительство и гражданстве.

• Несмотря на различия в подходах в разных странах, эти данные имеют много общего и могут считаться сопоставимыми. Если есть распределение по категориям въезда, то они дают представление о том, через какие “ворота” мигрантам разрешается проживать в стране.

• Еще одно преимущество: это понятные и сопоставимые статусы и данные, их относительно легко интерпретировать

В статистике миграции остро стоит вопрос определений и сопоставимости данных. В отличие от данных Росстата, статистика видов на жительство и приёма в гражданство относительно сопоставима в международном контексте

3

Лица, получающие разрешение на временное проживание, вид на жительство и гражданство РФ, представляют собой в значительной степени разные популяции. Имеются многочисленные исключения из общего порядка получения статуса.

РАЗРЕШЕНИЕ НА ВРЕМЕННОЕ ПРОЖИВАНИЕ (РВП)

В РАМКАХ КВОТЫ

БЕЗ ПРИМЕНЕНИЯ КВОТЫ (несколько категорий иностранцев) НА ОСНОВАНИИ РВП -

УЧАСТНИКИ ГОСПРОГРАММЫ СОДЕЙСТВИЯ ПЕРЕСЕЛЕНИЮ СООТЕЧЕСТВЕННИКОВ

ПРИЕМ В ГРАЖДАНСТВО

ВИД НА ЖИТЕЛЬСТВО

НА ОСНОВАНИИ РАЗРЕШЕНИЯ НА ВРЕМЕННОЕ ПРОЖИВАНИЕ

ВНЖ БЕЗ ОФОРМЛЕНИЯ РАЗРЕШЕНИЯ НА ВРЕМЕННОЕ ПРОЖИВАНИЕ

НА ОСНОВАНИИ ВИДА НА ЖИТЕЛЬСТВО

В ОСОБОМ ПОРЯДКЕ (БЕЗ РВП И ВИДА НА

ЖИТЕЛЬСТВО 4

Без РВП или вида на жительство

Вид на жительство без РВП

Гражданство без вида на жительство

Два основных закона определяют статусы, дающие иностранцу право на проживание и приобретение гражданства: - Закон о правовом положении иностранных граждан (ФЗ-115) (будут изменения) и - Закон о гражданстве РФ (ФЗ-62 и - с 26.10.2023 - ФЗ-138). (* Закон о миграционном учете № 109)

• В законе классы допуска или соответствия не выделяются. Позиции идут не в «тематической» последовательности.

• Но в них можно увидеть общее и попытаться сгруппировать

Статья 8. Постоянное проживание иностранных граждан в Российской Федерации

ГАРАНТ:

См. комментарии к статье 8 настоящего Федерального закона

1. Вид на жительство может быть выдан иностранному гражданину, прожившему в

Российской Федерации не менее одного года на основании разрешения на временное проживание.

Информация об изменениях:

Статья 8 дополнена пунктом 1.1 с 1 января 2023 г. - Федеральный закон от 14 июля 2022 г. N 357-

ФЗ

1.1. Вид на жительство может быть выдан иностранному гражданину, получившему

разрешение на временное проживание в целях получения образования, после завершения обучения

в государственной образовательной или государственной научной организации по программе

бакалавриата, программе специалитета, программе магистратуры, программе ординатуры,

программе ассистентуры-стажировки, имеющим государственную аккредитацию, или по программе

подготовки научных и научно-педагогических кадров в аспирантуре (адъюнктуре).

2. Без получения разрешения на временное проживание вид на жительство выдается:

1) иностранному гражданину, родившемуся на территории РСФСР и состоявшему в прошлом

в гражданстве СССР;

2) иностранному гражданину, который не достиг возраста восемнадцати лет и родитель

(усыновитель, опекун, попечитель) которого является иностранным гражданином и постоянно

проживает в Российской Федерации;

3) иностранному гражданину, не достигшему возраста восемнадцати лет, получающему вид

на жительство совместно с родителем (усыновителем, опекуном, попечителем) - иностранным

гражданином;

4) иностранному гражданину, имеющему родителя (усыновителя, опекуна, попечителя), сына

или дочь, состоящих в гражданстве Российской Федерации и постоянно проживающих в Российской

Федерации;

5) иностранному гражданину, достигшему возраста восемнадцати лет, в соответствии с

законодательством иностранного государства признанному недееспособным либо ограниченным в

дееспособности, получающему вид на жительство совместно с родителем (усыновителем, опекуном,

попечителем) - иностранным гражданином;

6) иностранному гражданину, который достиг возраста восемнадцати лет, в соответствии с

законодательством иностранного государства признан недееспособным либо ограниченным в

дееспособности и родитель (усыновитель, опекун, попечитель) которого является иностранным

гражданином и постоянно проживает в Российской Федерации;

7) иностранному гражданину, признанному носителем русского языка в соответствии со

статьей 33.1 Федерального закона от 31 мая 2002 года N 62-ФЗ "О гражданстве Российской

Федерации";

8) иностранному гражданину, который сам либо родственник по прямой восходящей линии

Категории соискателей

5

Описание данных • Регламентные статистические отчеты МВД России (ранее - ФМС) по форме

2-РД 1) «Регистрационно–визовая работа» (РВР) – в отношении действий в

рамках Закона о правовом положении иностранных граждан и Закона о миграционном учете иностранных граждан.

2) «Гражданство» – в основе Закон о гражданстве 3) «Внешняя трудовая миграция» - ВТМ, в этой работе не рассматривается • с распределением по странам гражданства мигрантов и регионам России

– данные по разрешениям на временное проживание, видам на жительство (постоянным) и приему в гражданство.

• И с распределением по отдельным позициям закона. • Хотя прием в гражданство не является собственно миграцией, но может

рассматриваться как её завершающая фаза - в смысле изменения статуса мигранта. 6

7

ФОРМА "2-РД" из них

за январь - декабрь 2021 года

РОССИЯ КОНТРОЛЬНОЕ ЗНАЧЕНИЕ 1-РД => 190 691 18 990 167 596 236 259

Код строки по 1-РД в3.20.1 в3.21.1 в3.22.1 в3.52.1

Код графы по 2-РД РВР15 РВР16 РВР17 РВР18 РВР19 РВР53 РВР54 РВР27 РВР23 РВР55 РВР24 РВР25 РВР56 РВР57 РВР29 РВР30

ВСЕГО c1 190 691 18 990 167 596 15 106 7 352 3 081 841 3 749 56 844 5 13 46 505 6 934 26 192 397 236 259

Абхазия 895 c2 69 7 85 9 7 1 1 4 61 0 0 0 2 0 0 205

Австралия 036 c3 22 5 18 5 0 0 0 1 12 0 0 0 0 0 0 24

Австрия 040 c4 18 0 16 4 2 0 0 0 10 0 0 0 0 0 0 40

Азербайджан 031 c5 11 357 1 120 9 990 937 407 193 50 174 7 161 0 1 617 401 0 0 23 255

Албания 008 c8 2 0 1 0 0 0 0 0 1 0 0 0 0 0 0 10

Алжир 012 c9 97 8 76 9 1 1 0 0 62 0 0 0 1 0 0 102

Американское Самоа 016 c10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Ангилья 660 c11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Ангола 024 c12 6 1 6 1 0 0 0 0 5 0 0 0 0 0 0 12

Антарктида 010 с259 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Андорра 020 c13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

подпункт

10

подпункт

11

подпункт

12

без учета квоты в соответствии с п. 3 ст. 6 Федерального закона от 25 июля 2002 г. № 115-ФЗ «О правовом положении иностранных граждан в Российской Федерации»

текущего

года

предыдущ

его года

подпункт

1

подпункт

2 и

подпункт

6

подпункт

3 и

подпункт

5

подпункт

4

подпункт

7

подпункт

8

подпункт

9

Р А З Р Е Ш Е Н И Я на В Р Е М Е Н Н О Е П Р О Ж И В А Н И Е

Раздел 1. Сведения о

результатах

разрешительно-визовой

работы

К о

д п

о О

К С

М

К о

д с

т р

о к и

Принято

заявлений

о выдаче

ИГ и ЛБГ

разрешени

й на

временное

проживани

е

Принято

решений о

выдаче

разрешени

й на

временное

проживани

е

в том числе Число ИГ

и ЛБГ,

имеющих

действите

льные

разрешени

я на

временное

проживани

е на конец

отчетного

в

пределах

установле

нной

квоты

текущего

года

в пределах установленной квоты

Пример таблицы из отчета 2-РД с распределением данных по отдельным позициям закона «О правовом положении иностранных граждан …» (ФЗ-115)

Аббревиатуры обозначают отдельные позиции закона «О правовом положении иностранных граждан» : например – пп.3 и пп. 5 части 3 статьи 6 ФЗ 115 – это несовершеннолетние дети и взрослые недееспособные лица, получающие РВП совместно с родителем. Группируются только схожие позиции.

пп. 9 части 3 статьи 6 ФЗ- 115 - участники Госпрограммы содействия добровольному переселению соотечественников

7

Разрешения на временное проживание

Ограничения данных

• Отсутствие детализации в статистике РВП до 2014 г., а видов на жительство до 2021 года (после перехода многих категорий «РВП вне квоты» в «Вид на жительство без РВП» - сдвиг в распределении потоков , уменьшение числа лиц, получающих РВП

• 2019- 2023 активное изменение законодательства, быстрое расширение перечня оснований для получения вида на жительство, минуя РВП , в течение последних трех лет (сопоставимость и агрегирование)

• Отставание модернизации бланка формы от изменения в законодательстве. Имеет место проблема больших не распределенных по категориям остатков, обратная связь с производителем статистики затруднена.

При изменении в законодательстве (в последние годы – интенсивном) происходит перегруппировка, перераспределение мигрантов по другим формальным каналам, при выборе мигрантом «ворот» допуска к статусу, доминируют соображения простоты достижения цели, а не точности «попадания» в соответствующий пункт закона.

Основные результаты классификации и проблемы.

Класс Категории Особенности или ограничения

Воссоединение семей Дети, родители, супруги, лица под опекой

Нет (не считая членов семей, включенных в другие классы – с основными заявителями, но это неизбежно) Есть вопросы по «супружеской» миграции, но нет доказательств…

Человеческий капитал Выпускники, ВКС, носители востребованных профессий, IT специалисты, таланты

-

Этнокультурные связи с Россией

Госпрограмма и носители русского языка

Участники Госпрограммы – по существу - два класса – квалифицированные работники и члены семей. Кроме того – в середине 2010-х и в 2022 многие участники Госпрограммы – это лица, получившие временное убежище

Преференции по стране рождения, гражданства или проживания

Несколько категорий, включая «в гуманитарных целях» при приеме в гражданство

Гражданство: основная часть соискателей в 2019-2022 гг, видимо, не были мигрантами. Гуманитарные цели на самом деле предполагают факт проживания в стране с соответствующими политическими процессами.

Урегулирование статуса Урегулирование, признание, восстановление (в гражданстве)

-

Студенты - с 2023 г. и только для разрешений на временное проживание

РВП с целью получения образования

Пока категория не наполнена – отражает лишь незначительную часть общего числа иностранных студентов , интерес избирательный и зависит от страны гражданства.

Класс «Убежище» не выделяется. Беженцев, получивших гражданство, единицы (отнесены при расчетах в «прочие»). «Временное убежище» – соискатели получают статусы уже как участники Госпрограммы

9

Распределение полученных РВП и видов на жительство по укрупненным группам, соответствующим критериям предоставления статуса, 2021-2023 гг., проценты. Источник: рассчитано по данным МВД России

10

Преобладают основания «воссоединение семей» . Большая доля РВП в классе «Этнокультурные связи с Россией» выдаются участникам Госпрограммы

Разрешения на временное проживание

Основные результаты в цифрах

36.66

35.8 39.9 37.0

27.80

24.7

29.6 26.9

4.14 3.1

6.5

4.2

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2021 2022 2023 2021-2023

Прочие неуточненные категории вне квоты

Студенты (с 2023 г.)

Человеческий капитал

Преференции по стране гражданства, рождения или проживания

Этнокультурные связи с Россией

Вне квоты, в том числе: Воссоединение семей

В рамках установленной квоты всего

31.3

20.7 16.7

23

51.9

65.1

52.3

56.9

11.6 7.1

23.9

13.7

1.8 2.4 2.6

2.3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2 021 2 022 2 023 2021-2023

Виды на жительство Прочие неуточненные

Урегулирование статуса

Человеческий капитал

Преференции по стране гражданства или стране рождения и историческая справедливость

Этнокультурная близость к России (носители русского языка)

Воссоединение семей

ВНЖ на основании РВП (с неуточненным классом получения)

Рассчитано по данным МВД России

Прием в гражданство. Структура последних лет - под влиянием массового приема граждан Украины (смещение структуры классов приема в гражданство в пользу преференций по стране гражданства, рождения или проживания). В 2023 г. структура классов стала приближаться к «традиционной»

16

36.2

55.9 43.215.2

14.3

16.3

15

50.1

48.7

26.5

40.8

17.8

0 0 0

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2 021 2 022 2 023 2022-2023

Воссоединение семей Этнокультурные связи с Россией

Преференции по стране гражданства, рождения или проживания Человеческий капитал

Урегулирование В общем порядке (без преференций)

В упрощенном порядке не уточненные категории 11

Распределение лиц. принятых в гражданство РФ по классам оснований, 2021-2023 гг., проценты

Рассчитано по данным МВД России

Распределение лиц, получивших гражданство России, и решений о выдаче разрешения на временное проживание и вида на жительство (всего) по классам оснований, 2023 г. проценты

18.2

26.5

3.7

0.6

48.7

55.9

9.4

16.3

3.3

0

16.6

0

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Всего решений о выдаче РВП и видов на жительство

Принято в гражданство

Преференции по стране гражданства или стране рождения

Человеческий капитал

Воссоединение семей

Этнокультурные связи с РФ

Студенты

Урегулирование статуса

Прочие , в т ч РВП по квоте и ВнЖ на основании РВП

Основной вывод: большинство иностранцев в России получают временные и постоянные виды на жительство и гражданство в порядке воссоединения семьи.

Рассчитано по данным МВД России

Потенциал административных данных из форм 2-РД

• Возможность анализа отдельных классов миграции и натурализации, их состава и изменения показателей

• Анализ различий в выборе каналов (и классов допуска) миграции и натурализации, гражданами разных государств

• Анализ правоприменительной практики в сфере выдачи РВП, ВнЖ и приема в гражданство на уровне отдельных регионов России

• Наличие данных за прошлые годы – ретроспективный анализ ситуации, ее динамики и по странам, и по регионам

13

14

86 85.4 80.4 78.6 71.8 69.6 67.3

56.4 55.9 48.8

41.3

25 10 9.1 6.3 2.1

12.9 12.6 16.1 19

22.8 28.8

18.9 37.6

16.3 28 53.9

10.4

8.2

39.9

0.3 0.4

0.2 0 1 1.6 4.1 0.6

8 5

26.5 13.7

3.5

5.1

81.8

50.8

92.3 97

4.2 9.1

59.3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Доля принятых в российское гражданство по основным классам допуска , страны с числом натурализованных от 1000 и более, 2023 г. , проценты Данные МВД России

В общем порядке без преференций

Урегулирование, восстано-вление, признание

Человеческий капитал

Преференции по стране гражданства, рождения или проживания

Этнокультурные связи с Россией

Воссоедине-ние семей Рассчитано по данным МВД России

Распределение лиц, принятых в гражданство РФ по укрупненным классам показало большие различия между гражданами разных стран в выборе формальных каналов натурализации (более выраженно, чем при детализированно м подходе)

Оценка масштабов трудовой , образовательной и вынужденной миграции • На основании данных о РВП, видах на жительство и приеме в гражданство

этого делать не следует.

• Большинство трудовых и образовательных мигрантов в рамках действующего законодательства могут законно находиться в России на основании других документов и имея статус временно пребывающего.

• В отношении трудовых мигрантов это отражает позицию государства – поддерживать их временный статус .

• По-видимому. квалифицированные мигранты получают РВП, вид на жительство и гражданство по другим каналам, в т.ч. – через Госпрограмму.

• Вынужденная миграция быстро «поглощается» миграцией в классе «Этнокультурные связи с РФ» – в силу происхождения большинства вынужденных мигрантов и соответствия их определению «соотечественника»

15

Законодательство меняется ..... Новые положения, новые категории мигрантов, имеющие право на получение вида на жительство или гражданства.

• С октября 2023 года действует новый закон о гражданстве, более 20 категорий соискателей

• С текущем году ожидается принятие федерального закона «Об условиях въезда (выезда) и пребывания (проживания) в Российской Федерации иностранных граждан и лиц без гражданства» - институт разрешений на временное проживание будет исключен. Появится «режим долгосрочного пребывания» дифференцированный по целям. Но пока не ясно, как это будет отражено в статистике. Статья 61. Предоставление права на постоянное проживание в упрощенном порядке - 10 категорий, которые тоже можно объединить в классы.

• Ожидается что будет изменена форма 2-РД. Сопоставимость показателей по отдельным категориям должна быть хорошей (т.к. формулировки унаследованы от прежних законов) 16

Заключительные замечания

• Использованные административные данные позволили закрыть пробел в понимании структуры потоков долгосрочной миграции по формальным классам допуска для проживания и натурализации

• Проведённое укрупнение категорий и их классификация упрощает анализ и дает представление об организации формальных каналов переезда на постоянное жительство в Россию и приема в российское гражданство.

• С помощью такого подхода можно объективно наблюдать преобладание оснований, связанных с воссоединением семей, что говорит о соответствии характера долгосрочной миграции в РФ наиболее распространенным ситуациям в принимающих мигрантов странах и соответствии миграционной политики РФ задаче упрощения воссоединения семей (хотя среди заявленных задач миграционной политики этот аспект не упоминается )

• Также виден дисбаланс в сторону формальных преференций по критериям, не связанным в наличием человеческого капитала, но они могут маскировать истинный состав мигрантов.

• Предложенный подход показал, что политика России в отношении квалифицированной трудовой миграции направлена на поддержание временного статуса иностранных работников.

• Формальные каналы долгосрочной миграции и натурализации никоим образом не отражают объемы миграции отдельных видов. Для понимания объемов квалифицированной трудовой и образовательной миграции (а также вынужденной) следует опираться на специальные виды статистики именно по этим категориям мигрантов

• Продолжение анализа административных данных будет возможно только при наличии доступа к данным

17

•Спасибо за внимание

18

  • Slide 1: Классификация международной миграции в России на основании административных данных
  • Slide 2: Мировая практика: описание каналов иммиграции в принимающую мигрантов страну часто базируется на статистике видов на жительство, которые уже на этапе выдачи относятся к определенному классу.
  • Slide 3: Наличие заметного пробела в знаниях о формальных каналах миграции и натурализации в России и возможные решения
  • Slide 4: Лица, получающие разрешение на временное проживание, вид на жительство и гражданство РФ, представляют собой в значительной степени разные популяции. Имеются многочисленные исключения из общего порядка получения статуса.
  • Slide 5: Два основных закона определяют статусы, дающие иностранцу право на проживание и приобретение гражданства: - Закон о правовом положении иностранных граждан (ФЗ-115) (будут изменения) и - Закон о гражданстве РФ (ФЗ-62 и - с 26.10.2023 - ФЗ-138
  • Slide 6: Описание данных
  • Slide 7
  • Slide 8: Ограничения данных
  • Slide 9: Основные результаты классификации и проблемы.
  • Slide 10: Распределение полученных РВП и видов на жительство по укрупненным группам, соответствующим критериям предоставления статуса, 2021-2023 гг., проценты. Источник: рассчитано по данным МВД России
  • Slide 11: Прием в гражданство. Структура последних лет - под влиянием массового приема граждан Украины (смещение структуры классов приема в гражданство в пользу преференций по стране гражданства, рождения или проживания). В 2023 г. структура классов стала
  • Slide 12: Распределение лиц, получивших гражданство России, и решений о выдаче разрешения на временное проживание и вида на жительство (всего) по классам оснований, 2023 г. проценты
  • Slide 13: Потенциал административных данных из форм 2-РД
  • Slide 14
  • Slide 15: Оценка масштабов трудовой , образовательной и вынужденной миграции
  • Slide 16: Законодательство меняется ..... Новые положения, новые категории мигрантов, имеющие право на получение вида на жительство или гражданства.
  • Slide 17: Заключительные замечания
  • Slide 18

Classification of international migration in Russia based on administrative data (Russian Federation)

Languages and translations
English

*Prepared by Olga Chudinovskikh, Lomonosov Moscow State University) NOTE: The designations employed in this document do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries.

Economic Commission for Europe Conference of European Statisticians Group of Experts on Migration Statistics Geneva, Switzerland, 7−8 May 2024 Item 6 of the provisional agenda Improvements in use of administrative data for migration statistics

Classification of international migration in Russia based on administrative data

Note by Lomonosov Moscow State University*

Abstract The paper proposes an approach to the identification of the main classes of admission of foreigners to obtain residence permits and citizenship of the Russian Federation and, on this basis, an assessment of the composition of the flow of persons obtaining these statuses by the corresponding classes. This allows us to see what formal channels of admission to the right of residence and naturalization are most often used by foreigners in Russia. Unlike many countries, where, for example, residence permits already belong to a certain admission class at the time of issuance, and statistics are available in aggregate form, in Russia, there is almost no such practice. Statistical information of this kind is not produced on a regular basis and is not available in open sources. Based on the analysis of the administrative data of the Russian Ministry of Internal Affairs, which shows the number of foreigners who have received different statuses in accordance with numerous provisions of the law, an attempt was made to group these provisions and identify classes of grounds (or classes of admission). Six classes were identified: family reunification, human capital, preferences based on country of citizenship, birth or residence, ethno-cultural ties with Russia, status regularization, and the class of international students. This made it possible to show that "Family reunification" was the largest class of admission among migrants who obtained residence permits and citizenship. Based on the statistics of citizenship acquisition our approach allowed us to demonstrate considerable differences in the ratio of classes and naturalization channels chosen by citizens of individual countries. The availability of statistics for previous years makes it possible to track the dynamics of this structure in retrospect. The main vulnerabilities of the administrative data used are noted - the inability to adequately reflect educational and skilled labour migration, but an assumption is made that this can be compensated by integrating data from other sources. In addition, the proposed approach showed that Russia's policy towards skilled labour migration is aimed at maintaining the temporary status of foreign workers.

Working paper 9

Distr.: General 29 April 2024 English

Working paper 9

2

I. Introduction

1. Researchers in many countries are able to study the composition of immigration flows by main classes of admission on the basis of ready-made, aggregated data published by national migration or statistical agencies1. When it comes to migration of foreigners with the right of permanent residence, statistics on residence permits are often used. The grounds for obtaining a residence permit are immediately grouped into aggregated classes, which determine the type of residence permit. Published statistics are structured according to this variable, which allows us to see the ratio of migration flows through family reunification, skilled labour migration, migration for the purpose of education, asylum, etc. When discussing the international comparability of migration statistics, the most active debate revolves around the definition of long-term migrants, who change the size and composition of a country's resident population. Differences in criteria of migrant identification and count applied in individual countries contribute to the persistent problem of the international comparability of migration statistics. It is not always possible to effectively apply the time criterion (duration of residence in the new place of usual residence) in statistics production. Moreover, the use of only this criterion does not always allow for adequately measuring the scale and structure of migration flows, since the most important characteristic - the status of the migrant in the host country is left out of attention. Meanwhile, this is the most important variable from the point of view of the implementation of the country's migration policy.

2. In this context, it can be assumed that residence permits statistics (and probably naturalization statistics) are relatively comparable across countries. Even if the countries apply different approaches to the issuance of residence permits and admission to citizenship, these statuses imply many similarities. They remove restrictions on admission to the labour market, provide a migrant with some (or all) political rights, and, perhaps most importantly, mark the transition of most recipients of such statuses into the permanent population of the receiving country. In addition, statistics on residence permits and citizenship admission can show the ratio of channels of long- term migration of foreigners or their naturalization. Of course, these are the official or formal channels established by the country of destination, through which foreigners are allowed to obtain a residence permit and then citizenship. The real motives or purposes of migration may be different. Nevertheless, it is important to know what legal "gates" migrants choose to move to the country for residence and how the flow is redistributed when new formal channels appear or old ones are closed. This makes it possible to assess the nature of a country's migration policy and the results of its implementation.

3. Russia is one of the countries with very significant numbers of international migrant stock and immigration flows. According to the UN Population Division estimates for 2020, the stock of lifetime international migrants amounted to 11.6 million people2. From 2010 to 2023, according to the Ministry of Internal Affairs3, 5.6 million residence permits were issued in Russia (of them - 2.4

1 US Yearbook of Immigration Statistics 2022. (Table 6. Persons Obtaining Lawful Permanent Resident Status by Type and Major Class of Admission: Fiscal Years 2013 to 2022). https://www.dhs.gov/ohss/topics/immigration/yearbook/2022 ; Federal Ministry of the Interior, Building and Community, Federal Office of Migration and or Refugees (Germany). Migration Report 2021 (p.6) Key findings. https://www.bamf.de/SharedDocs/Anlagen/EN/Forschung/Migrationsberichte/migrationsbericht-2021- zentrale-ergebnisse.pdf?__blob=publicationFile&v=6 2 United Nations Department of Economic and Social Affairs, Population Division (2020). International Migrant Stock 2020. https://www.un.org/development/desa/pd/content/international-migrant-stock 3 The data was provided upon request to the Department of Economics at Lomonosov Moscow State University. (consulates), records are kept separately.

Working paper 9

3

million temporary and 3.2 million permanent permits4), 4.4 million people were admitted to citizenship through the Ministry of Internal Affairs5. However we have a very vague idea of what grounds are most often used when the authorities make decisions on granting foreigners this or that status. There are no aggregated statistics of this kind in Russia, although there is information that sheds light on this important issue. For many years, the Russian Ministry of Internal Affairs (and until 2016, the Federal Migration Service) has been developing statistical reporting forms on temporary residence permits, (permanent) residence permits and admission to citizenship. Although the latter is not migration proper, it can be considered as its final phase in the sense of changing the migrant's status. These statistics are not available in open sources. Since 2010 the Faculty of Economics of the Lomonosov Moscow State University has been receiving it upon request.

II. Data description and its limitations

4. Statistical reporting form - the so called form 2-RD6 of the Ministry of Internal Affairs of Russia is a Microsoft Excel file with two sheets. In the first one, the row names are the countries of citizenship of migrants; in the second one, the row names are the regions of Russia. The names of the columns reflect various procedures or types of documents executed or issued in relation to foreign citizens: for example, decisions on issuing residence permits, on admission to citizenship, or on work permits issued. The Form 2-RD report is generated in three versions: "Permit and visa work" (including information on permanent residence permits and temporary residence permits), "Citizenship"7 and "International labour migration". The last type of statistics is not considered in this paper.

5. The peculiarity of the first two reports is that information on temporary residence permits, permanent residence permits or citizenship acquisition is detailed according to the individual provisions of the relevant laws: the law "On the Legal Status of Foreign Citizens..." 8 (the so-called "Law No. 115") and the law "On Citizenship..."9 (the so-called "Law No. 62"). The wording of the law describing individual categories of foreigners eligible to obtain status - is quite verbose. It is difficult to use them as column headings in a table. Therefore, these formulations are "coded". Visually, in the column titles we can see a set of abbreviations denoting the corresponding subparagraphs or paragraphs of individual articles of the law. For example, in the statistics of permanent residence permits it is indicated how many permits were issued on the basis of subparagraph 1 of paragraph 2 of Article 8 of the Federal Law № 115; subparagraphs 2 and 6 of paragraph 2 of Article 8; subparagraphs 3, 5 of paragraph 2 of Article 8, etc. If we refer to the text of the law, we will get a deciphering of these abbreviations.

4 The data partially overlap, as the majority of persons with a residence permit must subsequently obtain a residence permit. 5 Admission to Russian citizenship is also formalized through foreign institutions of the Russian Federation (consulates). 6 https://base.garant.ru/70145636/?ysclid=lvbfg231r9190591739 7 In the tables with data on admission to citizenship, children and adults are distinguished. Over the last five years, the proportion of adults was 76%, children -24%. 8Federal Law No. 115-FZ of July 25, 2002 "On the Legal Status of Foreign Citizens in the Russian Federation" (with amendments and additions) https://base.garant.ru/184755/?ysclid=lus031am7j84734668 9 Federal Law "On Citizenship of the Russian Federation" dated 05/31/2002 No. 62-FZ (expired on October 26, 2023 due to the entry into force of a new law (see Federal Law No. 138-FZ of April 28, 2023 "On Citizenship of the Russian Federation") https://base.garant.ru/406811055/?ysclid=lus0ao6zn9102234638

Working paper 9

4

6. Subparagraph 1 of paragraph 2 of Article 8, refers to foreigners who were born on the territory of the RSFSR and were former citizens of the USSR. Subparagraphs 2 and 6 of paragraph 2 of Article 8, refer to children under 18 years of age (2) or incapable adults (6) ), whose parents or guardians have a residence permit and permanently reside in Russia. Subparagraph 4 of the paragraph 2 of the Article 8 refers to foreigners who have a parent/adoptive parent, guardian or son or daughter who is a Russian citizen and permanently resides in the Russian Federation; subparagraph 9 of the paragraph 2 of the Article 8 refers to highly skilled specialists, etc.

7. Statistics on the acquisition of Russian citizenship are presented in the same way. The abbreviations in the names of the columns of the table refer us to the text of the Citizenship law, from which we can understand, for example, that persons who acquired citizenship under subparagraph "б" of paragraph 2 of Article 14 of the Federal Law-62 - are foreigners who have been married for at least three years to a citizen of the Russian Federation residing on the territory of the Russian Federation. Subparagraph "в" of part 2 of article 14 concerns foreigners who have a capable, adult child who is a citizen of the Russian Federation. Persons who obtained Russian citizenship on the basis of part 2.1. of Article 14 of FZ-62 are so-called "native speakers of the Russian language", etc.

8. The specialists who designed the reporting form 2-RD already combined some categories of migrants that are homogeneous in meaning. For example, as mentioned above, the sum can include statistics on minor children and incapacitated adults who receive the status at the same time with their parents or because their parents already have this status. However, the law mentions these categories in different subparagraphs. Highlighting the common in the main content of these provisions, we were able to combine them, with a certain degree of conventionality, into enlarged classes and see what grounds are the main ones for issuing temporary and permanent residence permits and for admission to Russian citizenship.

9. Changes in legislation imply corresponding changes in the number and content of columns in the statistical report 2-RD. When amendments to the law introduce new categories of persons with special conditions for obtaining a residence permit or admission to citizenship (or cancel some of the previous ones), the corresponding columns are created in (or deleted from) the table. It is not always possible to make these changes promptly. Therefore, the table for the period during which these changes took place may, for example, have an unallocated balance - a number of migrants with an unspecified basis of status. However, these problems are resolved over time. We encountered this issue with the 2021 data on admission to citizenship - the unallocated balance was very significant and amounted to almost 131 thousand or 18% of the total number of naturalized persons.

10. In this paper we used the data available to us on the grounds on which foreigners obtain three types of status - 1) "temporary resident" - on the basis of a temporary residence permit (issued under a quota and out of quota, in a simplified procedure), 2) "permanent resident" - on the basis of a (permanent) residence permit and10 3) citizenship of the Russian Federation. Statistics on temporary residence permits (TRP), permanent residence permits and admission to citizenship largely reflect different populations of foreigners. Formally, the first status on the way to citizenship is a temporary residence permit, which after a year gives a foreigner the right to apply for a permanent residence permit and then for citizenship. However, there are numerous exceptions, which concern very large populations of applicants. For example, participants of the State Program for Assistance

10 Law 115 defines: “a foreign citizen temporarily residing in the Russian Federation is a person who has received a temporary residence permit or a temporary residence permit for the purpose of education; a foreign citizen permanently residing in the Russian Federation is a person who has received a (permanent) residence permit”. https://base.garant.ru/184755/741609f9002bd54a24e5c49cb5af953b/

Working paper 9

5

to the Resettlement of Compatriots11 have always been able to apply for citizenship immediately on the basis of a temporary residence permit, they did not need a residence permit. Since 2021, almost two dozen categories of applicants can bypass the temporary residence permit and apply for a residence permit (and then for citizenship). Previously, this was available only to citizens of Belarus. Since 2019, admission to citizenship on special grounds (for humanitarian purposes), without first obtaining a temporary or a permanent residence permit, has become a mass practice. By 2024, this practice has significantly narrowed down.

11. It should be noted that the lists of grounds for obtaining a residence permit, residence permit and citizenship considerably differ both in terms of the number of categories and in the wordings given in the relevant laws. Therefore, the occupancy of "classes" from the viewpoint of grounds for obtaining a temporary residence permit, permanent residence permit and admission to citizenship is different. For example, among those who received a temporary residence permit, the class "Ethno- cultural ties with Russia" includes only participants of the State Program (because they can acquire citizenship, bypassing a permanent residence permit). Among permanent residence permit recipients, the "Ethno-cultural ties" class was represented only by Russian language native speakers. However, among those admitted to Russian citizenship, this class included both categories.

12. We also found out that there is no strict connection or continuity between the grounds on which a foreigner can receive a temporary residence permit, permanent residence permit and citizenship. This is a peculiarity of the Russian legislation. A person who got a permanent residence permit without a TRP as a holder of a profession in demand, may apply for citizenship via another channel, for example, as a spouse of a Russian citizen, especially if there is a common child in the marriage. Migrants choose the fastest and most convenient algorithm for obtaining the status, apparently requiring the least number of documents. Moreover, when new, more convenient opportunities arise, the redistribution of flows occurs very quickly, which shows the high adaptability of migrants to changing circumstances. Therefore, it makes sense to separately examine each type of data: on temporary residence permits, permanent residence permits and acquisition of citizenship. It seems possible to combine, for example, data on temporary and permanent residence permits issued during the same calendar year, since one person cannot obtain both statuses within a year and the data do not overlap. However, the classes of admission will have different content. We can treat statistics on admission to citizenship as a "resultant" representation of grounds, if not of migration itself, then of its final stage (in the sense of migrant status).

13. In identifying classes of grounds, we encountered objective limitations. Statistics on temporary residence permits are detailed (since 2014) only for permits, which are issued outside the established quota. We do not know, what criteria are applied when issuing a permit under the quota. We assume that the availability of work and relatives in Russia may have a positive impact on officials’ decision making. The statistics of permanent residence permits are detailed only from 2021 and in relation to those documents that were issued without a temporary residence permit. That is, permanent residence permits that were obtained on the basis of a temporary one also cannot be categorized by the grounds for obtaining them. Only statistics on citizenship are available in detail and from 2010 onwards (by country of migrants’ citizenship or region of Russia).

11This is the only full-fledged immigration program that exists in Russia and formally has a repatriation character. The program has been in effect since 2006. According to the Russian Ministry of Internal Affairs, as of December 31, 2023, the stock of registered Program participants since 2006 amounted to 1 million 128 thousand people.

Working paper 9

6

14. The data reflect only flows of persons receiving temporary or permanent residence permits, and citizenship. No data are developed for the stocks of foreigners holding valid temporary or permanent residence permits. The time horizon over which we show the main results of the work covers the last three years (2021-2023), as this was the period of the most significant changes in legislation regarding the categories of applicants for residence permits and citizenship. The objective of this work is rather to demonstrate the potential of administrative data than to analyze and explain trends and fluctuations in indicators. Citizenship acquisition statistics have been available since 2010, thus we have had an opportunity to perform a retrospective analysis of the data, taking into account changes in legislation. We did this kind of work several years ago12. It very well demonstrated that the changes in legislation also affected the distribution of flows by classes of admission to citizenship. Migrants quickly adapt to increasingly complex conditions and seek more convenient naturalization channels, with simpler and more easily fulfilled requirements. In addition, over time, this leads to very significant differences between nationals of individual states in the choice of naturalization channels.

III. Approaches to distinguishing classes of admission for obtaining residence permits and Russian citizenship

15. We could only partly apply the traditional approach of classifying migration grouping of grounds related to family reunification, labour, educational or forced migration, including due to the peculiarities of Russian legislation on migration and admission to citizenship. When combining individual grounds for obtaining temporary and permanent residence permit or citizenship, we focused on the semantic content of each category. It allowed us to identify six classes of admission: 1. Family reunification; 2. Ethno-cultural ties with Russia; 3. Preferences based on the principle of country of citizenship, birth or residence; 4. Human capital bearers; 5. Regularization of status. An independent (sixth) class of persons who came to study appeared only in 2023 and only in relation to the temporary residence permit. Since 2023 a special type of TRP "for the purpose of education" was introduced13.

16. We could not identify the class "Asylum", although there are special provisions in the laws referring to such migrants. However in the context of the statuses we are writing about, data on refugees are statistically insignificant, and there are no data on persons granted temporary protection and then receiving other statuses. In the five years from 2019 to 2023 only 42 persons with refugee status were admitted to Russian citizenship (in 2023 - 3 persons) in accordance with par. "в" of part 2 of article 13 of the law "On Citizenship". In Russia, refugee status is virtually unavailable, with only 244 people registered as refugees at the end of 202314. The main form of protection for migrants in Russia is temporary protection . Its widespread use began during the first wave of forced migrants from the southeastern regions of Ukraine in 2014-2015. At that time, a decision was made to allow those who received temporary protection immediately join the State Program for Assistance to Voluntary Resettlement of Compatriots (repatriates). They received all subsequent statuses - temporary residence permits and citizenship (bypassing residence permits) as participants of the Program. Therefore, the forced migrants actually ended up in a different class of admission after their initial status changed.

12 Adaptation and integration of migrants in Russia: challenges, realities, indicators. Mukomel V. I., Grigorieva K. S., Monusova G. A. et al; ed. by Mukomel V. I., Grigorieva K. S. FCTAS RAS.In Russian. – М. 2022. DOI: 10.19181/monogr.978-5-89697-407-9.2022. стр. 69-73. URL: https://www.fnisc.ru/publ.html?id=11644&type=publ 13 This type of document assumes study in full educational programs in the higher education system. 14 Rosstat 2024. The report "The socio-economic situation in Russia". January 2024, page 187. In Russian. https://rosstat.gov.ru/storage/mediabank/osn-01-2024.pdf

Working paper 9

7

17. The first class - "family reunification" was relatively easy to identify. It includes situations related to obtaining the status of a minor child at the request of a parent who already has such status or simultaneously with him/her, foreign spouses, adult children or parents of Russian citizens. We associated the second class with the presence of certain ethno-cultural ties of a migrant with Russia, since this approach to granting preferences to migrants is emphasized in almost all fundamental documents related to Russia's migration policy15. Participants of the State Program for Assistance to Voluntary Resettlement of Compatriots and Russian language native speakers were included in this class. The third class of preferences is formed according to the "geographical" principle, which provides for the applicant's citizenship of a certain country, birth or residence in a certain territory (in the present or in the past). This approach is applied in several cases and unites, for example, persons born on the territory of the RSFSR and having citizenship of the USSR, citizens of Belarus, Ukraine, Kazakhstan and Moldova, citizens of states that have agreements with Russia.

18. The greatest difficulties were caused by the attempt to distinguish the class of migration for the purpose of work (skilled labour migration). Formally, it can include (and these categories are listed separately in the legislation) highly skilled specialists and members of their families16, holders of professions in demand17 and IT specialists (this category is introduced from 2023). However, in the end, we came to the conclusion that it would be more correct to single out a class of not just "skilled workers", but a broader (fourth) class of bearers of human capital ("Human capital") and to include in it other categories of applicants who are interesting for Russia due to their special abilities, talents, and qualities. For example, these are graduates of Russian educational institutions of secondary and higher professional education (who unambiguously speak the Russian language and have been integrated into Russian society during their years of study), winners of Olympiads related to knowledge of Russia, and investors. These individuals can contribute to Russia's development in the same way as the skilled workers.

19. In relation to permanent residence permits and admission to citizenship, the fifth class "Regularization of status" was singled out, which is absent in the case of issuance of temporary residence permit. This class includes not only situations or regularization (the law has a special article for such cases18), but also restoration and recognition of Russian citizenship. Admission to citizenship of persons in regularization order is preceded by obtaining a permanent residence permit on the relevant grounds. Since 2023, it has been possible to allocate a sixth class - "Study". Students of the higher education system now can apply for a special type of temporary residence permit. This class can be distinguished only in the flow of migrants receiving temporary residence permits. When obtaining residence permits and citizenship simplified procedure of issuance of permanent residence permits and citizenship acquisition is defined only regarding graduates of Russian universities, but not students in the process of study. The internal content of each class of grounds is of great interest, but the limited volume of the report does not allow us to present this information in detail.

15 In paragraph 21 of the Concept of State Migration Policy of the Russian Federation for 2019-2025, the first task in the list of objectives is to facilitate the voluntary resettlement to the Russian Federation for permanent residence of compatriots living abroad, as well as other persons who are able to successfully integrate into Russian society. https://base.garant.ru/72092260/#block_1000 16 A category of foreign workers with special terms of employment and guaranteed salary limit, the only class of workers allowed to bring family members and get a special residence permit https://base.garant.ru/184755/7cb9e0279cd84d282026d8d6345390d4/ 17 The list is regularly approved by the Ministry of Labour and Social Protection of the Russian Federation. Currently, it contains 202 positions, 71 of which are medical workers with higher (47) and secondary (24) vocational education. https://ivo.garant.ru/#/document/407038416/paragraph/10:0 18 https://base.garant.ru/184539/96c297d9551856eaebdd4bf791f065eb/#block_4113

Working paper 9

8

IV. The main results

A. Temporary residence permits

20. In 2021-2023, Russia decided to issue almost 443 thousand temporary residence permits, of which - within the established quota - 42 thousand (less than 10%) and outside the quota (with distribution by classes) - 401 thousand. For each year, respectively, the figures (total decisions) amounted to 167 thousand, 181 thousand and 95 thousand TRPs.

Table 1 Distribution of decisions on issuance of temporary residence permits by class of admission, Russia, 2021- 2023, percent. 2021 2022 2023 2021-2023 Within the established quota 13,4 6,6 7,8 9,4 Outside the quota, including: Family reunification 36,7 35,8 39,9 37,0

Ethno-cultural ties with Russia 27,8 24,7 29,6 26,9 Preferences by country of citizenship, birth or residence 17,7 28,9 4,4 19,5

Human capital 4,1 3,1 6,5 4,2 Students (from 2023) n/a n/a 11,3 2,4

Other unspecified categories outside the quota 0,3 0,9 0,5 0,6 Calculated on the basis of the data of the Ministry of Internal Affairs of Russia

21. The main recipients of temporary residence permits were migrants belonging to the “Family reunification” class (Table 1). Their share averaged 37% (164 thousand over three years). Among them, the largest group were the spouses of Russian citizens (153 thousand). In second place (about 27%) were migrants of the "Ethno-cultural ties with Russia" class, which included the only category - participants in the State Program of Assistance to the Resettlement of Compatriots. According to the Ministry of Internal Affairs, about half of the participants of the State Program are family members of the applicants, so they can hypothetically also be classified as migrants through family reunification, which makes this class the largest in terms of the number of applicants for TRP and its share in the flow. At the same time, participants of the State Program are only formally refer to the class "Ethno-cultural ties…", just because of the definition of a compatriot19. In fact, they are subject to the requirements of age (only persons of working age are allowed to participate) and the availability of skills and professions in demand in the region of future residence20. The federal law does not mention these requirements, but since the Program is implemented at the regional level, regional by-laws definitely contain these conditions and criteria. The absence of the

19 https://ivo.garant.ru/#/document/12115694/paragraph/9989:3 , as well as the Federal Law of May 24, 1999 N 99-FZ “On the State Policy of the Russian Federation Concerning Compatriots Abroad” https://ivo.garant.ru/#/document/12115694/paragraph/14189:2 ; the Decree of the President of Russia introduced the concept of a “repatriate”. https://base.garant.ru/408052315/?ysclid=luz86sxd6l95772256 20 The decision to allow (or not) an applicant to participate in the State Program is made by the regional labour and employment office, see the example of the Leningrad Oblast: https://mosaiikki.info/rusinfofinland/texts/regions/documents/reglament_Leningradskay.pdf?ysclid=luz89g 6to2273092653

Working paper 9

9

required profession may be the reason for the refusal of admission of a migrant to participate in the State Program21.

22. Migrants who received a temporary residence permit in the class "Preference by country of citizenship, country of birth or residence" were also numerous. Their share averaged about 20% over three years and reached almost 30% in 2022. The main flow in this class are citizens of Belarus, Kazakhstan, Ukraine and Moldova, who in 2021 received – just on the basis of their citizenship a simplified access to a temporary residence permit and citizenship (from 2023 also to a permanent residence permit). The “Human capital” class accounts for only 4% of the flow. Among those who received temporary residence permit, this class included actually one category: graduates of Russian institutions of higher professional education (diploma without honors, since excellent students can apply immediately for a permanent residence permit). Previously, the class "Human capital" also included investors, but this status was not in demand. Only in 2021, five people of this category received a temporary residence permit. Since 2022, investors have been entitled to apply for permanent residence permits immediately, but the figures for 2022-2023 were zero.

23. Foreign students are not in a hurry to use the right to obtain a special temporary residence permit so far. Therefore, the numbers of issued temporary residence permit "For the purpose of education" in no way reflect the true scale of the presence of international students in Russia. Majority of them keep temporary status and legally stay in Russia with valid visa and/or confirmation from the university. According to the Ministry of Internal Affairs of Russia, in 2023, only 12.7 thousand people applied for a temporary residence permit for the purpose of education, and only 10.1 thousand obtained it during the year. Same time, according to the Ministry of Education and Science, the stock of foreign students in the system of higher education in Russia at the beginning of the academic year 2023/2024 exceeded 356 thousand people. The enrollment of students in Russian universities in 2023 reached 124 thousand people22.

B. Residence permits (permanent)

24. Since November 2019, several categories of foreigners who previously could get an out-of-quota temporary residence permit, have been entitled to apply for a permanent residence permit, bypassing the temporary one23. Therefore, the descriptions of these categories were redirected from the chapter 6 of the Law 115 on temporary residence permits to the chapter 8 on permanent residence permits. The list of persons receiving a permanent residence permit in a simplified manner has been extending. Citizens of Belarus have always had the right to a permanent residence permit without a TRP on the basis of the agreement on the union state Russia-Belarus. Numerous participants of the State program of resettlement of compatriots, who among the recipients of a temporary residence permit made up a very large class "Ethno-cultural ties with Russia", do not need a permanent residence permit. Having a TRP they can immediately apply for citizenship. Therefore, only so-called «native speakers of Russian language» represent this class of residence permit recipients.

25. In 2021, 2022 and 2023, 760 thousand decisions were made to issue permanent residence permits in Russia (248 thousand, 280 thousand and 232 thousand respectively). On average over the three years, 23% of permanent residence permits were issued to temporary residence permit holders, and thus cannot be categorized into any class of admission. Almost 57% of permanent residence

21 https://ruvek.mid.ru/upload/iblock/dd6/Programma-pereseleniya-_podprogramma-3_.pdf 22 https://www.minobrnauki.gov.ru/action/stat/highed/ 23 https://base.garant.ru/72360792/?ysclid=lvdo1mhyus204139908

Working paper 9

10

permits were granted on grounds related to family reunification, and about 14% to preferences related to citizenship, place of birth or residence in a particular country. Other categories were disproportionately smaller (Table 2). Some of the persons who received a permanent residence permit without a temporary one were not assigned to any of the categories identified in the 2-RD statistical report. The unallocated balance amounted to 6,000 in 2021, 7,700 in 2022, and 7,600 in 2023, respectively. The share of this group with unspecified grounds of admission amounted to about 2% of the flow on average over the three years.

26. The class "Regularization" (settlement) includes three positions of Law 115 (paragraphs 12, 13 and 14 of Article 8) and concerns persons whose Russian citizenship has been terminated, the decision on admission to citizenship has been canceled or the previously issued citizenship has been recognized as invalid. The number of persons who received permanent residence permits on this basis amounted to 5.9 thousand people over three years, and their share in the total flow was less than half a percent. The class "Human Capital" includes 7 categories: graduates of Russian professional educational institutions with honors, highly skilled specialists, holders of in-demand professions, investors, from 2023 - winners and finalists of the contest "Russia - the Country of Opportunities"24, IT specialists, etc. The share of migrants of this class did not exceed 2.3% in three years, and their number amounted to 17.2 thousand people.

Table 2 Distribution of decisions on issuance of permanent residence permits by class of admission, Russia, 2021- 2023, Percent

2 021 2 022 2 023 2021-2023

Residence permit on the basis of a residence permit (with unspecified class of receipt)

31,3 20,7 16,7 23,0

Family reunification 51,9 65,1 52,3 56,9 Ethno-cultural proximity to Russia (Russian language speakers) 3,1 1,9 1,2 2,1 Preference for country of citizenship or country of birth and historical justice

11,6 7,1 23,9 13,7

Human capital 1,8 2,4 2,6 2,3 Status regularization 0,03 0,04 0,02 0,03 Other unspecified 0,3 2,8 3,3 2,1

Based on the data of the Ministry of Internal Affairs of Russia

27. Taking into account that it is impossible for one person to obtain a temporary and a permanent residence permit within a year, and the categories do not overlap, we can try to combine the flow of persons receiving temporary and permanent residence permits on comparable classes of admission and estimate the overall structure of the flow. In 2023, the class "family reunification" accounted for 49% of all issued temporary and permanent residence permits, 18% referred to the "Preferences by country of citizenship", 9% - to the class "Ethno-cultural ties with Russia" and 3.3% to the class "Students". Other categories (quota-based temporary residence permits and permanent residence permit on the basis of a TRP, as well as permits not distributed by the grounds of obtaining) accounted for 14% of the flow.

24 https://rsv.ru/?ysclid=luvh5j45xw682605078

Working paper 9

11

C. Citizenship acquisition

28. The statistics on citizenship acquisition are of a particular interest for two reasons. First, they are available in detail by categories of grounds since 2010, i.e., for a significantly longer period of observation than the data on temporary residence permits (since 2014) and especially residence permits (since 2021). Second, the number of categories to be classified is also more numerous. Statistical report 2-RD does not group indicators, even if the grounds for citizenship are essentially similar, so the number of categories included in the reporting is quite large (at least 30). Third, although admission to citizenship is not the same as migration, it can be considered as the resulting phase of the migration process. It should be emphasized that the distribution by admission classes has been shifted in recent years due to the implementation from 2019 of special conditions for Ukrainian citizens, most of whom lived outside Russia at the time of obtaining citizenship and were not migrants.

29. In 2021, 2022 and 2023, a total of 1.8 million people (respectively: 735 thousand, 691 thousand and 379 thousand) acquired Russian citizenship through the bodies of the Ministry of Internal Affairs of Russia. The large values of recent years (the trend started in 2019) are related to the application, as noted above, since 2019, of a special provision of the Law on Citizenship on the admission to Russian citizenship for humanitarian purposes. We are mainly talking about citizens of Ukraine who meet the conditions established in the Decrees of the President of Russia in 201925. We included this category of applicants to the class "Preference by country of citizenship, birth or residence". In 2023, the number of such individuals decreased significantly. In 2021, it amounted for 324,000, in 2022 - 275,000, and in 2023 it dropped to 59,000.

30. Since the new provisions of the law came into effect in 2021 and the format of the table was not modified, there was a significant unallocated balance of 131,000 people by class of grounds. Therefore, we calculated the period average structure by grounds for two years (2022-2023) rather than three years.

31. In classification - 9 categories reflected different variants of kinship relations - spouses, children and parents and three reflected guardianship and custody situations, of which guardianship was the main one and the figures for others were close to zero26. These 12 categories were grouped into the Family Reunification class. 7 categories of applicants belonged to bearers of human capital, 2 - to persons with ethno-cultural ties with Russia (State Program and Russian language speakers), 5 categories were united by the principle of belonging to a certain country of birth or citizenship (including in the past). In the group "Status regularization" we included 4 positions: regularization proper (settlement of the legal status of migrants who arrived many years ago, restoration of citizenship and recognition of citizenship).

32. In 2022-2023 most of naturalized foreigners belonged to "Family Reunification" (43%) and "Preference by country of citizenship, birth or residence" (41%) classes. In the "Family Reunification" class, more than half were parents of Russian citizens, about one-fifth were children

25 https://base.garant.ru/72236792/?ysclid=lutu453pw5678248567 ; https://base.garant.ru/72229888/?ysclid=lutu593mi8719889690 26 In addition to cases of guardianship (over children and incapacitated persons), the law provides for situations of acquisition of citizenship by persons staying in Russian institutions for orphaned children and incapacitated persons placed under supervision in Russian educational, medical, social, etc. organizations. Since we are talking about a collective household, these cases can be conditionally attributed to the "family" category. In three years - 2021-2023, citizenship was granted to 1.5 thousand children and incapacitated persons under guardianship, to 140 persons in organizations for orphaned children, and to 28 persons in social organizations for incapacitated persons.

Working paper 9

12

(including adults), and the rest were the spouses of Russian citizens. In third place were migrants in the "Ethno-cultural ties with Russia" class (15%). The share of bearers of human capital remained very modest (0.4% or 4.7 thousand people over two years), of which the main category was persons with in-demand professions (3.4 thousand) and graduates of Russian universities and colleges (1.2 thousand).

33. The limitation of the classification of grounds for acquiring citizenship is connected with the extension (in 2023) of the list of categories of applicants for the simplified naturalization procedure was expanded. It was done within the framework of the old law. In October 2023, the new law on citizenship came into force27. The approaches it applies to the allocation of categories of applicants are mainly inherited from the previous law.

Table 3 Distribution of persons granted citizenship of the Russian Federation, by class of admission, 2021-2023, percent

2 021 2 022 2 023 2022-2023 Family reunification 16,0 36,2 55,9 43,2

Ethno-cultural ties with Russia 15,2 14,3 16,3 15,0 Preferences by country of citizenship, birth or residence 50,1 48,7 26,5 40,8 Human capital 0,2 0,3 0,6 0,4 Status regularization 0,6 0,5 0,7 0,6 In general procedure (without preferences) 0,05 0,03 0,03 0,03 In simplified procedure not specified categories 17,8 0,0 0,0 0,0

* The calculation for the period is made for the last two years due to a significant balance in 2021, not distributed by the grounds of citizenship. Based on the data of the Ministry of Internal Affairs of Russia.

34. As we have noted earlier, the recipients of citizenship in 2019-2023 were to a large extent non- migrants, they did not require either a residence permit or a residence permit. On so-called humanitarian grounds, 1.2 million Ukrainian citizens acquired Russian citizenship in 2019-2023 (more than 40% of the total number of naturalizations in these years). Most of them did not reside on the territory of Russia at the time of their admission to citizenship. This category radically changed the structure of grounds for admission to citizenship and the ratio of admission classes. If we make a calculation without taking this category into account, in 2021-2023 the share of those naturalized through family reunification would have reached 67% (instead of 42%), and the share of the class "Preference by country of citizenship..." would have fallen to 14% (instead of 41%).

V. The potential of administrative data from the Russian Ministry of Internal Affairs

35. The availability of data that we have worked with in the distribution by country of citizenship of foreigners or by regions of Russia allows us to perform a more in-depth analysis. We can see large differences between citizens of different countries in the choice of channels for obtaining residence permits or citizenship. Alternatively, it is possible to analyze regional differences in law enforcement practice, to see whether the structure of the classes of admission of foreigners differs

27 https://base.garant.ru/406811055/?ysclid=luz75utvdv341998839

Working paper 9

13

in certain regions of Russia. The built-in function of the 2-RD report allows to make uploads for an individual country of citizenship with distribution by all regions of Russia, or vice versa - for an individual region with detailing by country of citizenship of foreigners who received different statuses in this region.

36. To give an example, Table 4 shows the distribution of citizens of several states and stateless persons by classes of grounds for obtaining Russian citizenship in 2023. These countries accounted for 98% of all admissions to Russian citizenship. The minimum value of the number of naturalized persons was 1,029 (Germany28), the maximum - 158 thousand (Tajikistan). Family reunification is by a large margin the main channel of naturalization of citizens of Tajikistan (86%), Georgia (80%), Azerbaijan (78%) and Armenia (70%).

Table 4. Distribution of citizens of individual countries granted citizenship of the Russian Federation, by class of admission, 2023, percent

Family reunify- cation

Ethno- cultural ties with Russia

Preferences based on the country of citizenship, birth or residence

Human capital

Regularization , restoration, recognition

General order, no preferences

Azerbaijan 55,9 16,3 26,5 0,6 0,7 0,03 Armenia 78,6 19,0 1,6 0,3 0,4 0,12 Belarus 69,6 28,8 0,6 0,8 0,1 0,06 Vietnam 6,3 0,3 92,3 0,0 1,0 0 Germany 85,4 12,6 0,0 1,0 0,0 0,95 Georgia 48,8 28,0 13,7 0,4 9,1 0 Kazakhstan 80,4 16,1 1,0 0,1 2,4 0 Kyrgyzstan 9,1 39,9 50,8 0,0 0,2 0 Moldova 71,8 22,8 4,1 1,2 0,1 0,02 Tajikistan 10,0 8,2 81,8 0,0 0,0 0 Turkmenistan 86,0 12,9 0,2 0,8 0,0 0,01 Uzbekistan 41,3 53,9 3,5 0,9 0,4 0,03 Ukraine 56,4 37,6 5,0 0,9 0,1 0,11 Stateless persons 2,1 0,4 97,0 0,0 0,5 0 Other countries 25,0 10,4 5,1 0,2 59,3 0 All countries 67,3 18,9 8,0 4,2 1,2 0,29

Based on the data of the Ministry of Internal Affairs of Russia.

37. Citizens of Ukraine, Belarus and Moldova used the provisions of the law allowing them to obtain a Russian passport in an even more simplified procedure - only on the basis of their country of citizenship: respectively 97%, 92% and 82% of all applicants used this channel. Based on the grounds of the class "Ethno-cultural ties with Russia" 54% of citizens of Turkmenistan, about 40% of citizens of Uzbekistan and Kazakhstan obtained Russian citizenship. Almost 60% of stateless persons were naturalized by grounds related to status settlement.

28 Germany is one of the main destination countries for emigrants from Russia, mostly those who left in the 1990s. Some migrants or their descendants who lost or lacked Russian citizenship and returned to Russia subsequently regain it or acquire it on other grounds. In 2023, 27% of the 1,029 German citizens obtained citizenship through the Russian Ministry of Internal Affairs as participants in the State Program and Russian language speakers. (In 2023, 2,650 people living in Germany obtained citizenship through Russian consulates).

Working paper 9

14

VI. Concluding remarks

38. Suggested classification allows to get a clearer picture of the approaches used in Russia to organize formal channels of moving to Russia and admission to Russian citizenship. The proposed algorithm simplifies the description not only of international migration, but also of the nature of migration policy. Having distributed the detailed grounds for migrants' acquisition of residence status and citizenship into aggregated classes, we can objectively observe the predominance of grounds related to family reunification. This is very important, since the Concept of State Migration Policy of Russia does not reflect this type of migration at all29. Despite the priorities emphasized in the Concept of State Migration Policy - attraction of compatriots and other persons with ethno-cultural ties and proximity to Russia, skilled labour migrants, etc., in practice the structure of flows shows a more humanitarian nature of migration. The clear dominance of the family reunification class of migration, the multiplicity of categories in this class, and the simplification of admission of migrants for family reunification in recent years show Russia's respect for human rights to family life (in accordance with the Universal Declaration of Human Rights) and family reunification.

39. At the same time, the absence of a pronounced labour migration channel in the statistics of residence permits suggests that the policy towards this type of migration is still aimed at maintaining migrants’ temporary status. The absence of full-fledged immigration programs for skilled workers with the right to bring their families and receive other preferences does not allow this flow to grow. De facto, there is a redistribution of skilled workers through other channels and classes, including through their participation in the State Program. Forced migrants, if flows ever increase again, which we would not want, should be given more rights in their own status, rather than being redirected to other classes, as it happens in the current practice. Such terms can be created by appropriate amendments to the law. The hitherto unnoticed class of student migration may increase substantially in the future. The norm introduced in 2023 should become in demand among foreign students, as the temporary residence permit gives much more rights and comfort when living in Russia. Students from visa countries do not need to renew their visas repeatedly, and students from visa-free countries also get an opportunity to ensure a more relaxed - in terms of confirming the right to stay in Russia - mode of stay with one document.

40. The scale of long-term skilled labour migration or migration for the purpose of education, as well as forced migration, and their place in Russia's migration landscape cannot be assessed on the basis of statistics on residence permits or admission to citizenship. Foreign students have the right to live in Russia without applying for a residence permit. Highly skilled specialists can also live in Russia on the basis of a visa, without a residence permit. Persons holding a patent (among whom there may also be skilled workers) can extend it, thereby obtaining the right to long-term residence in Russia. Therefore, the estimates of the volumes of labour or educational migration should be based on relevant special data, respectively, data of the Ministry of Internal Affairs of Russia on the issued permits for migrant workers, data on administrative registrations of potential foreign workers at a place of temporary residence, and data of the Ministry of Science and Education on foreign students.

41. The same applies to forced migration, which is accounted for by the Ministry of Internal Affairs, but the statistics covers only foreigners. The problem of measuring forced migration in 2022-2023 remains the availability of Russian citizenship for the majority of such migrants, which was actively granted to residents of Donbas and then other regions of Ukraine in recent years. In the future, when the new software of the Russian Ministry of Internal Affairs is debugged, it will probably become possible to distinguish migrants with different classes of admission to long-term

29 https://base.garant.ru/72092260/?ysclid=lv0mdzvcoj588550241#friends

Working paper 9

15

stay and residence in Russia, migrants with different status and duration of stay in Russia. Information on their socio-demographic characteristics will also be available.

42. Russia is currently in the process of reforming its migration legislation. In 2023, a new law "On Citizenship of the Russian Federation" was adopted and came into force. As we noted above, it largely reflects the same approaches to granting citizenship and highlights categories that were present in the old law, which should ensure comparability of data in retrospect and in the future. An important draft law in the area of regulating the entry, stay and residence of foreigners in Russia is being prepared for adoption, which will replace several laws currently in force30. The temporary residence permit system will be abolished, although the list of categories of foreigners receiving permanent residence permits will not change significantly. It is likely that the format of statistical report 2-RD will be changed in accordance with the provisions of the future law. In any case, the approach we have proposed to classify the channels for admitting migrants to reside and obtain citizenship in Russia can be applied under the new conditions.

30 Draft federal law "On the conditions of entry (exit) and stay (residence) in the Russian Federation of foreign citizens and stateless persons". https://base.garant.ru/56883645/?ysclid=luz72qscr2190085111

  • I. Introduction
  • II. Data description and its limitations
  • III. Approaches to distinguishing classes of admission for obtaining residence permits and Russian citizenship
  • IV. The main results
    • A. Temporary residence permits
    • B. Residence permits (permanent)
    • C. Citizenship acquisition
  • V. The potential of administrative data from the Russian Ministry of Internal Affairs
  • VI. Concluding remarks
Russian

*Prepared by Ольга Чудиновских, МГУ имени М.В. Ломоносова) NOTE: The designations employed in this document do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries.

Economic Commission for Europe Conference of European Statisticians Group of Experts on Migration Statistics Geneva, Switzerland, 7−8 May 2024 Item 6 of the provisional agenda Improvements in use of administrative data for migration statistics

Классификация международной миграции в России на основании административных данных

Note by МГУ имени М.В. Ломоносова*

Аннотация В работе предлагается подход к выделению основных классов допуска иностранцев к получению вида на жительство и гражданства Российской Федерации и, на этой основе, оценка структуры потока лиц, получающих эти статусы-, по соответствующим классам. Это позволяет увидеть, какие формальные каналы допуска к праву на проживание и натурализацию чаще всего используют иностранцы в России. В отличнее от многих стран, где, например, виды на жительство сразу распределяются по классам-, и такая статистика доступна в агрегированном виде, в России подобная практика почти отсутствует. Статистическая информация такого рода не разрабатывается в регулярном порядке и отсутствует в открытых источниках. На основе анализа административной статистики МВД России, которая отражает число иностранцев, получивших разные статусы в соответствии с многочисленными положениями закона, была предпринята попытка сделать группировку этих положений и выделить классы оснований: воссоединение семей, человеческий капитал, преференции на основании страны гражданства и/или рождения, этнокультурные связи с Россией и др. Это позволило показать, что в потоках лиц, получивших виды на жительство и гражданство, самый большой класс - это «Воссоединение семей». На примере статистики приема в гражданство продемонстрированы большие различия в соотношении классов и каналов натурализации, которые выбирают граждане разных стран. Наличие статистики за предыдущие годы позволяет отслеживать динамику этой структуры в ретроспективе. Отмечены основные уязвимости использованных административных данных - невозможность адекватно отразить образовательную и квалифицированную трудовую миграцию, но делается предположение, что это можно компенсировать с помощью интеграции данных из других источников. Кроме того, предложенный подход показал, что политика России в отношении квалифицированной трудовой миграции направлена на поддержание временного статуса иностранных работников

Working paper 9

Distr.: General 29 April 2024 English

Working paper x

2

I. Введение

1. Исследователи во многих странах мира имеют возможность изучать состав потоков иммиграции по основным классам приема или причинам уже на основании готовых, агрегированных данных, которые публикуют национальные миграционные или статистические агентства1. Если речь идет о миграции иностранцев с правом постоянного проживания, часто используется статистика видов на жительство. Основания для получения вида на жительство дифференцированы по укрупненным классам, что и определяет тип вида на жительство. Публикуемая статистика структурирована по этой переменной, что позволяет увидеть соотношение потоков миграции по линии воссоединения семей, квалифицированной трудовой миграции, миграции с целью обучения, вынужденной миграции и т.д. При обсуждении международной сопоставимости статистики миграции наиболее активная полемика ведется вокруг определения долгосрочных мигрантов, которые оказывают влияние на численность и состав постоянного населения страны. Разные критерии учета, применяемые в отдельных странах, обусловливают хроническую проблему сопоставимости статистики миграции на международном уровне. Критерий времени (длительность проживания в новом месте обычного жительства) не всегда можно эффективно применить при формировании статистики. И использование только этого критерия не всегда позволяет адекватно отразить масштабы и структуру миграционных потоков, поскольку за пределами внимания остается важнейшая характеристика – статус мигранта в принимающей стране. Между тем, с точки зрения реализации миграционной политики эта переменная имеет большое значение.

2. В этом контексте можно предположить, что статистика видов на жительство (и, вероятно, статистика натурализации) в разных странах относительно сопоставима. Даже если применяются разные подходы к выдаче видов на жительство и приему в гражданство, эти статусы подразумевают много общего. Они снимают ограничения при допуске на рынок труда, дают мигранту определенные (или все) политические права, и, что вероятно важнее всего, знаменуют переход большинства получателей таких статусов в состав постоянного населения приминающей страны. Кроме того, статистика видов на жительство и приема в гражданство может показать соотношение каналов долгосрочной миграции иностранцев или их натурализации. Понятно, что речь идет об установленных страной формальных каналах, по которым иностранные граждане прибывают в страну и получают вид на жительство, а затем и гражданство. Истинные мотивы или цели миграции могут быть иными. Тем не менее, важно знать, какие легальные «ворота» выбирают мигранты для переезда в страну, как перераспределяется поток при появлении новых или закрытии старых формальных каналов. Это дает возможность оценить характер миграционной политики страны и результаты ее реализации.

3. Россия является одной из стран с очень большими показателями численности международных

мигрантов и потоков прибывших. Численность пожизненных международных мигрантов, по оценкам Отдела населения ООН, на 2020 год составила 11,6 млн человек2. Только с 2010 по 2023 гг. по данным МВД в России3 было оформлено 5,6 млн видов на жительство (из их временных - 2,4 млн и постоянных

1 US Yearbook of Immigration Statistics 2022. (Table 6. Persons Obtaining Lawful Permanent Resident Status by Type and Major Class of Admission: Fiscal Years 2013 to 2022). https://www.dhs.gov/ohss/topics/immigration/yearbook/2022 ; Federal Ministry of the Interior, Building and Community, Federal Office of Migration and or Refugees (Germany). Migration Report 2021 (p.6); Key findings. https://www.bamf.de/SharedDocs/Anlagen/EN/Forschung/Migrationsberichte/migrationsbericht-2021- zentrale-ergebnisse.pdf?__blob=publicationFile&v=6 2 United Nations Department of Economic and Social Affairs, Population Division (2020). International Migrant Stock 2020. https://www.un.org/development/desa/pd/content/international-migrant-stock 3 Данные были предоставлены по запросу экономическому факультету МГУ имени М.В. Ломоносова.

Working paper

3

- ,3,2 млн) 4, 4,4 млн человек были приняты в гражданство через органы МВД5. Но мы имеем весьма размытое представление о том, какие основания чаще всего используются при принятии властями решений о выдаче иностранцам того или иного статуса. Агрегированной статистики такого рода в России нет, хотя есть информация, позволяющая пролить свет на этот важный вопрос. На протяжении многих лет МВД России (а до 2016 г. – Федеральная миграционная служба) разрабатывает регламентную статистическая отчетность о разрешениях на временное проживание, (постоянных) видах на жительство и приеме в гражданство. Хотя последнее и не является собственно миграцией, но может рассматриваться как её завершающая фаза в смысле изменения статуса мигранта. Эта статистика недоступна в открытых источниках. Но с 2010 года экономический факультет МГУ имени М.В. Ломоносова получает ее по письменному запросу.

II. Описание данных и их ограничения

4. Регламентная статистическая отчетность по форме 2-РД6 МВД России представляет собой файл Microsoft Excel, состоящий из двух таблиц. В первой - названия строк – это страны гражданства мигрантов, во второй – регионы России. Называния колонок отражают разнообразные процедуры или виды документов, выполненные или оформленные в отношении иностранных граждан: например, решения о выдаче видов на жительство, о приеме в гражданство или оформленные разрешения на работу. Отчет по форме 2-РД формируется в трех вариантах: «Разрешительно-визовая работа» (содержит в том числе сведения о видах на жительство и разрешениях на временное проживание), «Гражданство»7 и «Внешняя трудовая миграция». Последний вид статистики в этой работе не рассматривается.

5. Особенность первых двух отчетов состоит в том, что сведения о разрешениях на временное

проживание, видах на жительство или приеме в гражданство детализируются по отдельным положениям соответствующих законов: закона «О правовом положении иностранных граждан…»8 (так называемый «Закон № 115») и закона «О гражданстве…» («Закон № 62»)9. Формулировки закона - описание разных категорий иностранцев, которые имеют право получить статус, весьма многословны. Их трудно использовать в качестве заголовков колонок в таблице. Поэтому эти формулировки «кодируются». Визуально в названиях столбцов это представлено как набор аббревиатур, обозначающих соответствующие подпункты или пункты отдельных статей закона. Например, в статистике видов на жительство, указано, помимо общего количества принятых решений, сколько РВП в том числе должны быть выданы на основании подпункта 1 пункта 2 статьи 8 №115-ФЗ; - подпунктов 2, 6 пункта 2 статьи 8 Федерального закона № 115-ФЗ; подпунктов 3, 5 пункта 2 статьи 8 Федерального закона № 115-ФЗ и так далее. Если обратиться к тексту закона (№ 115), то мы получим расшифровку этих аббревиатур.

4 Данные частично перекрываются, т.к. большинство лиц с РВП впоследствии должны получить вид на жительство. 5 Прием в гражданство России оформляется также через заграничные учреждения Российской Федерации (консульства), учет ведется отдельно. 6 https://base.garant.ru/70145636/?ysclid=lvbfg231r9190591739 7 В таблицах с данными по приему в гражданство выделяются категории - дети и взрослые. За последнее пятилетие доля взрослых составляла 76%, детей -24%. 8 Федеральный закон от 25 июля 2002 г. N 115-ФЗ "О правовом положении иностранных граждан в Российской Федерации" (с изменениями и дополнениями) https://base.garant.ru/184755/?ysclid=lus031am7j84734668 9 Федеральный закон "О гражданстве Российской Федерации" от 31.05.2002 N 62-ФЗ (утратил силу с 26 октября 2023 г. в связи с вступлением в силу нового закона (см. Федеральный закон от 28 апреля 2023 г. N 138-ФЗ "О гражданстве Российской Федерации") https://base.garant.ru/406811055/?ysclid=lus0ao6zn9102234638

Working paper x

4

6. Подпункт 1 пункта 2 статьи 8 относится к иностранцам, которые родились на территории РСФСР и состояли в прошлом в гражданстве СССР; подпункты 2 и 6 пункта 2 статьи 8 – это дети до 18 лет (пп. 2) или недееспособные взрослые (пп.6), родители или попечители которых имеют вид на жительство и постоянно проживают в России; подпункт 4 – это иностранцы, имеющие родителя/усыновителя, опекуна, либо сына или дочь, которые являются гражданами России и постоянно проживают в Российской Федерацию, пункт 9 - это высококвалифицированные специалисты и т.д.

7. Аналогично представлена статистика приобретения российского гражданства. Аббревиатуры в

названиях столбцов таблицы отсылают нас к тексту закона, из которого можно понять, например, что лица, получившие гражданство по подпункту «б» части 2 статьи 14 ФЗ-62 – это иностранцы, которые состоят не менее трех лет в браке с гражданином Российской Федерации, проживающим на территории Российской Федерации. Подпункт «в» части 2 статьи 14 касается принятых в гражданство иностранцев, имеющих дееспособного, совершеннолетнего ребенка-гражданина Российской Федерации. Лица, получившие российское гражданство на основании части 2.1. статьи 14 ФЗ-62 – это т.н. «носители русского языка» и т.д.

8. Разработчики таблицы 2-РД уже на этапе проектирования объединили некоторые однородные по

смыслу категории мигрантов. В сумме, например, приводится статистика по несовершеннолетним детям и недееспособным взрослым, получающим статус одновременно с родителями или потому, что у родителей этот статус уже есть. В законе эти категории указаны в разных подпунктах. Выделяя общее в основном содержании этих формулировок, мы смогли с определенной степенью условности объединить их в укрупненные классы и посмотреть, какие основания являются главными при выдаче временных и постоянных видов на жительство и при приеме в гражданство РФ.

9. Изменения в законодательстве предполагают соответствующие изменения в количестве и содержании

колонок в статистическом отчете 2-РД. Когда поправки к закону вводят новые категории лиц с особыми условиями получения вида на жительство или приема в гражданство (или отменяют какие-то из прежних), то в таблице создаются (или удаляются) соответствующие колонки. Эти изменения не всегда удается сделать оперативно. Поэтому в таблице за период, в течение которого эти изменения имели место, может, например, остаться нераспределенный остаток – совокупность мигрантов с неуточненным основанием получения статуса. Но эти проблемы со временем решаются. Мы столкнулись с этой проблемой в отношении данных за 2021 год по приему в гражданство - нераспределенный остаток был весьма значительным и составил почти 18% всего количества натурализованных лиц.

10. В этой работе мы использовали имевшиеся в нашем распоряжении данные по основаниям получения

иностранцами трех видов статуса – 1) «временно проживающего» - по разрешению на временное проживание (выдаваемое по квоте и вне квоты, в упрощенном порядке), 2) «постоянно проживающего»10 - на основании (постоянного) вида на жительство и 3) гражданства РФ. Статистика по разрешениям на временное проживание (РВП), видам на жительство и приему в гражданство во многом отражает разные популяции иностранцев. Формально, первым статусом на пути к гражданству является разрешение на временное проживание, которое через год дает иностранцу право обратиться за видом на жительство, а затем и за гражданством. Но имеются многочисленные исключения, которые касаются весьма больших по численности совокупностей соискателей. Например, участники

10 В Законе 115 даны определения: «временно проживающий в Российской Федерации иностранный гражданин - лицо, получившее разрешение на временное проживание или разрешение на временное проживание в целях получения образования; постоянно проживающий в Российской Федерации иностранный гражданин - лицо, получившее вид на жительство». https://base.garant.ru/184755/741609f9002bd54a24e5c49cb5af953b/

Working paper

5

Государственной программы содействия переселению соотечественников11 всегда могли на основании разрешения на временное проживание сразу обращаться за гражданством, вид на жительство им не требовался. С 2021 года почти два десятка категорий соискателей могут, минуя РВП, обращаться за видом на жительство (а затем и гражданством). Ранее это было доступно только гражданам Беларуси. С 2019 года массовым стал прием в гражданство по особым основаниям (в гуманитарных целях), без предварительного получения РВП или вида на жительство. К 2024 году эта практика существенно сузилась.

11. Нужно отметить, что перечни оснований для получения РВП, вида на жительство и гражданства в

значительно степени различаются как по количеству категорий, так по формулировкам, приведенным в соответствующих законах. Поэтому наполняемость классов в смысле оснований для получения РВП, вида на жительство и приема в гражданство - разная. Например, среди лиц, получивших РВП, в класс «Этнокультурные связи с Россией» вошли только участники Госпрограммы (потому что они получают гражданство, минуя постоянный вид на жительство). Среди получателей постоянного вида на жительство класс «Этнокультурные связи» был представлен только носителями русского языка. А среди лиц, принятых в российское гражданство, этот класс включил в себя обе категории.

12. Также мы выяснили, что особенностью российского законодательства является отсутствие жесткой

связи или преемственности между основаниями, по которым иностранец получает РВП, вид на жительство или гражданство. Например, человек, получивший вид на жительство без РВП как носитель востребованной профессии, может обратиться за гражданством в другом качестве, например, как супруг гражданки России, особенно, если в браке есть общий ребенок. Мигранты избирают наиболее быстрый и удобный алгоритм получения статуса, требующий, по-видимому, наименьшее количество документов. И при появлении новых, более удобных возможностей, перераспределение потоков происходит очень быстро, что показывает высокую адаптивность мигрантов к меняющимся обстоятельствам. Поэтому, каждый вид данных – по РВП, видам на жительство и приобретению гражданства имеет смысл рассматривать отдельно. По-видимому, можно объединить, например, данные по РВП и видам на жительство за один и тот же календарный год, т.к. в течение года один человек не может получить оба статуса и данные не перекрываются. Но классы оснований будут иметь разное наполнение. Статистика приема в гражданство может считаться «результирующим» представлением оснований, если не самой миграции, то ее завершающего этапа (в смысле статуса мигранта).

13. При выделении классов оснований мы столкнулись с объективными ограничениями. Статистика по

разрешениям на временное проживание детализируется (с 2014 года) только по РВП, которые выдаются вне установленной квоты. Не известно, какие критерии применяются при выдаче РВП по квоте; (предполагаем, что учитывается наличие работы и родственников в России). Статистика видов на жительство детализируется только с 2021 года и в отношении тех документов, которые были выданы без РВП. То есть, виды на жительство, которые были получены на основании РВП также нельзя распределить по основаниям получения. Только статистика по вопросам гражданства разрабатывается в деталях и начиная с 2010 года (по странам и регионам).

14. Данные отражают только потоки лиц, получающих, соответственно, РВП, виды на жительство или

гражданство. По контингентам иностранных граждан, имеющих действительные виды на жительство или РВП, данные не разрабатываются. Временной интервал, на котором мы демонстрируем основные результаты работы, охватывает три последних года (2021-2023), т.к. это был период наиболее значительных изменений в законодательстве в отношении категорий соискателей видов на жительство

11 Это единственная полноценная иммиграционная программа, существующая в России и формально имеющая репатриационный характер. Программа действует с 2006 года. По данным МВД России, на 31 декабря 2023 года на учете состояли 1 млн 128 тыс. человек, принявших участие в Госпрограмме с момента ее запуска.

Working paper x

6

и приема в гражданство. Задачей этой работы является в большей степени демонстрация потенциала административных данных, нежели анализ и объяснение трендов и колебаний показателей. Поскольку статистика приема в гражданство доступна с 2010 года, у нас была возможность выполнить ретроспективный анализ данных – с учетом изменений законодательства. Такого рода разработки мы выполнили несколько лет назад12. Мы увидели, каким образом изменения в законодательстве меняли распределение потоков по классам допуска к гражданству. Мигранты быстро адаптируются к усложняющимся условиям и ищут более удобные каналы натурализации, с более простыми и выполнимыми требованиями. И это со временем ведет к очень значительным различиям между гражданами отдельных государств в выборе каналов натурализации.

III. Подходы к выделению классов оснований для получения видов на жительство и гражданства РФ

15. Традиционный подход – выделение класса миграции по линии воссоединения семей, трудовой, учебной и вынужденной миграции, оказался применим лишь частично, в том числе, из-за особенностей российского законодательства в сфере миграции и приема в гражданство. При объединении отдельных оснований получения РВП, вида на жительство или гражданства внимание обращалось на смысловое содержание каждой категории. В итоге нам удалось выделить несколько классов: 1. Воссоединение семей; 2. Этнокультурные связи с Россией; 3. Преференции по принципу страны гражданства или рождения; 4. Носители человеческого капитала; 5. Урегулирование статуса. Самостоятельный (шестой) класс лиц, прибывших на учебу, появился только в 2023 году и только в отношении РВП. С 2023 года введен особый вид разрешения на временное проживание «с целью получения образования»13.

16. Мы не смогли выделить класс «Убежище», хотя в законах есть специальные положения, относящиеся к

таким мигрантам. Но в контексте статусов, о которых мы пишем, данные по беженцам статистически незначимы, а по лицам, получившим временно убежище, отсутствуют. В соответствии с пп. «в» части 2 статьи 13 закона «О гражданстве»- , за пять лет с 2019 по 2023 гг. были приняты в гражданство РФ всего 42 лица со статусом беженца (в том числе – в 2023 г. - 3 человека). В России статус беженца практически не предоставляется, и на конец 2023 г. на учете в качестве беженцев состояли всего 244 человека14. Основная форма защиты мигрантов в России – временное убежище. Её широкое применение началось во время первой волны вынужденных мигрантов из юго-восточных регионов Украины в 2014-2015 гг. Тогда же было принято решение практически незамедлительно разрешить лицам, получившим временное убежище, присоединиться к государственной программе переселения соотечественников. Все последующие статусы – разрешение на временное проживание и гражданство (минуя вид на жительство) они получали уже как участники Госпрограммы содействия добровольному переселению соотечественников (репатриантов). Поэтому вынужденные мигранты фактически оказались в ином классе допуска после того, как изменился их первоначальный статус.

17. Первый класс - «воссоединение семей» было относительно легко выделить. К нему относятся ситуации,

связанные с получением статуса несовершеннолетним ребенком по заявлению родителя, уже имеющего такой статус или одновременно с ним, иностранными супругами, взрослыми детьми или

12 Адаптация и интеграция мигрантов в России: вызовы, реалии, индикаторы: [монография] / В. И. Мукомель, К. С. Григорьева, Г. А. Монусова [и др.]; отв. ред. В. И. Мукомель, К. С. Григорьева ; ФНИСЦ РАН. – М. 2022. DOI: 10.19181/monogr.978-5-89697-407-9.2022. стр. 69-73. URL: https://www.fnisc.ru/publ.html?id=11644&type=publ 13 Этот вид документа предполагает учебу по полным образовательным программам в системе высшего образования. 14 Росстат 2024. Доклад «Социально-экономическое положение России». Январь 2024, стр. 187 https://rosstat.gov.ru/storage/mediabank/osn-01-2024.pdf

Working paper

7

родителями граждан России. Второй класс мы связали с наличием у мигранта определенных этнокультурных связей с Россией, т.к. этот подход к предоставлению мигрантам преференций подчеркивается буквально во всех основополагающих документах, связанных с миграционной политикой России15. К этому классу были отнесены участники Государственной программы содействия добровольному переселению соотечественников и носители русского языка. Третий класс преференций сформирован по «географическому» принципу, который предусматривает наличие у соискателя гражданства определенной страны, рождение или проживание на определенной территории (в настоящем или прошлом). Такой подход применяется в нескольких случаях и объединяет, например, лиц, родившихся на территории РСФСР и имевших гражданство СССР, граждан Беларуси, Украины, Казахстана и Молдовы, граждан государств, имеющих с Россией соглашения.

18. Наибольшие трудности вызвала попытка выделения класса миграции с целью работы

(квалифицированной трудовой миграции). Формально к нему можно отнести (и эти категории отдельно перечисляются в законодательстве) высококвалифицированных специалистов и членов их семей16, носителей востребованных профессий17 и IT специалистов (эта категория введена с 2023 года). Но в итоге мы пришли -к выводу, что более правильным будет выделить класс не просто «квалифицированных работников», а более широкий (четвертый) класс носителей человеческого капитала («Человеческий капитал») и включить в него и другие категории соискателей, которые интересны для России как носители особых способностей, талантов и качеств. Например, это выпускники российских учебных заведений среднего и высшего профессионального образования (однозначно владеющие русским языком и за годы учебы интегрированные в российское общество), победители олимпиад, связанных со знанием России, инвесторы. Эти лица могут внести свой вклад в развитие России так же, как и квалифицированные работники, о которых шла речь выше.

19. В отношении постоянных видов на жительство и приема в гражданство и был выделен пятый класс

«Урегулирование статуса», который отсутствует при выдаче РВП. К нему мы отнесли не только собственно ситуации урегулирования (в законе есть особая статья для таких случаев18), но и восстановление в гражданстве или признание наличия гражданства РФ. Приему в гражданство лиц в порядке урегулирования предшествует получение ими вида на жительство по соответствующему основанию. С 2023 года появилась возможность выделения шестого класса - «Учеба». Студенты системы высшего образования получили право обратиться за специальным видом РВП. Этот класс можно выделить только в потоке мигрантов, получающих разрешения на временное проживание. При получении вида на жительство и гражданства он не определяется упрощенный порядок получения вида на жительство и гражданства предусмотрен для выпускников российских вузов, а не для студентов в процессе обучения. Внутренне наполнение каждого класса оснований представляет большой интерес, но ограниченный объем доклада не позволяет представить эту информацию в деталях.

15 В пункте 21 Концепции государственной миграционной политики РФ на 2019-2025 гг., в перечне задач

первой называется задача содействия добровольному переселению в Российскую Федерацию на постоянное место жительства соотечественников, проживающих за рубежом, а также иных лиц, которые способны успешно интегрироваться в российское общество. https://base.garant.ru/72092260/#block_1000

16 Категория иностранных работников с особыми условиями найма и гарантированным лимитом заработной платы, единственный класс работников, которым разрешено привозить с собой членов семьи и получать специальный вид на жительство. https://base.garant.ru/184755/7cb9e0279cd84d282026d8d6345390d4/

17 Перечень регулярно утверждается Министерством труда и социальной защиты РФ. В настоящее время в нем 202 позиции, большинство из них 71 – медицинские работники с высшим (47) и средним (24) профессиональным образованием. https://ivo.garant.ru/#/document/407038416/paragraph/10:0

18 https://base.garant.ru/184539/96c297d9551856eaebdd4bf791f065eb/#block_4113

Working paper x

8

IV. Основные результаты

A. Разрешения на временное проживание

20. В 2021-2023 гг. в России было принято решений о выдаче почти 443 тысячи разрешений на временное проживание, из них – в пределах установленной квоты 42 тыс. (менее 10 %) и вне квоты (с распределением по классам) – 401 тыс. За каждый год, соответственно, показатели (всего решений) составили 167 тыс., 181 тыс. и 95 тыс. РВП.

Таблица 1. Распределение принятых решений о выдаче разрешений на временное проживание по классам оснований, Россия, 2021-2023, проценты 2021 2022 2023 2021-2023 В рамках установленной квоты всего 13,4 6,6 7,8 9,4 Вне квоты, в том числе: Воссоединение семей 36,7 35,8 39,9 37,0

Этнокультурные связи с Россией 27,8 24,7 29,6 26,9 Преференции по стране гражданства, рождения или

проживания 17,7 28,9 4,4 19,5

Человеческий капитал 4,1 3,1 6,5 4,2 Студенты (с 2023 г.) Не применимо Не применимо 11,3 2,4

Прочие неуточненные категории вне квоты 0,3 0,9 0,5 0,6 Рассчитано по данным МВД России

21. Основными получателями РВП были лица, отнесенные к классу «Воссоединение семей» (таблица 1). Их доля в среднем составила 37% (164 тыс. за три года). Среди них самой большой группой были супруги граждан России (153 тыс.). На втором месте (около 27%) были мигранты класса «Этнокультурные связи с Россией», в который вошла единственная категория – участники Государственной программы содействия переселению соотечественников. По данным МВД, около половины участников Госпрограммы - это члены семей заявителей, поэтому они гипотетически тоже могут быть отнесены к мигрантам по линии воссоединения семей, что делает этот класс самым большим по числу соискателей РВП и по удельному весу в потоке. В то же время, участники Госпрограммы только формально относятся к классу «Этнокультурные связи», на основании определений соотечественника19. На деле, к ним предъявляются требования к возрасту (допускается участие только лиц трудоспособного возраста) и наличия квалификации или профессии, востребованной в регионе будущего проживания20. В законе это не упоминается, но подзаконные акты, принятые на региональном уровне, совершенно определенно содержат это требование. Отсутствие необходимой профессии может служить основанием для отказа в допуске мигранта к участию в Госпрограмме21.

19 https://ivo.garant.ru/#/document/12115694/paragraph/9989:3 , а также Федеральный закон от 24 мая 1999 г. N 99-ФЗ "О государственной политике Российской Федерации в отношении соотечественников за рубежом" https://ivo.garant.ru/#/document/12115694/paragraph/14189:2; Указ Президента России ввел понятие репатрианта https://base.garant.ru/408052315/?ysclid=luz86sxd6l95772256 20 Решение о допуске соискателя к участию в Госпрограмме принимает региональное ведомство по труду и занятости, см. пример Ленинградской области: https://mosaiikki.info/rusinfofinland/texts/regions/documents/reglament_Leningradskay.pdf?ysclid=luz89g 6to2273092653 21 https://ruvek.mid.ru/upload/iblock/dd6/Programma-pereseleniya-_podprogramma-3_.pdf

Working paper

9

22. Мигранты, получившие РВП в классе "Преференции по стране гражданства, стране рождения или проживания" также были многочисленны. Их доля в среднем за три года составила почти 20%, а в 2022 г. достигала почти 30%. Основной поток в этом классе - это граждане Беларуси, Казахстана, Украины и Молдовы, получившие на основании своего гражданства упрощенный допуск к РВП (а с 2023 года и к виду на жительство). С 2021 года для них было также предусмотрено упрощение приема в гражданство. Класс "Человеческий капитал" составляет всего 4 % потока. Среди получивших РВП этот класс включил фактически одну категорию – выпускников российских учебных заведений высшего профессионального образования (диплом без отличия, т.к. отличники могут обращаться сразу за постоянным видом на жительство). Ранее к классу «Носители человеческого капитала» могли быть отнесены также инвесторы, но этот статус среди них был не востребован. Только в 2021 г. РВП получили пять человек этой категории. С 2022 г. инвесторы получили право сразу обратиться за видом на жительство, но за 2022-2023 гг. показатели были нулевыми.

23. Иностранные студенты пока не спешат пользоваться правом на получение специального разрешения на

временное проживание. Поэтому показатели по РВП «с целью получения образования» никоим образом не отражают истинных масштабов присутствия международных студентов в России. Большинство из них имеют временный статус и легально находятся в России, имея действующую визу и/или подтверждение от университета. По данным МВД России, в 2023 г. заявления на выдачу РВП с целью получения образования подали всего 12,7 тыс. человек, оформили его в течение года только 10,1 тыс. человек. А по данным Министерства образования и науки22 численность иностранных студентов в системе высшего образования России в начале 2023/2024 ученого года превышала 356 тыс. человек. Прием студентов в российские вузы в 2023 г. достиг 124 тыс. человек.

B. Виды на жительство

24. С ноября 2019 года несколько категорий иностранцев, ранее получавших вне квоты РВП, получили право сразу обратиться за видом на жительство23. Поэтому они были исключены из главы о разрешения на временное проживание и включены в главу о видах на жительство. Перечень лиц, получающих сразу вид на жительство, расширяется. Граждане Беларуси всегда имели право на вид на жительство без РВП на основании договора о союзном государстве Россия-Беларусь. Многочисленные участники Государственной программы переселения соотечественников, которые среди получателей РВП составляли весьма объемный класс «Этнокультурные связи с Россией», не нуждаются в виде на жительство: имея РВП они сразу могут обратиться за гражданством. Поэтому этот класс получателей вида на жительство представлен только т.н. «носителями русского языка».

25. В 2021, 2022 и 2023 гг. в России было принято 760 тысяч решений о выдаче постоянного вида на

жительство (соответственно 248 тыс., 280 тыс. и 232 тыс.). В среднем за три года 23% видов на жительство оформлялись на основании РВП, которые невозможно отнести к какому-либо классу приема. Почти 57% постоянных видов на жительство были получены по основаниям, связанным с воссоединением семей, и около 14% - на основе преференций, связанных с гражданством или местом рождения в определенной стране. Прочие категории были несоизмеримо меньше (таблица 2). Часть лиц, получивших вид на жительство без РВП, не были отнесены ни к одной из выделенных в отчете 2- РД категорий. Нераспределенный остаток составил, соответственно, 6 тыс. в 2021, 7,7 тыс. в 2022 и 7,6 тыс. в 2023 гг. Удельный вес этой группы с неуточненными основаниями в среднем за три года был равен примерно 2% потока.

26. Класс «Урегулирование статуса» включает три позиции Закона 115 (пп. 12, 13 и 14 статьи 8) и касается

лиц, у которых было прекращено гражданство РФ, отменено решение о приеме в гражданство или

22 https://www.minobrnauki.gov.ru/action/stat/highed/ 23 https://base.garant.ru/72360792/?ysclid=lvdo1mhyus204139908

Working paper x

10

ранее оформленное гражданство признано недействительным. Количество лиц, получивших на этом основании вид на жительство, за три года составило 5,9 тыс. человек, а их доля в общем потоке была меньше половины процента. Класс «Человеческий капитал» включает 7 категорий, в том числе, выпускников российских профессиональных учебных заведений с отличием, высококвалифицированных специалистов, носителей востребованных профессий, инвесторов, с 2023 года – победителей и финалистов конкурса «Россия - страна возможностей»24, IT специалистов и др. Удельный вес мигрантов этого класса за три года не превысил 2,3%, а их число составило 17,2 тыс. человек.

Таблица 2 Распределение принятых решений о выдаче (постоянных) видов на жительство по классам оснований, Россия, 2021-2023, проценты

2 021 2 022 2 023 2021-2023 ВНЖ на основании РВП (с неуточненным классом получения)

31,3 20,7 16,7 23,0

Воссоединение семей 51,9 65,1 52,3 56,9 Этнокультурная близость к России (носители русского языка)

3,1 1,9 1,2 2,1

Преференции по стране гражданства или стране рождения и историческая справедливость

11,6 7,1 23,9 13,7

Человеческий капитал 1,8 2,4 2,6 2,3 Урегулирование статуса 0,03 0,04 0,02 0,03 Прочие неуточненные 0,3 2,8 3,3 2,1

Рассчитано по данным МВД России

27. Учитывая, что получение РВП и вида на жительство одним человеком в течение года невозможно, и категории не перекрываются, можно попытаться объединить поток лиц, получающих временный и постоянный виды на жительство по сопоставимым классам оснований и оценить общую структуру потока. В 2023 году на класс "воссоединение семей пришлось 49% всех оформленных РВП и видов на жительство, 18% - на "Преференции по стране гражданства", 9% - на класс "Этнокультурные связи с Россией" и 3,3% составили класс "студенты". Прочие категории (РВП по квоте и вид на жительство на основании РВП, а также совокупность лиц, не распределенных по основаниям получения вида на жительство) составили 14% потока.

C. Прием в гражданство

28. Статистика приема в гражданство представляет особый интерес по двум причинам. Во-первых, в деталях по категориям оснований она доступна с 2010 года, т.е., за существенно большее число лет наблюдения, чем данные по РВП и видам на жительство. Во-вторых, число самих категорий, которые поддаются классификации, тоже более многочисленно. В таблице 2-РД показатели не группируются, даже если основания получения гражданства по существу похожи, потому число категорий, включенных в отчетность, достаточно большое (их не менее 30). В-третьих, хотя прием в гражданство не то же самое, что миграция, она может рассматриваться как результирующая фаза миграционного процесса. Но надо помнить, что распределение по классам допуска в последние годы смещено из-за применения с 2019 г. особых условий для граждан Украины, большая часть которых на момент получения гражданства жили за пределами России и не были мигрантами.

24 https://rsv.ru/?ysclid=luvh5j45xw682605078

Working paper

11

29. В 2021, 2022 и 2023 гг. российское гражданство через органы МВД России приобрели в сумме 1,8 млн человек (соответственно: 735 тыс., 691 тыс. и 379 тыс.). Большие величины последних лет (тренд начался в 2019 г.) связаны с применением, как отмечалось выше, с 2019 года особого положения Закона о гражданстве о приеме в гражданство РФ в гуманитарных целях. Речь преимущественно идет о гражданах Украины, соответствующих условиям, установленным в соответствующих Указах Президента России, подписанных в 2019 г.25. Мы отнесли эту категорию соискателей к классу «Преференции по стране гражданства, рождения или проживания». В 2023 г. число таких лиц существенно сократилось. В 2021 г. их число составило 324 тыс. человек, в 2022 – 275 тыс., а в 2023 сократилось до 59 тыс.

30. Поскольку в 2021 году начали действовать новые положения закона, а формат таблицы не был

модифицирован, имелся значительной нераспределенный по классам оснований остаток, составивший 131 тыс. человек. Поэтому среднюю за период структуру по основаниям мы рассчитали не за три, а за два года (2022-2023).

31. При классификации 9 категорий отражали разные варианты родственных отношений - супруги, дети и

родители и три - ситуации опеки и попечительства, из которых опека была основной, а показатели по прочим26 близки к нулю. Эти 12 категорий были объединены в класс «Воссоединение семей». 7 категорий соискателей относились к носителям человеческого капитала, 2 - к лицам с этнокультурными связями с Россией (Госпрограмма и носители русского языка), 5 категорий объединял принцип принадлежности к определенной стране рождения или гражданства (в т.ч. в прошлом). В группу "Урегулирование статуса" мы включили 4 пункта (собственно урегулирование, в особенности правового положения мигрантов, прибывших в отделенные годы, восстановление в гражданстве и признание наличия гражданства).

32. В 2022-2023 гг. среди натурализованных лиц основными классами были «Воссоединение семей» (43%)

и «Преференции по стране гражданства, рождения или проживания» (41%). В классе «Воссоединение семей» более половины составляли родители граждан России, около пятой части – дети (включая взрослых), остальное пришлось на супругов граждан РФ. На третьем месте были мигранты класса «Этнокультурные связи с Россией» (15%). Весьма скромной оставалась доля носителей человеческого капитала (0,4% или 4,7 тыс. человек за два года), из которых основной категорией были лица с востребованными профессиями (3,4 тыс.) и выпускники российских вузов и колледжей (1,2 тыс.)

33. Ограничение классификации оснований приобретения гражданства связано с тем, что в 2023 году – в

рамках действия старого закона, был расширен перечень категорий соискателей для упрощенного порядка натурализации. А в октябре 2023 г. начал действовать новый закон о гражданстве27. Подходы, которые в нем применяются к выделению категорий соискателей, в основном унаследованы от прежнего закона.

25 https://base.garant.ru/72236792/?ysclid=lutu453pw5678248567 ; https://base.garant.ru/72229888/?ysclid=lutu593mi8719889690 26 Помимо случаев опеки и попечительства (над детьми и недееспособными лицами), в законе предусмотрены ситуации приобретения гражданства лицами, находящихся в российских учреждениях для детей-сирот и недееспособными лицами, помещенными под надзор в российские образовательные, медицинские, социальные и др. организации. Поскольку речь идет о коллективном домохозяйстве, эти случаи можно условно отнести к «семейной» категории. За три года -2021-2023 гражданство получили 1,5 тыс. детей и недееспособных лиц, находящихся под опекой, 140 лиц, находящихся в организациях для детей-сирот, и 28 лиц - в социальных организациях для недееспособных лиц. 27 https://base.garant.ru/406811055/?ysclid=luz75utvdv341998839

Working paper x

12

Таблица 3. Распределение лиц, приобретших гражданство Российской Федерации, по классам оснований, 2021-2023, проценты

2 021 2 022 2 023 2022-2023 Воссоединение семей 16,0 36,2 55,9 43,2

Этнокультурные связи с Россией 15,2 14,3 16,3 15,0 Преференции по стране гражданства, рождения или проживания 50,1 48,7 26,5 40,8 Человеческий капитал 0,2 0,3 0,6 0,4 Урегулирование 0,6 0,5 0,7 0,6 В общем порядке (без преференций) 0,05 0,03 0,03 0,03 В упрощенном порядке не уточненные категории 17,8 0,0 0,0 0,0

* Расчёт за период выполнен по двум последним годам из-за значительного, нераспределённого по основаниям получения гражданства, остатка в 2021 г. Рассчитано по данным МВД России.

34. Как было отмечено ранее, получателями гражданства в 2019-2023 гг. в значительной мере были лица, не являющиеся мигрантами, которым не требовалось ни РВП, ни вида на жительство. По так называемым гуманитарным основаниям, в 2019- 2023 гг. были приняты в российское гражданство 1,2 миллиона граждан Украины (более 40% общего числа принятых в гражданство РФ в эти годы). Большинство из них на момент приема в гражданство не проживали на территории России. Эта категория коренным образом изменила структуру оснований приема в гражданство и соотношение классов допуска. Если сделать расчет без учета этой категории, то в 2021-2023 гг. доля натурализованных по линии воссоединения семей достигла бы 67% (а не 42%), а доля класса «Преференции по стране гражданства…» снизилась бы до 14% (вместо 41%).

V. Потенциал административных данных МВД России

35. Наличие данных, с которыми мы работали, в распределении по странам гражданства иностранцев или по регионам России позволяет выполнять более углубленный анализ, показать большие отличия между гражданами разных стран в выборе каналов получения вида на жительство или гражданства. Или же, проанализировать региональные различия в правоприменительной практике, увидеть, отличается ли структура классов допуска иностранцев в отдельных регионах страны. Встроенная функция отчетности 2-РД позволяет делать выгрузки по отдельной стране гражданства с распределением по регионам России, или наоборот - по отдельному региону с детализацией по странам гражданства иностранцев, которые в этом регионе получали различные статусы.

36. В качестве примера в таблице 4 приведено распределение граждан ряда государств и лиц без

гражданства по классам оснований получения российского гражданства в 2023 году. На эти страны пришлось 98% всех принятых в гражданство России. Минимальная величина числа натурализованных составила 1029 человек (Германия28), максимальная – 158 тыс. (Таджикистан). Воссоединение семей с большим отрывом является главным каналом натурализации граждан Таджикистана (86%), Грузии (80%), Азербайджана (78%) и Армении (70%).

28 Германия является одной из основных стран назначения эмигрантов из России , в основном выбывших в 1990-е годы. Некоторые мигранты или их потомки, утратившие или не имеющие российского гражданства, и вернувшиеся в Россию, впоследствии восстанавливают его или приобретают по другим основаниям. В 2023 г. 27% из 1029 граждан Германии получили через органы МВД России гражданство как участники Госпрограммы и носители русского языка. (Через российские консульства гражданство в 2023 г. оформили 2650 человек, проживающих в Германии).

Working paper

13

37. Граждане Украины, Беларуси и Молдовы использовали положения закона, позволяющие в еще более упрощенном порядке - только на основании страны гражданства- получить российский паспорт: соответственно 97%, 92% и 82% всех соискателей воспользовались этим каналом. По основаниям класса «Этнокультурные связи с Россией» получили гражданство 54% граждан Туркменистана, около 40% граждан Узбекистана и Казахстана. Почти 60% лиц без гражданства были натурализованы в порядке урегулирования статуса.

Таблица 4. Структура оснований приобретения гражданства России гражданами отдельных государств, по классам допуска, 2023 г., проценты

Воссо- едине- ние семей

Этноку льтурн ые связи с Россие й

Преференции по стране гражданства, рождения или проживания

Человеческ ий капитал

Урегулирован ие, восстано- вление, признание

В общем порядке без преференц ий

Все страны 55,9 16,3 26,5 0,6 0,7 0,03 Азербайджан 78,6 19,0 1,6 0,3 0,4 0,12 Армения 69,6 28,8 0,6 0,8 0,1 0,06 Беларусь 6,3 0,3 92,3 0,0 1,0 0 Вьетнам 85,4 12,6 0,0 1,0 0,0 0,95 Германия 48,8 28,0 13,7 0,4 9,1 0 Грузия 80,4 16,1 1,0 0,1 2,4 0 Казахстан 9,1 39,9 50,8 0,0 0,2 0 Киргизия 71,8 22,8 4,1 1,2 0,1 0,02 Молдова 10,0 8,2 81,8 0,0 0,0 0 Таджикистан 86,0 12,9 0,2 0,8 0,0 0,01 Туркмения 41,3 53,9 3,5 0,9 0,4 0,03 Узбекистан 56,4 37,6 5,0 0,9 0,1 0,11 Украина 2,1 0,4 97,0 0,0 0,5 0 Лица без гражданства 25,0 10,4 5,1 0,2 59,3 0 Другие страны 67,3 18,9 8,0 4,2 1,2 0,29

Рассчитано по данным МВД России.

VI. Заключительные замечания

38. Предложенная классификация позволяет получить более четкое представление о применяемых в России подходах к организации формальных каналов переезда в Россию и приема в российское гражданство. Предложенный алгоритм упрощает описание не только международной миграции, но и характера миграционной политики. Распределив детализированные основания приобретения мигрантами статуса резидента и гражданства по укрупненным классам, мы можем объективно наблюдать преобладание оснований, связанных с воссоединением семей. Это очень важно, поскольку в Концепции государственной миграционной политики России этот вид миграции вообще не отражен29. Несмотря на приоритеты, на которых сделан акцент в Концепции государственной миграционной политики - привлечение соотечественников и других лиц с этнокультурными связями и близостью к России, квалифицированных трудовых мигрантов и пр., на практике структура потоков показывает в

29 https://base.garant.ru/72092260/?ysclid=lv0mdzvcoj588550241#friends

Working paper x

14

большей степени гуманитарный характер миграции. Явное доминирование класса миграции по линии воссоединения семей, многочисленность категорий в этом классе, упрощения допуска мигрантов к воссоединению семей в последние годы показывают уважение России к правам человека на семейную жизнь (в соответствии с Всеобщей декларацией прав человека) и на воссоединение семей.

39. Отсутствие в статистике видов на жительство выраженного по наполненности канала трудовой

миграции говорит о том, что политика в отношении этого вида миграции по-прежнему направлена на сохранение у этих мигрантов временного статуса. Отсутствие полноценных иммиграционных программ для квалифицированных работников с правом привезти семьи и получить какие-либо еще преференции, не позволяет этому потоку приобрести большие объемы. Де-факто, происходит перераспределение квалифицированных работников по иным каналам и классам, в том числе путем их участия в Госпрограмме. Вынужденные мигранты, если потоки когда-либо вновь увеличатся, чего мы бы не хотели, должны получать больше прав в их собственном статусе, а не перенаправляться в другие классы, как это имеет место в рамках современной практики. Такие условия можно создать соответствующими поправками к закону. Пока незаметный класс студенческой миграции в будущем может существенно увеличиться. Введенная в 2023 г. норма должна получить признание и стать востребованной среди иностранных студентов, т.к. РВП дает намного больше прав и комфорта при проживании в России. Студентам из визовых стран не нужно продлевать неоднократно визу, а студенты из безвизовых стран также получают возможность одним документом обеспечить себе более спокойный – в смысле подтверждения права находиться в России - режим пребывания.

40. В то же время, масштабы квалифицированной трудовой миграции или миграции с целью обучения, а

также вынужденной миграции и их места в миграционном ландшафте России нельзя оценивать на основании статистики видов на жительство или приема в гражданство. Иностранные студенты имеют право жить в России, не обращаясь за РВП. Высококвалифицированные и просто квалифицированные специалисты также могут жить в России на основании визы, без вида на жительство. Лица, имеющие патент (среди которых также могут быть квалифицированные работники), могут продлевать его действие, тем самым получая право на долгосрочное пребывание в России. Поэтому оценка объемов трудовой или образовательной миграции должна основываться на соответствующих специальных данных, соответственно, данных МВД России о документировании трудовых мигрантов, о миграционном учете с целью «работа» и данных Министерства науки и образования об иностранных студентах.

41. То же самое касается вынужденной миграции, которая учитывается МВД, но статистика охватывает

лишь иностранцев. Проблемой измерения вынужденной миграции в 2022-2023 гг. остается наличие у большинства таких мигрантов российского гражданства, которое активно предоставлялось жителям Донбасса, а потом и других областей Украины в последние годы. В будущем, когда пройдет отладку новое программное обеспечение МВД России, вероятно станет возможным выделять мигрантов с разными классами допуска к долгосрочному пребыванию и проживанию в России, мигрантов с разным статусом и длительностью пребывания в России. Также станет доступна информация об их социально- демографических характеристиках.

42. В настоящее время в России осуществляется реформа миграционного законодательства. В 2023 г. был

принят и вступил в силу новый закон «О гражданстве Российской Федерации». Как мы отмечали выше, он преимущественно отражает те же подходы к предоставлению гражданства и выделяет категории, которые присутствовали в старом законе, что должно обеспечить сопоставимость данных в ретроспективе и в будущем. Готовится к принятию важный законопроект30 в сфере регулирования вопросов въезда, пребывания и проживания иностранце в России, который заменит несколько ныне

30 Проект федерального закона «Об условиях въезда (выезда) и пребывания (проживания) в Российской Федерации иностранных граждан и лиц без гражданства» https://base.garant.ru/56883645/?ysclid=luz72qscr2190085111

Working paper

15

действующих законов. Система РВП будет отменена, хотя перечень категорий иностранцев, получающих вид на жительство, изменится не столь значительно. Вероятно, будет изменен формат статистической отчетности 2-РД в соответствии с положениями будущего закона. В любом случае, предложенный нами подход к классификации каналов допуска мигрантов к проживанию и получению гражданства в России можно будет применить и в новых условиях.

  • I. Введение
  • II. Описание данных и их ограничения
  • III. Подходы к выделению классов оснований для получения видов на жительство и гражданства РФ
  • IV. Основные результаты
    • A. Разрешения на временное проживание
    • B. Виды на жительство
    • C. Прием в гражданство
  • V. Потенциал административных данных МВД России
  • VI. Заключительные замечания

Russian Coal in the Era of Climate Change, by Mr. Maxim Titov, ENERPO Research Center

Languages and translations
English

Russian Coal in the Era of Climate Change Maxim Titov

Research Fellow ENERPO Research Center

Russian Coal in the Era of Climate Change

The book has been released in November 2023 by Palgrave Macmillan publisher, in English.

The Russian coal sector today faces challenges in the form of global decarbonization, sanctions, and difficulties in retaining

market access and competitiveness. This new market environment warrants an examination of what lessons can be

drawn from the past tribulations of one of Russia’s most iconic

and strategically important industries. This book analyzes the Russian coal industry developments in historical and political

contexts, including those of energy transition. The book addresses the question of the coal industry as a sacrifice for

the sake of Russia’s rapprochement with the West. The

contribution of this book, therefore, is filling the gap between two large research inquiries – climate change mitigation and

energy transition on the one hand, and the existing large coal industry of a given country on the other.

Link: https://link.springer.com/book/10.1007/978-981-99-5370-7

Authors

Nikita Lomagin Michael Oshchepkov

Maxim Titov Irina Mironova

Key questions and hypothesis

How have the reforms affected development of the coal industry?

Hypothesis: the reforms of the World Bank gave a powerful impetus to the further development of the industry

Why is the Russian coal industry still developing?

Hypothesis: coal is important for the socio-economic stability of the сoal-mining regions, it provides for energy security of many regions in the East, and it is profitable. Also, the state provides support both for coal towns and exporters by managing coal transportation by rail.

How the global climate agenda shapes further development of the industry?

Hypothesis: the climate agenda does not yet affect the development of the Russian coal industry, since there is a high demand for cheap energy sources in the Asia-Pacific region

Domestic coal consumption began to decline gradually, from 54% in 1960 to 25.2% by 1980.

In 1983, the "Energy Program of the USSR for the Long Term" was adopted, and an intensive growth of coal production began. High production indicators were achieved due to open-pit coal mining.

In 1989, a crisis began in the coal industry. This provoked miners' strikes.

A general decline in industrial production. The Russian economy has begun to restructure itself into a market economy.

The World Bank has allocated more than $1.1 billion to repair unprofitable mines.

The final stage of privatization of the industry. The Russian coal industry has come out of the crisis.

Economic Crisis

Economic Crisis

Paris Agreement

Key milestones of the coal industry development

Percentage of coal in the energy balance of the USSR

• From 1928 to 1955, coal's share of the domestic market increased from 29% to 59%, while the share of oil and gas was only 15% to 28% of the domestic market.

• Then, in the early 1960s, significant reserves of oil and natural gas were explored in Western Siberia.

The Oil Age Begins

Coal production in the USSR

Labor resources of the coal industry in Russia and the USSR, thousand people

Regions Coal production, mmt Specific index, % RSFSR 425.4 55.1 Ukrainian SSR 191.7 24.9 Kazakh SSR 143.1 18.5 Uzbek SSR 5.5 0.7 Kyrgyz SSR 4.0 0.5 Georgian SSR 1.4 0.2 Tajik SSR 0.7 0.1 Total 771.8 100.0

Table 1. Distribution of coal production in the USSR by 1988

• By 1988, the Soviet Union had reached its peak of 771.8 million tons of coal.

• At the same time, the core of coal production was Russia (RSFSR) with a share of 55.1%, and the main coal basin was Kuzbass (Western Siberia) with the production of 159.2 million tons of coal.

• By 1988, the overall level of artificial lift had increased from 41.6 per cent to 84.1 per cent in 1965.

• (!) At the same time, there was no reduction in the number of people employed in the industry

The peak of the coal industry in the USSR

• In 1990, underground coal mines produced about 380 million tons of coal (54% of the total), with more than half of them commissioned before 1960 and never upgraded since.

• About 210 mines throughout the USSR were in operation for more than 35 years, 64 of which were commissioned before World War II and 73 were commissioned within the next five years of its beginning;

• The foreign economic situation for the Soviet Union also did not contribute to the development of the country's coal industry (an increase in the balance of payments deficit, external debt, a shortage of food and consumer goods, the beginning of the collapse of the USSR);

• Miners' strike movement

Return on Capital of Soviet Coal Industry Enterprises

Coal Legacy: Old Infrastructure and Lack of Modernization

In the new economic realities, Russia was no longer able to support the functioning of inefficient enterprises and, starting from 1995, the volume of state support for the coal industry began to decline ($1.63 billion against $3.25 billion in 1994).

Financing of the Russian coal industry in 1993-2002 (million $US)

Restructuring of the coal industry

Changes in Labor Productivity in the Russian Coal Industry • Budget cuts from 1994 to 2001 and the involvement of the World Bank ($1,350 billion) led to the elimination of inefficient mines and a reduction in the number of mines from 229 to 103;

• Despite the economic crisis of 1998, the coal industry became profitable as early as 1999;

• By 2002-2003, the main stage of the restructuring of the coal industry was almost complete. The market was formed by high-performing companies;

• An increase in coal consumption in the east of the country and demand from partners in the Asia-Pacific region led to the development of railway infrastructure.

Results of the reform: growth in the productivity

11

Key production indicators of the Russian coal industry in 2022

2021 2022 Change

Total Mining 432 Mt 437 Mt +1,2% (+5,0 Mt)

Open-pit 65,6% 76,8% +11,5 Mt

Mines 34,3% 23,2% -10,3 Mt

Coking coal 23,1% 25,9% +12,8 Mt

Thermal coal 76,9% 74,1% -11,6 Mt

Source: CDU TEK, Ministry of Energy

• Coal production increased in 2022 • Open-pit coal mining is more cost-effective and

continues to grow • Coking coal is more expensive and more in demand

in other countries

Geopolitical turmoil and sanctions have not led to a significant change in coal production in Russia

Major Coal Exporters: Competitors' Assessment

12 Major exportesr in 2002, 2014 and 2021, Exajoules (BP 2022)

Major Coal Importers: Actual and Potential Target Markets

13 Major importers in 2002, 2010 and 2021, Exajoules (BP 2022)

Major coal assets in Russia

14

Today, the Russian coal sector is facing new challenges: global decarbonization, increased Western sanctions, and internal and external competition.

The new state of the market environment in which the Russian coal industry finds itself requires the study of what lessons can be learned from the past of one of the most significant and strategically important industries in Russia.

ENERPO Research Center

EVAL_E264ToR_Oct2023

TERMS OF REFERENCE, Project Evaluation, October 2023
E264: Enhancing coherence and integration of economic and social statistics in support of the implementation of 2008 System of National Accounts

Languages and translations
English

1

TERMS OF REFERENCE

E264: Enhancing coherence and integration of economic and social statistics in support of the implementation of 2008 System of National Accounts

I. Purpose The purpose of this evaluation is to assess the extent to which the objectives of the UNECE project E264 “Enhancing coherence and integration of economic and social statistics in support of the implementation of 2008 System of National Accounts” were achieved.

The evaluation will assess the relevance, effectiveness, efficiency and sustainability of the project.

The evaluation will also assess any impacts the project may have had on progressing human rights, gender equality, disability inclusion, climate change and disaster risk reduction in the context of this engagement. The evaluation will finally look at the activities repurposed to address the impact of the COVID-19 crisis, and assess, to the extent possible, UNECE’s COVID-19 early response through this project.

II. Background The project aimed at increasing the capacity of countries in Eastern Europe, Caucasus and Central Asia in compiling national accounts and supporting statistics according to the recommendations of international statistical standards. The activities of the project included the following four modules:

1) Methods and compilation techniques for implementation of the 2008 SNA and supporting economic statistics.

2) Price statistics - development of price indices and deflators for national accounts aggregates. 3) Statistical business registers - improving coverage and accuracy of national statistical business

registers. 4) Expanding the use of household surveys for the estimates of the informal sector and

remittances.

The activities were carried out in cooperation with other international and regional partners, including CIS-STAT, EFTA, Eurostat, ILO, IMF, OECD and World Bank.

III. Evaluation objectives, scope and questions The evaluation will be guided by the objectives, indicators of achievement and means of verification established in the logical framework of the project document. The evaluation will be conducted in Q3 of 2023. It will cover the full implementation of the project, from June 2017 to June 2023 in Eastern Europe, Caucasus and Central Asia (Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Russian Federation, Tajikistan, Turkmenistan, Ukraine and Uzbekistan)

The final evaluation of the project has the following specific objectives:

• Determine as systematically and objectively as possible the relevance, effectiveness, efficiency and sustainability of the project results in light of its goals and objectives.

• Assess how the project activities contributed to gender equality and women’ s empowerment, as well as the realization of human rights, with an emphasis on ‘leaving no one behind’ and, if needed, it will make recommendations on how these considerations can be better addressed in future activities of the subprogramme.

• Identify good practices and lessons learned from the project and formulate action-oriented, forward-looking recommendations addressed to the subprogramme for improving future interventions.

2

The evaluation criteria are relevance, effectiveness, efficiency and sustainability.

Relevance 1. To what extent was the project design appropriate for meeting the needs of beneficiary

countries? 2. To what extent did the project respond to the priorities and needs of national statistical offices

in beneficiary countries? How relevant were the project activities to the countries’ needs and priorities?

3. To what extent was the project aligned with the SDGs? 4. What takeaways are there for ensuring relevance of future projects of UNEE Statistical

Division? 5. To what extent were gender, human rights and disability perspectives integrated into the design

and implementation of the project? What results can be identified from these actions? How can gender and human rights perspectives be better included in future projects design and implementation?

Effectiveness

6. To what extent were the project objectives and expected results achieved? 7. To what extent did the project improve the competencies of national statistical offices in the

beneficiary countries to produce and use macroeconomic statistics in their countries? 8. To what extent are the project activities coherent and harmonized with those of other partners

operating within the same context, particularly those of other UN system entities? 9. What were the challenges/obstacles (including COVID-19 and sub-regional instability) to

achieving the expected results? How successfully did the project overcome these? 10. What (if anything) has prevented the project from achieving the desired results?

Efficiency

11. Were the resources adequate for achieving the results? 12. Were the results achieved on time and were all activities organized efficiently? 13. To what extent were the resources used economically and how could the use of resources be

improved? Sustainability

14. What measures were adopted to ensure that project outcomes would continue after the project ended and to what extent have these measures addressed the existing risks for sustainability?

15. To what extent do the partners and beneficiaries ‘own’ the outcomes of the work? How is the stakeholders’ engagement likely to continue, be scaled up, replicated, or institutionalized?

IV. Evaluation approach and methodology The evaluation will be conducted in accordance with: the ECE Evaluation Policy1; the Administrative instruction guiding Evaluation in the UN Secretariat 2; and the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation3. Human rights and gender equality considerations will be integrated at all stages of the evaluation4: (i) in the evaluation scope and questions; (ii) in the methods, tools and data analysis techniques; (iii) in the findings, conclusions and recommendations of the final report. The evaluator will explicitly explain how human rights, gender, disability, SDGs, and climate change considerations will be taken into account during the evaluation.

1 UNECE Evaluation policy 2 ST/AI/2021/3 3 UNEG 2016 Norms and Standards for Evaluation 4 In line with UNEG Guidance contained in Integrating Human Rights and Gender Equality in Evaluations

3

The evaluator is required to use a mixed-method approach, including qualitative as well as quantitative data gathering and analysis as the basis for a triangulation exercise of all available data to draw conclusions and findings.

The evaluation should be conducted based on the following mixed methods to triangulate information:

1. A desk review of all relevant documents, including the project document and information on project activities (monitoring data); materials developed in support of the activities (agendas, plans, participant lists, background documents, donor reports and publications); Proposed programme budgets covering the evaluation period; project reports to the donor.

2. Online survey of key stakeholders and beneficiaries: the survey will be developed by the consultant on her/his preferred platform.

3. Interviews (in-person and/or by telephone/video): the evaluator shall interview a wide range of diverse stakeholders and beneficiaries.

4. Remote observation of virtual workshops and meetings, including recordings of meetings.

The evaluator will further elaborate on the evaluation methodology in the Inception Report that will among others include the survey questions and whether any of the 12 countries will be selected for an in-depth assessment. The evaluation report will be written in English, will consist of approximately 30 pages and will include an executive summary (max. 2 pages) describing the evaluation methodology, key findings, conclusions and recommendations. The evaluator will also produce an Evaluation Brief summarizing key evaluation findings, lessons learned and recommendations, including through images and infographics.

V. Evaluation schedule5 May 2023 ToR finalized June 2023 Evaluator selected June 2023 Contract signed. Evaluator starts the desk review July 2023 Evaluator submits inception report including survey design August 2023 Launch of data gathering, including survey and interviews September 2023 Evaluator submits draft evaluation report and evaluation brief October 2023 Evaluator submits final evaluation report and evaluation brief

VI. Resources and Management of the evaluation An independent consultant will be engaged to conduct the evaluation under the management of the PMU. Payment will be made upon satisfactory delivery of work. The Programme Management Unit (PMU) will manage the evaluation and will be involved in the following steps: Selection of the evaluator; Preparation and clearance of the Terms of Reference; Provision of guidance to the Project Manager and evaluator as needed on the evaluation design and methodology; Clearance of the final report after quality assurance of the draft report. The Project Manager, in consultation with the Division Director, will be involved in the following steps: Provide all documentation needed for desk review, contact details, support and guidance to the evaluation consultant as needed throughout the timeline of the evaluation; Advise the evaluator on the recipients for the questionnaire and for follow-up interviews; Process and manage the consultancy contract of the evaluator, along the key milestones agreed with PMU.

5 Final timetable to be agreed following engagement of the evaluator

4

VII. Intended use / Next steps The results of the evaluation will be used in the planning and implementation of future activities of the UNECE Economic Cooperation and Integration Subprogramme. Findings of this evaluation will be used when possible to:

• improve direct project’s follow up actions, implementation of products by project beneficiaries and dissemination of the knowledge created through the project;

• assess the gaps and further needs of countries in the area of this project; • formulate tailored capacity building projects to strengthen the national capacity in enhancing

innovation. The results of the evaluation will be reported to the Conference of European Statisticians Following the issuance of the final report, the Project Manager will develop a Management Response for addressing the recommendations made by the evaluator. The final evaluation report, the management response and the progress on implementation of recommendations will be publicly available on the UNECE website.

VIII. Criteria for evaluators The evaluator should have:

1. An advanced university degree or equivalent background in relevant disciplines. 2. Knowledge of and experience in working on data and statistics; knowledge of and experience in

economic statistics desirable. 3. Relevant professional experience in design and management of evaluation processes with multiple

stakeholders, survey design and implementation, project planning, monitoring and management, gender mainstreaming and human-rights due diligence.

4. Demonstrated methodological knowledge of evaluations, including quantitative and qualitative data collection and analysis for end-of-cycle project evaluations, including demonstrated experience in conducting questionnaires and interviews.

5. Fluency in written and spoken English.

Evaluators should declare any conflict of interest to UNECE before embarking on an evaluation project, and at any point where such conflict occurs.

  • I. Purpose
  • II. Background
  • III. Evaluation objectives, scope and questions
    • Relevance
    • Effectiveness
    • Efficiency
    • Sustainability
  • IV. Evaluation approach and methodology
  • V. Evaluation schedule4F
  • VI. Resources and Management of the evaluation
  • VII. Intended use / Next steps
  • VIII. Criteria for evaluators

EVAL-E264_EvalBrief_Oct24

Evaluation Brief of the project E 264

Enhancing coherence and integration of economic and social statistics in support of the implementation of 2008 System of National Accounts (SNA)
UNECE PROJECT E264 – 2017-2023

Languages and translations
English

EVALUATION BRIEF

Enhancing coherence and integration of

economic and social statistics in support of

the implementation of 2008 System of

National Accounts (SNA)

UNECE PROJECT E264 – 2017-2023

BACKGROUND

Since 2010, the UNECE has been actively working to support the implementation of 2008 System of National Accounts and supporting statistics in countries of Eastern Europe, Caucasus, and Central Asia (EECCA). This project was intended to enable the UNECE in carrying out its mandate to support its implementation in the region and build on the activities and progress achieved since 2010. The project was part of the ECASTAT regional trust fund established by the World Bank for assisting countries to improve their statistical systems. The work on the project was guided by the Steering Group on National Accounts. METHODOLOGY The evaluation was conducted using a mixed- method approach to assess whether the planned objectives were met and to assess their relevance, effectiveness, efficiency, and sustainability. Cross-cutting issues such as human rights and gender equality were considered at all stages of the evaluation. The review consisted of a desk review of over 75 available project documents, analysis of the results from the two on-line surveys designed for the purpose of this evaluation and in-depth interviews with key stakeholders/partners to gather more information and assessment of results. Key findings were triangulated and presented in line with the questions contained in the Evaluation Matrix. FINDINGS

The evaluation determined that the project was fully relevant to the objective of the UNECE to increase the capacity of target countries in compiling national accounts and supporting statistics according to the recommendations of the 2008 SNA and other relevant international standards by focusing on the outstanding

priorities identified by the countries. The needs of the countries formed the basis of regional workshops and special sessions designed to address their specific issues or challenges.

The evaluation determined that the project was effective by deploying both traditional and innovative methods to ensure that the objectives and results were achieved. The project consisted of four-interrelated modules with the goal to enhance coherence and integration of economic and social statistics in support of the implementation of 2008 SNA. All activities were completed, and additional ones added with the project extension.

The most significant challenge the project faced was the onset of the COVID-19 pandemic in early 2020. All planned results for 2020 were delayed and had to be realigned. The UNECE quickly adapted to the new global reality and implemented the use of online/virtual meetings to continue the momentum of the project during unprecedented times.

The evaluation determined that the project efficiently achieved its objectives within the allocated budget and adjusted timeframe. The savings associated with the new way of doing business (described above) and the UNECE’s ability to draw on established partnerships to obtain additional resources through in-kind or financial contributions, extended the project to the benefit of the targeted countries. A no-cost extension (granted from 2020-2023) expanded the scope of the project to address outstanding issues, identified by the EECCA countries, resulting from the pandemic.

The benefits of the project will continue after its completion although there will be some challenges. By nature of their mandates, the UNECE Statistical Division and the Steering Group on National Accounts are committed to maintaining and progressing the work undertaken during the project. For the EECCA countries, lack of funding, insufficient experienced staff, staff turnover and limited technical capabilities are likely to be a significant challenge in sustaining the work.

3

CONCLUSIONS

The evaluation has determined that the project met is overall aim and achieved its stated objectives. However, the following limitations were identified and should be considered in future action plans. All UNECE work is designed to be as inclusive as possible, however the human rights dimension is a difficult aspect to cover. The human rights dimension was not incorporated in the project proposal and the project outcomes make no explicit reference to human rights. It was evident through the evaluation that women and youth were involved in the project activities and in the evaluation process. However, no gender consideration was recorded in the project’s proposal and no indicator established. It was difficult to determine whether the knowledge and expertise of the staff in national statistical offices improved and/or their capacity to implement standards and recommendations in practice increased. To properly assess these elements could only be achieved through an in-depth country peer review or an update to the Global Assessments. This was not done as part of this evaluation.

RECOMMENDATIONS

In the spirit of continuous learning and based on the findings and good practices identified throughout evaluation, a series of recommendations are intended to help improve future UNECE projects.

1. The global pandemic forced the UNECE to rethink the way it conducted business to ensure continuity of the project. These innovative solutions introduced good practices that should be continued. The UNECE is encouraged to use a hybrid model when organizing its meetings, workshops, and regional sessions in the future. The

introduction of short webinars and the video capture of in-person sessions provided additional learning opportunities. A combination of face-to-face activities and virtual offerings reaches a broader group of people. Using appropriate and tested technologies that accommodates everyone’s participation is encouraged.

2. To properly assess the degree to which the knowledge and expertise of the staff in the national statistical offices of EECCA countries improved and/or their capacity to implement standards and recommendations increased, an in-depth country peer review or update to the Global Assessments should be conducted.

3. Continued sharing of experience and lessons

learned is critical to the ongoing improvement of the statistical system in the beneficiary countries. EECCA participation in the Expert Group meetings is encouraged and where possible regional workshops or special sessions would continue to offer continuous growth and learning.

4. To ensure sustainability, future activities

should continue to be linked to the work programme of the UNECE Statistical Division. Although the budget of the UNECE Statistical Division is limited, it is encouraged to use its exceptional collaboration and outreach skills to secure additional funding from key project stakeholders, be it in-kind or financial resources to expand the scope of its work.

5. For gender and human rights perspectives to

be better included in future projects, there should be an explicit objective built into the project design that can be monitored and evaluated during and at the end of the project. Expertise in gender quality, disability inclusion, environment and or human rights should be sought in the development of these objectives and with the evaluation process.

EVAL_E264-EvaluationReport

UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE - PROJECT E264
ENHANCING COHERENCE AND INTEGRATION OF ECONOMIC AND SOCIAL STATISTICS IN SUPPORT OF
THE IMPLEMENTATION OF 2008 SYSTEM OF NATIONAL ACCOUNTS (SNA)
2017-2020 (EXTENSION FOR 2021-2023)
EVALUATION REPORT
Period of evaluation: July – October 2023
Evaluator: Constance Graziadei
Evaluation commissioned by United Nations Economic Commission for Europe

Languages and translations
English

UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE - PROJECT E264

ENHANCING COHERENCE AND INTEGRATION OF ECONOMIC AND SOCIAL STATISTICS IN SUPPORT OF THE IMPLEMENTATION OF 2008 SYSTEM OF NATIONAL ACCOUNTS (SNA)

2017-2020 (EXTENSION FOR 2021-2023)

EVALUATION REPORT

Period of evaluation: July – October 2023

Evaluator: Constance Graziadei

Evaluation commissioned by United Nations Economic Commission for Europe

3

EVALUATION REPORT

The Evaluation report compiled by Constance Graziadei evaluates the UNECE project “Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 System of National Accounts” implemented from 2017-2023.

The views and interpretations are those of the author and do not necessarily represent those of the United Nations Economic Commission for Europe (UNECE).

ACKNOWLEDGEMENTS

Ms. Graziadei would like to thank all who contributed to successful completion of this evaluation: to the UNECE Statistical Division who provided access to all the project documentation and support to the process, to all who responded to the two online surveys and to those who generously gave their time to be interviewed. Your contributions are truly appreciated.

4

TABLE OF CONTENTS ABBREVIATIONS AND ACRONYMS ........................................................................................................................ 5

EXECUTIVE SUMMARY .......................................................................................................................................... 6

1.0 INTRODUCTION ......................................................................................................................................... 9

1.1 PURPOSE OF THE EVALUATION .................................................................................................................. 10 1.2 OBJECTIVES OF THE EVALUATION .............................................................................................................. 10 1.3 SCOPE OF THE EVALUATION ....................................................................................................................... 10 1.4 EVALUATION METHODOLOGY ................................................................................................................... 11

2.0 FINDINGS ................................................................................................................................................ 12

2.1 RELEVANCE ................................................................................................................................................. 12 Figure 1: Evaluation of the UNECE Project on Economic and Social Statistics ................................................. 13

2.2 EFFECTIVENESS ........................................................................................................................................... 15 2.3 EFFICIENCY .................................................................................................................................................. 17

Table 1: Budget Allocation ................................................................................................................................. 17 Table 2: Initial Project Allocation of Funds and expenditures per module (2017-2020) .................................. 18 Table 3: Budget Allocation for 2021-2023 ......................................................................................................... 18

2.4 SUSTAINABILITY ......................................................................................................................................... 19 Figure 2: Evaluation Survey – Evaluation of the UNECE Project on Economic and Social Statistics ................ 20

2.5 GENDER AND HUMAN RIGHTS CONSIDERATIONS ..................................................................................... 21

3.0 CONCLUSIONS AND RECOMMENDATIONS .............................................................................................. 22

3.1 CONCLUSIONS ............................................................................................................................................ 22 3.2 RECOMMENDATIONS ................................................................................................................................. 23

4.0 ANNEXES ................................................................................................................................................. 24

4.1 TERMS OF REFERENCE FOR THE EVALUATION ........................................................................................... 24 4.2 ONLINE SURVEY QUESTIONNAIRE – BENEFICIARY COUNTRIES ................................................................. 28 4.3 ONLINE SURVEY RESULTS – BENEFICIARY COUNTRIES .............................................................................. 30 4.4 ONLINE SURVEY QUESTIONNAIRE – STAKEHOLDERS/PARTNERS ............................................................. 36 4.5 INTERVIEW GUIDE – KEY STAKEHOLDERS .................................................................................................. 38 4.6 LIST OF PROJECT ACTIVITIES BY MODULE .................................................................................................. 39 4.7 STAKEHOLDERS INVITED FOR IN-DEPTH INTERVIEW ................................................................................. 41 4.8 LIST OF DOCUMENTS REVIEWED ................................................................................................................ 42

5

ABBREVIATIONS AND ACRONYMS

CES Conference of European Statisticians

CIS Commonwealth of Independent States

ECASTAT European and Central Asian Statistics

EECCA Eastern Europe, Caucasus, and Central Asia

EFTA European Free Trade Association

ILO International Labour Association

IMF International Monetary Fund

NSO National Statistical Office

OECD Organization for Economic Cooperation and Development

ROSSTAT Russian Federal State Statistical System

SBR Statistical Business Registers

SNA System of National Accounts

SEE South East Europe

SDGs Sustainable Development Goals

UNECE United Nations Economic Commission for Europe

UNESCAP United Nations Economic and Social Commission for Asia and the Pacific

UNSC United Nations Statistical Commission

UNSD United Nations Statistical Division

WB World Bank

6

EXECUTIVE SUMMARY The purpose of this evaluation is to assess the extent to which the objectives of the UNECE Project “Enhancing coherence and integration of economic and social statistics in support of the implementation of 2008 SNA” were achieved. This evaluation assessed the relevance, effectiveness, efficiency, and sustainability of the project considering its stated goals and objectives.

The evaluation also attempted to assess how the project’s activities contributed to gender equality and women’s empowerment as well as the realization of human rights. The evaluation considered good practices and lessons learned from the project to formulate recommendations aimed at improving future projects.

The evaluation was carried out through a desk review of all relevant project documentation, analysis of the results of two on-line surveys designed for the purpose of this evaluation and in-depth interviews with key project stakeholders/partners.

The evaluation determined that the project was fully relevant to the objective of the UNECE to increase the capacity of target countries 1 in compiling national accounts and supporting statistics according to the recommendations of the 2008 SNA and other relevant international standards by focusing on the outstanding priority areas identified by the countries. These priority areas were derived from the Global Assessments of countries in the EECCA region, which was jointly carried out by the UNECE, Eurostat and EFTA as well as the regional and national plans for implementation of the 2008 SNA. The project work was guided by the UNECE Steering Group on National Accounts. The project built on the activities and progress achieved in the work to support the implementation of the 2008 SNA and supporting economic statistics in the EECCA region, prior to its inception. Throughout the project, the needs and priorities of the EECCA countries formed the basis of regional workshops and special sessions designed to address their specific issues or challenges.

The most significant challenge to achieving the expected results on time was the onset of the global pandemic in early 2020. All planned activities for 2020 were delayed and had to realigned. The UNECE quickly adapted to the new global reality and implemented the use of online/virtual meetings in order to complete the planned activities. Face-to-face meetings were replaced with online sessions and adjustments were made to accommodate the new way of conducting the project (i.e. shorter sessions over many days). A dedicated wiki page was created to share the results of the activities.

The evaluation determined that the project was effective deploying both traditional and innovative methods to ensure the objectives and results were achieved. The project consisted of four inter-related modules with the goal to enhance coherence and integration of economic and social statistics in support of the implementation of the 2008 SNA. Each module had specific objectives with activities designed to achieve the objectives. All the activities were completed, and additional activities were added with the project extension.

The evaluation determined that the project efficiently achieved its objectives within the allocated budget and adjusted timeframe. As mentioned above, the UNECE introduced virtual meetings to continue the momentum of the project during unprecedented challenges. The savings associated with the new way of doing business and its ability to draw on established partnerships to obtain additional resources through in-kind or financial contributions extended the project to the benefit of the targeted countries. The no-cost extension expanded the scope of the project to address outstanding issues, identified by EECCA countries, resulting from the COVID-19 pandemic. The use of virtual meeting platforms granted a broader access allowing additional stakeholders and partners to participate. This included not only experts from national statistical offices but from their central banks and ministries of finance.

1 The target countries for the project were the 12 EECCA countries of Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Russian Federation, Tajikistan, Turkmenistan, Ukraine, and Uzbekistan

7

There were limitations to the evaluation. Although all UNECE work is designed to be as inclusive as possible, the human rights dimension is a difficult aspect to cover. The human rights dimension was not incorporated in the project proposal and the project outcomes make no explicit reference to the human rights. No gender consideration was recorded in the project’s proposal and no indicator established.

In addition, to properly assess whether the knowledge and expertise of the staff in the national statistical offices of EECCA countries improved and/or their capacity to implement standards and recommendations in practice could only be achieved through an in-depth country peer review or update to the Global Assessments. This was not done as part of this evaluation.

The benefits of the project will continue after its completion although there will be some challenges. The UNECE has a mandate to support the implementation of 2008 SNA and supporting economics statistics in the EECA region. The Steering Group on National Accounts, established by the Bureau of the Conference of European Statisticians guides the work of the UNECE Statistical Division in the area of national accounts and supporting economic statistics, including activities to support and monitor the implementation of the 2008 SNA in the EECCA and to advise on the provision of methodological recommendations, training and other capacity building activities on national accounts. Country representatives from the EECCA are members of the Steering Group. By nature of their mandates these partners/stakeholders are committed to maintaining and progressing the work undertaken during the project.

Continued sharing of experiences and lessons learned is critical to the ongoing improvement of the statistical system in the beneficiary countries. EECCA participation in the Expert Group meetings is encouraged and where possible special regional workshops would continue to offer continuous growth and learning opportunities. For the EECCA countries, lack of funding, insufficient experienced staff, staff turnover, and limited technical capabilities are likely to be significant challenges in sustaining the work.

In the spirit of continuous learning and based on the findings and good practices identified throughout the evaluation, a series of recommendations are intended to help improve future UNECE activities.

1. The global pandemic forced the UNECE to rethink the way it conducted business to ensure continuity of the project. These innovative solutions introduced good practices that should be continued. The UNECE is encouraged to use a hybrid model when organizing its workshops and regional sessions in the future. The introduction of short webinars and the video capture of in person sessions provided additional learning opportunities. A combination of face-to-face activities and virtual offerings reaches a broader group of people. Using appropriate and tested technologies that accommodates everyone’s participation is encouraged.

2. To properly assess the degree to which the knowledge and expertise of the staff in the national statistical offices of EECCA countries improved and/or their capacity to implement standards and recommendations increased, an in-depth country peer review or update to the Global Assessments should be conducted.

3. Continued sharing of experience and lessons learned is critical to the ongoing improvement of the

statistical system in the beneficiary countries. EECCA participation in the Expert Group meetings is encouraged and where possible regional workshops or special sessions would continue to offer continuous growth and learning.

4. To ensure sustainability, future activities should be linked to the work programme of the UNECE Statistical Division. Although the budget of the UNECE Statistical Division is limited, it is encouraged to use its exceptional collaboration and outreach skills to secure additional funding from key project stakeholders, be it in-kind or financial resources to expand the scope of its work.

8

5. For gender and human rights perspectives to be better included in future projects, there should be an explicit objective built into the project design that can be monitored and evaluated during and at the end of the project. Expertise in gender quality, disability inclusion, environment and or human rights should be sought in the development of these objectives and with the evaluation process.

9

1.0 INTRODUCTION2 In 2009 at the 40th session of the United Nations Statistical Commission (UNSC), it supported the Global Implementation Strategy for the 2008 System of National Accounts (SNA) and requested to pay particular attention to improving basic economic statistics and to the need for consistency and harmonization across sectoral statistics from other macroeconomic frameworks. The UNSC recognized the role of the UN regional commissions in facilitating coordination and regional cooperation towards the implementation of 2008 SNA. 3 Following the decision, the UNSC, the United Nations Statistics Division (UNSD) requested that regional commissions develop regional implementation plans and establish Steering Groups to guide and coordinate activities.

Since 2010, the UNECE has actively been working to support the implementation of 2008 SNA and supporting statistics in countries of Eastern Europe, Caucasus and Central Asia (EECCA)4. At the request of the UNSC, UNECE developed a regional plan for implementation of 2008 SNA in close cooperation with EECCA and South-East European (SEE) countries.

The regional implementation plan was developed in consultation with the Steering Group on National Accounts, established by the Bureau of the Conference of European Statisticians (CES). The Steering Group on National Accounts guides the work of the UNECE Statistical Division in the area of national accounts and supporting economics statistics, including activities to support and monitor the implementation of the 2008 SNA in the EECCA region and to advise on the provision of methodological recommendations, training and other capacity building activities on national accounts. The Steering Group is composed of representatives of all major regional partners such as CIS-Stat, Eurostat, International Monetary Fund (IMF), Organization for Economic Cooperation and Development (OECD), and UNSD, as well as country representatives from each UNECE sub-region, including from the EECCA.

The ECASTAT project was intended to build on the activities and progress achieved in the work to support the implementation of 2008 SNA and supporting economic statistics in the EECCA region. The project was intended to enable the UNECE in carrying out its mandate to support the implementation of 2008 SNA and supporting statistics in the region.

The project was part of the ECASTAT regional trust fund established by the World Bank for assisting countries to improve their statistical systems.

The overall objective of the Project was to increase the capacity of countries in Eastern Europe, Caucasus and Central Asia in compiling national accounts and supporting statistics according to the recommendations of international statistical standards.5 Another objective was to include overall improvement of the knowledge and

2 ECASTAT Project for 2017-2020. Implementation of 2008 SNA. Enhancing Coherence and Integration of Economic and Social Statistics Support of the Implementation of 2008 SNA. Project Proposal – Concept note 2016

3 Statistical Commission. Report on the fortieth session (24-27 February 2009). https://unstats.un.org/unsd/statcom/doc09/Report-English.pdf

4 The target countries for the project were the 12 EECCA countries of Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Russian Federation, Tajikistan, Turkmenistan, Ukraine, and Uzbekistan.

5Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 SNA https://www.statswiki.unece.org/display/EC/ECASTAT+Home

10

expertise of the staff in the NSOs of EECCA countries regarding relevant international standards and recommendations in practice.

The project was intended to build on existing UNECE programmes to support the implementation of 2008 SNA and supporting statistics in the region, in addition to the global assessments of national statistical systems. It intended to draw on established partnerships with national and international statistical agencies involved in statistical capacity building activities in the region, and in particular, the WB, UNESCAP, IMF, Eurostat, OECD, European Free Trade Association (EFTA), International Labour Organization (ILO), Russian Federal State Statistical System (ROSSTAT), CIS-STAT and other relevant multinational and bilateral partner organizations.

1.1 PURPOSE OF THE EVALUATION The purpose of this evaluation is to assess the extent to which the objectives of the UNECE Project “Enhancing coherence and integration of economic and social statistics in support of the implementation of 2008 SNA” were achieved.

The evaluation assessed the relevance, effectiveness, efficiency, and sustainability of the aforementioned project.

The evaluation also attempted to assess any impacts the project may have had on progressing human rights, gender equality, disability inclusion, climate change and disaster risk reduction in the context of this engagement. The evaluation considered the activities repurposed to address the impact of the COVID-19 crisis and assessed UNECE’s COVID-19 early response.

1.2 OBJECTIVES OF THE EVALUATION

The evaluation of the project has the following specific objectives:

1. Determine, as systematically and objectively as possible, the relevance, effectiveness, efficiency, and sustainability of the project results considering its goals and objectives.

2. Assess how the project activities contributed to gender equality and women’s empowerment, as well as the realization of human rights, with an emphasis on “leaving no one behind” and, if needed, make recommendations on how these considerations can be better addressed in future activities of the sub programme.

3. Identify good practices and lessons learned from the project and formulate action-oriented, forward- looking recommendations addressed to the sub programme for improving future interventions.

1.3 SCOPE OF THE EVALUATION The evaluation covered the full project implementation period 2017-2023 (initially 2017-2020), including the no- cost extension that was granted to the project to cover the period of 2021-2023.

The evaluation was guided by the objectives, indicators of achievement and means of verification established in the initial project proposal and the no-cost extension.

The scope was limited to the activities described in project documents and the evaluation criteria established by the UNECE in its Terms of Reference (see Annex 4.1) for the project evaluation.

The evaluation was conducted ensuring accessible data collection and inclusive consultation, adhering to ethical standards and triangulation of data for inclusion.

11

1.4 EVALUATION METHODOLOGY

The evaluation was conducted in accordance with: the ECE Evaluation Policy6; the Administrative instruction guiding Evaluation in the UN Secretariat7; and the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation8. Human rights and gender equality considerations were integrated at all stages of the evaluation.

The evaluation was guided by the evaluation matrix articulated in the Inception Report and assessed the degree of alignment between the project’s stated objective and outcomes with the results of the activities carried out by the UNECE in the implementation of the project.

The evaluation process included a desk review of all available project documentation, analysis of the results from the two on-line surveys designed for the purpose of this evaluation and interviews with key stakeholders and partners to gather more in-depth information and assessment of results.

1.4.1 Desk Review

The desk review included all project documentation provided by the UNECE Statistical Division, relevant websites and a review of the global assessments conducted jointly by the UNECE, Eurostat and EFTA. The documents reviewed included: the Project proposals - 2017-2020 and 2021-2023; the 11 semi-annual Project progress reports; reports from the meetings, workshops and webinars (including a review of the material presented); the meeting evaluations; the draft World Bank’s Implementation Completion and Results Report, and the 8 available unaudited interim financial reports (June to December 2017 to 30 June 2022). Unfortunately, the final interim financial report covering the final year of the project and the final program financial report were not completed in time for inclusion in the evaluation process. A complete list of documents reviewed are provided in Annex 4.8.

1.4.2 Surveys

Two surveys were conducted in English; one with all beneficiary countries and the second with a select group of stakeholders and partners. The survey questions are included in Annex 4.2 and 4.4 of this report. The surveys were conducted online and available to respondents from 4 - 29 September 2023.

The first survey was sent to beneficiaries of the target countries for the project. 36 individuals (20 women; 16 men) representing 12 countries were invited to participate in the survey. At the close of the collection period, 15 individuals from 8 countries responded (41.6% response rate). Of these responses, 12 respondents represented national statistical offices; 3 represented other domestic agencies; 11 were women and 4 were men. Full results are included in Annex 6.4 to this report.

The second survey was sent to 18 key stakeholders and partners (6 women; 12 men). Even with an extension of the collection period, only 6 responses were received (33% response rate) from 4 men and 2 women. The low response rate limited the type of analysis that could be conducted. The results were used to confirm/challenge the findings identified through the desk review and the in-depth interviews.

6 UNECE Evaluation Policy. https://unece.org/sites/default/files/2021-12/Item10_ECE_EX_2021_35_Evaluation%20Policy.pdf

7 ST/AI/2021/3

8 UNEG 2016 Norms and Standards for Evaluation. https://www.unevaluation.org/document/detail/1914

12

1.4.3 Interviews

A series of interviews with the stakeholders/partners were conducted to further gather both qualitative and quantitative information. The interviews were conducted by video conference during the month of September 2023. The interview guide is included in Annex 4.5 to this report.

The invitation was sent to 10 key stakeholders/partners (3 women; 7 men) who represented the various components of the project (SNA, CPI, SBR, Informal Sector and Remittances). Nine individuals (2 women; 7 men) graciously gave their time and feedback regarding the various activities and outcomes. Their insights and views were valuable in forming the findings, recommendations and the lessons learned of this evaluation. A list of the key stakeholders is included in Annex 4.7 of this report.

2.0 FINDINGS

2.1 RELEVANCE

2.1.1 To what extent was the project design appropriate for meeting the needs of beneficiary countries?

The project design was appropriate for meeting the needs of the beneficiary countries.

The overall aim of the project was to increase the capacity of EECCA countries in compiling national accounts and supporting statistics according to the recommendations of the 2008 SNA and other relevant international standards, with focus on outstanding priority areas identified by the countries. These priority areas were derived from the Global Assessments of countries in the EECCA region, which was jointly carried out by the UNECE, Eurostat and EFTA as well as the regional and national plans for implementation of the 2008 SNA. The project work was guided by the UNECE Steering Group on National Accounts. The project built on the activities and progress achieved in the work to support the implementation of the 2008 SNA and supporting economic statistics in the EECCA region, prior to its inception.

The project activities included participation of experts from beneficiary countries in UNECE Group meetings, regional workshops and special sessions to ensure capacity building and shared learning within the EECCA countries. The design also included simultaneous interpretation in meetings and workshops and translation of project documentation, training materials and handbooks to Russian to facilitate participation and ongoing work.

At the end of each workshop or meeting, participants were encouraged to share suggestions for improvement and ideas for future agenda items. The feedback and input were used to formulate the agenda and organization of subsequent meetings.

13

2.1.2 To what extent did the project respond to the priorities and needs of national statistical offices in beneficiary countries? How relevant were the project activities to the countries’ needs and priorities?

The project responded “to a high extent” to the priorities and needs of national statistical offices in the beneficiary countries.

FIGURE 1: EVALUATION OF THE UNECE PROJECT ON ECONOMIC AND SOCIAL STATISTICS

• Source: Project Evaluation Online survey – Question 5

The results of the online survey of beneficiary countries showed 73% of respondents indicated the project’s activities met their country’s needs to a high extent and 27% to a moderate extent. This finding was also supported in the review of the evaluations of the various activities that were conducted throughout the project. Overall the participants were very satisfied with the workshops and meetings as well as the quality of the discussions and presentations.

During the span of the project, the meetings of the Group of Experts generally included special sessions for the beneficiary countries with topics identified by the countries. For example, the 2017 and 2019 meetings of the Group of Experts on Business Registers included Special Sessions for the EECCA and SSEE countries to encourage exchanges and presentations aimed at improving the coverage and accuracy of Statistical Business Registers and short-term business statistics. In 2017 the UNECE Guidelines on SBR – activities in the CIS was presented and produced in both English and Russian. In 2019 the special session focussed on different CPI data collection methods; the treatment of seasonal items and missing observation; index calculation and weighting; owner- occupied housing and rentals in the CPI; and how to meet user needs in terms of documentation, dissemination and communication of the CPI. All of these topics were developed to achieve the project’s objectives and respond to the beneficiary countries’ needs.

Due to the global pandemic, several activities were delayed, postponed or moved to a virtual platform rather than face-to-face meetings. There were mixed reviews regarding the online forum. The online forum increased the number of experts who could participate in the meetings and workshops as travel was no longer required. However, there were some technical difficulties raised with some of the platforms that were used. Several people consulted during the evaluation process spoke about the value in having face-to-face meetings as many extended discussions and personal learning happen during the breaks and outside the formal meeting agenda.

2.1.3 To what extent was the project aligned with the SDGs?

Partnerships (SDG 17) between stakeholders including governments, international and regional organizations, businesses and academia and civil society are a driving force of UNECE’s work. This includes work to improve connectivity with the regions (SDGs 7, 8, 9, 11,13), and contributing to creating more dynamic and resilient economies (SDGs 7, 8, 9,11, 13). The UNECE works for member countries by supporting evidence-based decision making through reliable statistics and improved monitoring, analysis and policy advice; fostering cooperation

To what extent did the event respond to your country's statistical needs and priorities?

To a high extent To a moderate extent To a low extent Not at all

14

among all stakeholders at the country and regional level; building countries’ capacities to implement its norms, conventions and standards.9

Basic sound economics, including key figures of the national accounts are fundamental in relation to the SDGs. Several proposed SDG indicators on income, consumption, and production depend on the availability of comprehensive and high-quality national statistics that are internationally comparable.

The project was designed to help EECCA countries improve the coherence, timeliness and internationally comparability of socio-economic statistics that are crucial for the monitoring of a country’s social and economic progress. The project design encouraged the collaboration of experts from the statistical offices as well as central agency representatives from national banks and finance ministries which improved the coherence of the statistical outputs.

2.1.4 What takeaways are there for ensuring relevance of future projects of UNECE Statistical Division?

The project activities were linked to the work programme of the UNECE Statistical Division to ensure continuity and coordination with international work on national accounts and supporting statistics. This is a critical element for continued success of future projects.

The global pandemic forced the UNECE to examine and implement innovative ways to continue with the activities essential in achieving a successful outcome to the project.

UNECE’s ability to draw on established partnerships and to obtain additional resources through in-kind or financial contributions extended the project to the benefit of the targeted countries and expanded the scope of the project to address outstanding issues, identified by EECCA countries, resulting from the COVID-19 pandemic.

2.1.5 To what extent were gender, human rights and disability perspectives integrated into the design and implementation of the project? What results can be identified from these actions? How can gender and human rights perspectives be better included in future projects design and implementation?

There is no evidence from document review or information gathered throughout this review that gender, human rights and disability perspectives were integrated into the design and implementation of the project. However, it is evident from the evaluation that women and youth were involved in all aspects of the project and persons with disabilities were accommodated throughout the various meetings, workshops and special sessions.

To ensure gender and human rights perspectives be better included in future projects, there should be an explicit objective built into the project design that can be monitored and evaluated during and at the end of the project. Expertise in gender quality, disability inclusion, environment and or human rights should be sought in the development of these objectives and with the evaluation process.

9 SDG Priorities. Supporting countries to achieve the SDGs https://unece.org/sdg-priorities

15

2.2 EFFECTIVENESS

2.2.1 To what extent were the project objectives and results achieved?

The overall aim of the project and the expected outcomes were achieved.

The project consisted of four inter-related modules with the overall goal to enhance coherence and integration of economic and social statistics in support of the implementation of the 2008 SNA. The four modules were: Methods and compilation techniques for the implementation of the 2008 SNA and supporting economic statistics; Price Statistics – development of price indices and deflators for national accounts aggregates; Statistical business registers – improving coverage and accuracy of national statistical business registers; and Expanding the use of household surveys for the estimates of the informal sector and remittances.

Each module had specific objectives with activities designed to achieve the objectives (see Annex 6.6 for full details). All the activities within each module were completed and additional activities were added with the project extension. The project activities included participation of experts from EECCA countries in international statistical Expert Group meetings, regional workshops and special sessions designed specifically to address their needs. The project activities also included the development of statistical methods/tools for integrating migration in household surveys and development of training material. The desk review, supported by the online survey results and interviews with key stakeholders, provided evidence that the capacity of countries in compiling national accounts and supporting statistics increased.

However, there were limitations in the evaluation. For example to properly assess whether the knowledge and expertise of the staff in the national statistical offices of EECCA countries improved and/or their capacity to implement standards and recommendations in practice could only be achieved through an in-depth country peer review or update to the Global Assessments.

2.2.2 To what extent did the project improve the competencies of national statistical offices in the beneficiary countries to produce and use macroeconomic statistics in their countries?

The activities of the project were designed and implemented to improve the competencies of national statistical offices in the beneficiary countries.

An explicit example is the Special Session that was organized to support the EECCA and SEE countries on the use of Supply and Use tables (SUTs) as well as Input Output tables (IOTs). SUTs are at the core of National Accounts and represent a framework to ensure better quality, coherency, and consistency of macroeconomic aggregates. A survey among the target countries carried out before the session revealed that 15 countries were developing SUTs, some of them being at initial or experimental estimates. The main challenges identified by countries were lack of suitable data sources, human resources and time constrains. The focus of the special session was to help the EECCA & SEE countries address their challenges. During this session, different IT tools for the compilation of SUTs and IOTs were presented which including a short training session on two tools developed by the IMF to support balancing of SUTs and conversion into IOTs. The Special Session also helped EECCA and SEE countries share experience on data sources and compilation techniques and informed them on new developments in SUTs and IOTs. The evaluation survey at the end of the session showed that 95 percent of the participants considered the session to be very helpful or useful for their future work.

The second concrete example is the development of a harmonized survey module on migration and remittances for countries in EECCA. The survey module, developed within the context of the project, was designed to enable a regular sustainable collection of internationally comparable data on migration and remittances. Improvements in the coverage, exhaustiveness, and comparability of statistics on migration & remittances support SDG monitoring needs, in line with international standards. The module was tested in Tajikistan in late 2019 with a technical assistance mission supported by the project fund. Findings from the pilot test were shared with the EECCA countries in 2020 and the harmonized module was released in 2021 as a standard tool that has been produced in Russian and English.

16

2.2.3 To what extent are the project activities coherent and harmonized with those of other partners operating with the same context, particularly those of other UN system entities?

This UNECE-managed project is coherent and harmonized with those of partner organizations. Representatives from EFTA, Eurostat, ILO, OECD, UNSD and World Bank were all consulted during the evaluation and were all very complimentary of the UNECE’s collaboration and coordination expertise. Apart from the World Bank, who was the sponsor of the project, these partners are members of the UNECE Steering Group on National Accounts who guided and advised on the activities undertaken throughout the project.

During this evaluation the UNECE was recognized for its exceptional organization skills and the collaboration with other partners in the international statistical system to provide high quality learning activities. The UNECE was effective at bringing together expertise and ensuring no duplication with other international activities. The team members were also recognized for constantly seeking feedback and input and acting on the information gathered to improve subsequent events/activities. Consultation regarding agenda items and effective advanced planning ensured a positive experience for participants.

In the development of the various activities carried out throughout the project, the UNECE considered the CIS-Stat activities financed by the Trust Fund for Statistical Capacity Building and ensured there were no overlaps between the two projects.

2.2.4 What were the challenges/obstacles (Including COVID-19 and sub-regional instability) to achieving the expected results? How successfully did the project overcome these?

The most significant challenge to achieving the expected results on time was the onset of the global pandemic in early 2020. All planned activities for 2020 had to be adjusted. Face-to-face meetings were replaced with online sessions and adjustments were made to accommodate the new way of conducting the project (ie shorter sessions over many days). A dedicated wiki page was created to share the results of the activities.

In October 2020 a three-day special online session was conducted with a focus on the EECCA countries to provide an overview of the migration and remittances situation in these countries during the pandemic. The session explored the impact of the pandemic on the measurement of migration and remittances and review the use of administrative data in producing migration statistics. A consultant was also hired to carry out a survey on migration and remittances measurement in the pandemic which included data sources, methods and challenges faced and solutions applied. The results of the survey were presented and discussed during the meeting.

The June 2020 meeting of the Group of Experts on Consumer Price Indices had to be postponed to 2021 due to the pandemic. During the meeting in 2021 a draft Guide on producing CPI under lockdown was presented. The Guide provided recommendations and good practices for data collection, imputation methods and communication approaches that countries can refer to in periods of lockdown or extenuating circumstances. The Guide was published in August 2021 and is available in English on the UNECE website.

The second challenge that the project faced was the political instability in the region in 2022. Few mitigation measures are possible as events organized under ECASTAT are technical and not political events.

2.2.5 What (if anything) has prevented the project from achieving the desired results?

All the challenges and obstacles the project faced were addressed. The global pandemic and the forced change in the way the activities were conducted resulted in a no-cost project extension and additional workshops and sessions being organized and conducted. The use of online platforms for workshops and sessions increased the number of participants from targeted countries including statisticians as well as representatives of their central agencies.

17

2.3 EFFICIENCY

2.3.1 Were the resources allocated adequate for achieving results?

The project was sponsored by the ECASTAT regional trust fund established by the World Bank for assisting countries to improve their statistical system. According to the Project proposal a budget of $912,00010 (Table 1) was established to support the project.

TABLE 1: BUDGET ALLOCATION Activity Planned events Budget Amount - $912,000

Seminars and workshops

- Expert Group Meetings - Regional Workshops &

Seminars - Interpretation & logistics

- 5 - 6

$508,600

Translation of training material $29,400

Consultants & contractual services $178,000

Consultant – project administration $96,000

Staff Travel - 6 regional workshops $24,000

Miscellaneous costs $58,000

Independent project evaluation $18,000

During the initial project timeframe (2017-2020) $399,000 was used to support the project module activities, project administration and UN Programme support (Table 2). The unused $513,000 was the result of delayed or postponed activities due to the global pandemic as well as an increase in resources to support the project provided by EFTA and the UN Development Account Programme on Statistics and Data. The $513,000 was re-allocated to the no cost project extension that was approved for 2021-2023.

10 All financial figures are in US Dollars

18

TABLE 2: INITIAL PROJECT ALLOCATION OF FUNDS AND EXPENDITURES PER MODULE (2017-2020) Modules Budget Expenditures Unused Funds

Methodology & compilation techniques of 2008 SNA

$234,000 $38,000 $196,000

Price Statistics & compilation of national accounts aggregates at constant prices

$142,000 $58,000 $84,000

Improving the coverage and accuracy of statistical business registers & short-term statistics

$116,000 $47,000 $69,000

Expanding the use of household surveys for the estimates of the informal sector and remittances

$248,000 $183,000 $65,000

Project Administration, UN Programme Support and Project Evaluation

$172,000 $73,000 $99,000

TOTAL $912,000 $399,000 $513,000

The following activities were designed and implemented within the project extension: 3 meetings of Group of Experts on National Accounts; 2 meetings of the Group of Expert on Consumer Prices Indices; 1 meeting of the Group of Experts on Business Registers; a workshop on migration statistics and two regional workshops on National Accounts. Table 3 presents the budget allocation for the 2021-2023 project extension.

TABLE 3: BUDGET ALLOCATION FOR 2021-2023

Year Activity Budget

2021 Meeting of the Group of Experts on National Accounts Meeting of the Group of Experts on CPI Workshop on Migration Statistics

$126,000

2022 Meeting of the Group of Experts on National Accounts Meeting of the Group of Experts on Business Registers Regional workshop on National Accounts

$144,000

2023 Meeting of the Group of Experts on National Accounts Meeting of the Group of Experts on CPI Regional workshop on National Accounts

$144,000

Project Administration $49,000

UN Programme support costs $32,410

Independent project evaluation $17,880

TOTAL $513,290

19

The final financial reporting information was not completed at the time of this evaluation. However, the evaluation concluded that all the planned activities, including those added as a result of the no-cost extension, were completed. There were 24 Expert Group meetings, webinars, regional workshops, special sessions successfully conducted from 2017-2023 (see Annex 4.6). In addition, training materials were developed, a Guide on Producing CPI under Lockdown11 was developed and a harmonized survey module on migration and remittances for countries in EECCA was created (see 2.2.2 for details).

2.3.2 Were the results achieved on time and were all activities organized efficiently?

The project was completed within the allocated timeframe. Some activities were postponed due to COVID-19 and political unrest in the region, but all were eventually completed albeit in formats different from the original plan. The UNECE quickly adapted to the new global reality and implemented the use of online/virtual meetings in order to complete the planned activities. Efficient use of funds and inter-agency collaboration supported a no-cost extension to the project. The use of virtual meeting platforms increased the number of participants expanding the exchange and experiential learning encouraged by the project.

2.3.3. To what extent were the resources used economically and how could the use of resources be improved?

The financial resources were effectively and efficiently managed. In addition to the funding providing by the ECASTAT Trust Fund, the UNECE collaborated with EFTA which provided funding and in-kind contributions for many of the planned activities UNECE was also able to augment the available resources by using funds from the UN Development Account Programme on Statistics and Data to cover costs such as the participation of national experts in meetings and workshops as well as the critical translation of training materials. The no-cost extension for 2021-2023 was possible due to these additional sources of funding that occurred during the initial project timeframe (2017-2020) as well as the cost savings during COVID-19 due to the delay, postponement or replacement of face-to-face meetings with virtual sessions.

2.4 SUSTAINABILITY

2.4.1 What measures were adopted to ensure that project outcomes would continue after the project ended and to what extent have these measures addressed the existing risks for sustainability?

The following measures have been adopted to ensure the project outcomes continue: all project documentation is available through online channels and all activities encouraged the exchange of good practices and participation of EECCA experts in workshops, meetings and regional sessions. The project stimulated the creation of a network of experts from the EECCA region. This network which includes representatives from the national statistical offices as well as representatives from central banks and ministries of finance aims to improve the institutional cooperation and consistency between the national accounts and balance of payments.

In addition, the guidance notes of the SNA and Balance of Payments Manual, 6th Edition (BPM6) developed by the joint UN Task Team on Informal Economy has been translated into Russian to ensure that the EECCA region is fully involved in the consultation process.

The development of the harmonized survey module for a standard measurement of migration and remittances for countries in EECCA establishes the way in which these countries can arrive at estimates of migration and remittances that support SDG monitoring needs, are in line with international standards and are harmonized among CIS countries. The module is available as a standard tool and produced in Russian and English.

11 United Nations Geneva, 2021 Guide on producing CPI under lockdown https://www.unece.org/statistics/publications/guide-producing-cpi- under-lockdown

20

The results of the online survey of beneficiary countries rated the quality of support provided by the UNECE as “excellent” (60%) and “good” (40%). It also showed the extent of the work of the activities still relevant to their country as “to a high extent” (73%) and “to a moderate extent” (20%). When asked whether the outputs of the activities are sustainable, 47% said “to a high extent”; 40% said to a “moderate extent”, and 13% said “to a low extent”. The major risks identified for sustainability included lack of expert staff (67%), staff turnover (47%), limited technical capabilities (53%) and lack of funding (40%).

FIGURE 2: EVALUATION SURVEY – EVALUATION OF THE UNECE PROJECT ON ECONOMIC AND SOCIAL STATISTICS

* source – Project Evaluation Survey – Questions 8 and 6

2.4.2 To what extent do the partners and beneficiaries “own” the outcomes of the work? How is the stakeholders’ engagement likely to continue, be scaled up, replicated or institutionalized?

The UNECE has a mandate to support the implementation of 2008 SNA and supporting economics statistics in the EECA region. The Steering Group on National Accounts, established by the Bureau of the Conference of European Statisticians guides the work of the UNECE Statistical Division in the area of national accounts and supporting economic statistics, including activities to support and monitor the implementation of the 2008 SNA in the EECCA and to advise on the provision of methodological recommendations, training and other capacity building activities on national accounts. Country representatives from the EECCA are members of the Steering Group.

By nature of their mandates these partners/stakeholders are committed to maintaining and progressing the work undertaken during the project.

Continued sharing of experience and lessons learned is critical to the ongoing improvement of the statistical system in the beneficiary countries. EECCA participation in the Expert Group meetings is encouraged and where possible regional workshops would continue to offer continuous growth and learning. Lack of resources and funding are likely to be a major challenge in sustaining the work.

How would you rate the quality of the support provided by the UNECE

Excellent Good Average Poor

To what extent is the work still relevant to your country?

To a high extent To a moderate extent

To a low extent Not at all

21

2.5 GENDER AND HUMAN RIGHTS CONSIDERATIONS

Throughout the review, it was evident that women and men were involved in the various project activities, and in the evaluation process. Youth were also involved in the various activities. Consideration was given to those with physical disabilities during the workshops conducted. Training materials were translated to ensure beneficiary countries could learn in their preferred language. However, no gender consideration was recorded in the project’s proposal and no indicator established.

The evaluation process was inclusive by ensuring that a diverse group of stakeholders, partners and beneficiary countries were consulted and encouraged to provide information and input. This included women, men and youth. It is unknown if anyone who was consulted during the process was a person with disabilities. The two online surveys included a question on gender and the survey intended for stakeholders asked a question regarding the participation of women, persons with disabilities, minority groups and youths in the project’s activities and events. The interviews conducted with international and regional partners explored cross cutting questions regarding the inclusion of women, persons with disabilities, minority groups and youth as well as gender and human rights.

Although all UNECE projects are as inclusive as possible, this one included, the human rights dimension is a difficult and complex aspect to cover. The human rights dimension was not incorporated in the project proposal and the project outcomes make no explicit reference to the human rights. It should be considered however, that the Generic Law on Official Statistics (GLOS)12 requires “equal and simultaneous access” to official statistics for all users and is aligned with the United Nations Fundamental Principles of Official Statistics-Principle 1 of Relevance, Impartiality and Equal Access.13 The GLOS was developed within the UNECE Project “Strengthening national capacity in the most vulnerable UNECE countries for the sustainable development of statistics”. It was developed primarily for the beneficiary countries of the project and EECCA countries were consulted throughout its development.

12 https://unece.org/DAM/stats/publications/2016/ECECESSTAT20163_E.pdf

13 https://unstats.un.org/unsd/dnss/hb/E-fundamental%20principles_A4-WEB.pdf

22

3.0 CONCLUSIONS AND RECOMMENDATIONS

3.1 CONCLUSIONS

Based on the evaluation findings, the evaluator has the following conclusions.

The overall aim of the project was to increase the capacity of EECCA countries in compiling national accounts and supporting statistics according to the recommendations of the 2008 SNA and other relevant international standards. This was accomplished through a series of four inter-related modules with specific activities designed to achieve the goal to enhance coherence and integration of economic and social statistics in support of the implementation of the 2008 SNA. The project design was appropriate for meeting the needs and priorities of the beneficiary countries. All the activities designed to support the project’s objectives were completed on time and on budget.

Basic sound economics, including key figures of the national accounts are fundamental in relation to the SDGs. Several proposed SDG indicators on income, consumption, and production depend on the availability of comprehensive and high-quality national statistics that are internationally comparable. The project was designed to help EECCA countries improve the coherence, timeliness and internationally comparability of socio-economic statistics that are crucial for the monitoring of a country’s social and economic progress. The project design encouraged the collaboration of experts from the statistical offices as well as central agency representatives from national banks and finance ministries which improved the coherence of the statistical outputs.

The project activities were linked to the work programme of the UNECE Statistical Division to ensure continuity and coordination with international work on national accounts and supporting statistics. This is a critical element for continued success of future projects.

The most significant challenge that the project faced was the unprecedented global pandemic in early 2020 that forced the UNECE to examine and implement innovative ways to continue with the activities essential in achieving a successful outcome to the project. The use of virtual platforms saved costs and increased participation in the workshops and session organized within the project.

Although the UNECE Statistical Division has limited resources, its ability to draw on established partnerships and to obtain additional resources through in-kind or financial contributions extended the project to the benefit of the targeted countries and expanded the scope of the project to address outstanding issues, identified by EECCA countries, resulting from the COVID-19 pandemic.

During this evaluation the UNECE was recognized for its exceptional organization skills and the collaboration with other partners in the international statistical system to provide high quality learning activities. The UNECE was effective at bringing together expertise and ensuring no duplication with other international activities. The team members were also recognized for constantly seeking feedback and input and acting on the information gathered to improve subsequent events/activities. Consultation regarding agenda items and effective advanced planning ensured a positive experience for participants.

All UNECE work is designed to be as inclusive as possible, however, the human rights dimension is a difficult aspect to cover. The human rights dimension was not incorporated in the project proposal and the project outcomes make no explicit reference to the human rights. No gender consideration was recorded in the project’s proposal and no indicator established.

There were limitations in the evaluation. For example to properly assess whether the knowledge and expertise of the staff in the national statistical offices of EECCA countries improved and/or their capacity to implement standards and recommendations in practice could only be achieved through an in-depth country peer review or update to the Global Assessments. This was not done as part of this evaluation.

23

3.2 RECOMMENDATIONS

Based on the findings and the good practices identified throughout the evaluation of the project, the evaluator offers the following recommendations for consideration.

1. The global pandemic forced the UNECE to rethink the way it conducted business to ensure continuity of the project. These innovative solutions introduced good practices that should be continued. The UNECE is encouraged to use a hybrid model when organizing its workshops and regional sessions in the future. The introduction of short webinars and the video capture of in person sessions provided additional learning opportunities. A combination of face-to-face activities and virtual offerings reaches a broader group of people. Using appropriate and tested technologies that accommodates everyone’s participation is encouraged.

2. To properly assess the degree to which the knowledge and expertise of the staff in the national statistical offices of EECCA countries improved and/or their capacity to implement standards and recommendations increased, an in-depth country peer review or update to the Global Assessments should be conducted.

3. Continued sharing of experience and lessons learned is critical to the ongoing improvement of the

statistical system in the beneficiary countries. EECCA participation in the Expert Group meetings is encouraged and where possible regional workshops or special sessions would continue to offer continuous growth and learning.

4. To ensure sustainability, future activities should be linked to the work programme of the UNECE Statistical Division. Although the budget of the UNECE Statistical Division is limited, it is encouraged to use its exceptional collaboration and outreach skills to secure additional funding from key project stakeholders, be it in-kind or financial resources.

5. For gender and human rights perspectives to be better included in future projects, there should be an explicit objective built into the project design that can be monitored and evaluated during and at the end of the project. Expertise in gender quality, disability inclusion, environment and or human rights should be sought in the development of these objectives and with the evaluation process.

24

4.0 ANNEXES

4.1 TERMS OF REFERENCE FOR THE EVALUATION

I. Purpose

The purpose of this evaluation is to assess the extent to which the objectives of the UNECE project E264 “Enhancing coherence and integration of economic and social statistics in support of the implementation of 2008 System of National Accounts” were achieved.

The evaluation will assess the relevance, effectiveness, efficiency and sustainability of the project.

The evaluation will also assess any impacts the project may have had on progressing human rights, gender equality, disability inclusion, climate change and disaster risk reduction in the context of this engagement. The evaluation will finally look at the activities repurposed to address the impact of the COVID-19 crisis, and assess, to the extent possible, UNECE’s COVID-19 early response through this project.

II. Background

The project aimed at increasing the capacity of countries in Eastern Europe, Caucasus and Central Asia in compiling national accounts and supporting statistics according to the recommendations of international statistical standards. The activities of the project included the following four modules:

1) Methods and compilation techniques for implementation of the 2008 SNA and supporting economic statistics. 2) Price statistics - development of price indices and deflators for national accounts aggregates. 3) Statistical business registers - improving coverage and accuracy of national statistical business registers. 4) Expanding the use of household surveys for the estimates of the informal sector and remittances.

The activities were carried out in cooperation with other international and regional partners, including CIS-STAT, EFTA, Eurostat, ILO, IMF, OECD and World Bank.

III. Evaluation objectives, scope and questions

The evaluation will be guided by the objectives, indicators of achievement and means of verification established in the logical framework of the project document. The evaluation will be conducted in Q3 of 2023. It will cover the full implementation of the project, from June 2017 to June 2023 in Eastern Europe, Caucasus and Central Asia (Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Russian Federation, Tajikistan, Turkmenistan, Ukraine and Uzbekistan)

The final evaluation of the project has the following specific objectives:

· Determine as systematically and objectively as possible the relevance, effectiveness, efficiency and sustainability of the project results in light of its goals and objectives. · Assess how the project activities contributed to gender equality and women’ s empowerment, as well as the realization of human rights, with an emphasis on ‘leaving no one behind’ and, if needed, it will make recommendations on how these considerations can be better addressed in future activities of the sub-programme. · Identify good practices and lessons learned from the project and formulate action-oriented, forward- looking recommendations addressed to the sub-programme for improving future interventions.

25

The evaluation criteria are relevance, effectiveness, efficiency and sustainability.

Relevance 1. To what extent was the project design appropriate for meeting the needs of beneficiary countries? 2. To what extent did the project respond to the priorities and needs of national statistical offices in beneficiary countries? How relevant were the project activities to the countries’ needs and priorities? 3. To what extent was the project aligned with the SDGs? 4. What takeaways are there for ensuring relevance of future projects of UNEE Statistical Division? 5. To what extent were gender, human rights and disability perspectives integrated into the design and implementation of the project? What results can be identified from these actions? How can gender and human rights perspectives be better included in future projects design and implementation? Effectiveness 6. To what extent were the project objectives and expected results achieved? 7. To what extent did the project improve the competencies of national statistical offices in the beneficiary countries to produce and use macroeconomic statistics in their countries? 8. To what extent are the project activities coherent and harmonized with those of other partners operating within the same context, particularly those of other UN system entities? 9. What were the challenges/obstacles (including COVID-19 and sub-regional instability) to achieving the expected results? How successfully did the project overcome these? 10. What (if anything) has prevented the project from achieving the desired results? Efficiency 11. Were the resources adequate for achieving the results? 12. Were the results achieved on time and were all activities organized efficiently? 13. To what extent were the resources used economically and how could the use of resources be improved? Sustainability 14. What measures were adopted to ensure that project outcomes would continue after the project ended and to what extent have these measures addressed the existing risks for sustainability? 15. To what extent do the partners and beneficiaries ‘own’ the outcomes of the work? How is the stakeholders’ engagement likely to continue, be scaled up, replicated, or institutionalized? IV. Evaluation approach and methodology

The evaluation will be conducted in accordance with: the ECE Evaluation Policy; the Administrative instruction guiding Evaluation in the UN Secretariat; and the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation. Human rights and gender equality considerations will be integrated at all stages of the evaluation: (i) in the evaluation scope and questions; (ii) in the methods, tools and data analysis techniques; (iii) in the findings, conclusions and recommendations of the final report. The evaluator will explicitly explain how human rights, gender, disability, SDGs, and climate change considerations will be taken into account during the evaluation.

The evaluator is required to use a mixed-method approach, including qualitative as well as quantitative data gathering and analysis as the basis for a triangulation exercise of all available data to draw conclusions and findings.

The evaluation should be conducted based on the following mixed methods to triangulate information: 1. A desk review of all relevant documents, including the project document and information on project activities (monitoring data); materials developed in support of the activities (agendas, plans, participant lists, background documents, donor reports and publications); Proposed programme budgets covering the evaluation period; project reports to the donor.

26

2. Online survey of key stakeholders and beneficiaries: the survey will be developed by the consultant on her/his preferred platform. 3. Interviews (in-person and/or by telephone/video): the evaluator shall interview a wide range of diverse stakeholders and beneficiaries. 4. Remote observation of virtual workshops and meetings, including recordings of meetings.

The evaluator will further elaborate on the evaluation methodology in the Inception Report that will among others include the survey questions and whether any of the 12 countries will be selected for an in-depth assessment. The evaluation report will be written in English, will consist of approximately 30 pages and will include an executive summary (max. 2 pages) describing the evaluation methodology, key findings, conclusions and recommendations. The evaluator will also produce an Evaluation Brief summarizing key evaluation findings, lessons learned and recommendations, including through images and infographics.

V. Evaluation schedule

May 2023 ToR finalized June 2023 Evaluator selected June 2023 Contract signed. Evaluator starts the desk review July 2023 Evaluator submits inception report including survey design August 2023 Launch of data gathering, including survey and interviews September 2023 Evaluator submits draft evaluation report and evaluation brief October 2023 Evaluator submits final evaluation report and evaluation brief

VI. Resources and Management of the evaluation

An independent consultant will be engaged to conduct the evaluation under the management of the PMU. Payment will be made upon satisfactory delivery of work.

The Programme Management Unit (PMU) will manage the evaluation and will be involved in the following steps: Selection of the evaluator; Preparation and clearance of the Terms of Reference; Provision of guidance to the Project Manager and evaluator as needed on the evaluation design and methodology; Clearance of the final report after quality assurance of the draft report.

The Project Manager, in consultation with the Division Director, will be involved in the following steps: Provide all documentation needed for desk review, contact details, support and guidance to the evaluation consultant as needed throughout the timeline of the evaluation; Advise the evaluator on the recipients for the questionnaire and for follow-up interviews; Process and manage the consultancy contract of the evaluator, along the key milestones agreed with PMU.

VII. Intended use / Next steps

The results of the evaluation will be used in the planning and implementation of future activities of the UNECE Economic Cooperation and Integration Subprogramme. Findings of this evaluation will be used when possible to: · improve direct project’s follow up actions, implementation of products by project beneficiaries and dissemination of the knowledge created through the project; · assess the gaps and further needs of countries in the area of this project; · formulate tailored capacity building projects to strengthen the national capacity in enhancing innovation.

The results of the evaluation will be reported to the Conference of European Statisticians

27

Following the issuance of the final report, the Project Manager will develop a Management Response for addressing the recommendations made by the evaluator. The final evaluation report, the management response and the progress on implementation of recommendations will be publicly available on the UNECE website.

VIII. Criteria for evaluators

The evaluator should have: 1. An advanced university degree or equivalent background in relevant disciplines. 2. Knowledge of and experience in working on data and statistics; knowledge of and experience in economic statistics desirable. 3. Relevant professional experience in design and management of evaluation processes with multiple stakeholders, survey design and implementation, project planning, monitoring and management, gender mainstreaming and human-rights due diligence. 4. Demonstrated methodological knowledge of evaluations, including quantitative and qualitative data collection and analysis for end-of-cycle project evaluations, including demonstrated experience in conducting questionnaires and interviews. 5. Fluency in written and spoken English.

Evaluators should declare any conflict of interest to UNECE before embarking on an evaluation project and at any point where such conflict occurs.

28

4.2 ONLINE SURVEY QUESTIONNAIRE – BENEFICIARY COUNTRIES

4.2.1 Introductory text

This online survey is conducted in the context of the evaluation of the UNECE Project: Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 System of National Accounts.

The purpose of this survey is to gather the views of target countries and beneficiaries on the events and activities organized by the UNECE associated with this project.

The survey, which contains 15 questions, will be conducted in English and take approximately 15 minutes to complete. All responses are confidential, and no individual can be identified from their responses. UNECE will not have access to the raw survey results but only to aggregated information. Should you have any question or concerns, please contact the evaluator, Constance Graziadei at [email protected].

The survey will be available until Friday 22 September 2023.

Thank you in advance for your participation.

4.2.2 Questionnaire

1. Country • (drop down menu)

2. Which institution do you represent? • National Statistical Office • Other Domestic Agencies (Ministry of Finance, Central Bank) • Other: _______________________

3. Gender of respondent • Male • Female • Other: _________________ • Prefer not to respond

4. What event (meeting or workshop) organised by UNECE have you participate in? • National Accounts event • Price statistics event • Statistical business registers event • Events on expanding the use of household surveys for the estimates of the informal sector and

remittances

5. To what extent did the event (s) respond to your country’s statistical needs and priorities? • To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

6. To what extent is the work of the event (s) still relevant to your county? • To a high extent • To a moderate extent

29

• To a low extent • Not at all • Do not know

7. To what extent did the activities and outputs from the event help to improve your statistics? • To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

8. How would you rate the quality of the support provided by the UNECE? • Excellent • Good • Average • Poor • Do not know

9. How effective were the regional workshops or the UNECE Expert Group meetings? • Excellent • Good • Average • Poor • Do not know

10. How effective was the sharing of best practices throughout the event (s)? • Excellent • Good • Average • Poor • Do not know

11. To what extent did the project respond to the impact of the COVID-19 pandemic on planned activities? • To a high extent • To a moderate extent • To a low extent • Not at all • Do not know • Not applicable

12. To what extent are the outputs from the event sustainable? • To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

13. What are the main risks for the results to be maintained? Please select all that apply • Lack of expert staff • Staff turnover • Limited technical capacities

30

• Lack of funding • Government change • None • Other: ___________________________

14. Are there any lessons learned or good practices from the project that should be considered for future

projects?

Text box

15. Do you have any comments or recommendations?

Text box

4.3 ONLINE SURVEY RESULTS – BENEFICIARY COUNTRIES 15 responses (11 women; 4 men) were received and results analyzed. On average it took respondents 15:27 minutes to complete the questionnaire.

Q1. Country

4

1

4

1 0 0 1 0 1 2

0 0 1

Arm en

ia

Azer baij

an

Belar us

Geo rgi

a

Kaza kh

sta n

Kyrg yzs

tan

Moldova

Mongo lia

Russi an

Fe dera

tio n

Ta jik

ist an

Tu rkm

enist an

Ukra ine

Uzb ek

ist an

Countries Responses

31

Q2. Which institution do you represent?

Q3. Gender

Q4. What events (meeting or workshop) organized by the UNECE did you participate in?

National Statistical Office Other domestic agencies

Gender

Woman Man Prefer not to say Other

11

4 0 0

GENDER Gender

53% 27%

7% 13%

Event participants

System of National Accounts

Price Statistics

Statistical Business Register

Measuring Migration and Remittances

32

Q5. To what extent did the event (s) respond to your country’s statistical needs and priorities?

Q6. To what extent is the work of the event (s) still relevant to your country?

Q7. To what extent did the activities and outputs from the event (s) help to improve your statistics?

11 4

0 0

0 2 4 6 8 10 12

TO A HIGH EXTENT TO A MODERATE EXTENT

TO A LOW EXTENT NOT AT ALL

Statistical needs & priorities

Statistical needs & priorities

To a high extent 7

To a moderate

extent 8

To a low extent 0

Not at all 0

Improve Statistics

To a high extent To a moderate extent

To a low extent Not at all

33

Q8. How would you rate the quality of the support provided by the UNECE?

Q9. How effective were the regional workshops or the UNECE Expert Group meetings?

Very effective 12 Somewhat ineffective 0

Somewhat effective 3 Very ineffective 0

Neither effective or ineffective 0 Do not know 0

Q10. How effective was the sharing of best practices throughout the event (s)?

Very effective 12 Somewhat ineffective 0

Somewhat effective 3 Very ineffective 0

Neither effective or ineffective 0 Do not know 0

Q11. To what extent did the project respond to the impact of the COVID-19 pandemic on planned activities?

60%40%

QUALITY OF SUPPORT

Excellent Good

0

1

2

3

4

5

6

7

8

COVID-19 response

To a high extent

To a moderate extent

To a low extent

Not at all

Do not know

34

Q12. To what extent are the outputs from the various events sustainable?

Q13. What are the main risks for the results to be maintained? Please select all that apply

Q14. Are there any lessons learned or good practices from the project that should be considered for future projects?

- Measuring and consistent recording of non-observed economy in balance of payments the recording of e-commerce cross boarder transactions cryptocurrencies SPE MNE Groups data and others.

- A lot of theoretical information. - From the learned lessons we have planned the improvement of the statistical data production and tools. - Presentation of the experience of countries with good practices. - Good practices from the project helped us to establish Business Demography statistics and introduce geographical

coordinates in the Statistical Business Register. - Try to give the floor for all participation. - We made a research of card transactions, for 2019, by MCC and reflecting the results by EBOPS, after a presentation of

similar experience during NA Group of expert’s meetings and had a close collaboration with some colleagues to share our experience after 2023 meeting and presentation of our model of estimations of refugee’s transactions in BOP.

Q.15 Do you have any comments or recommendations?

- All UNECE events on price statistics are always very useful and helpful and informative. - We would like to see more workshops on price statistics, with participation from both regional statistical offices and

from countries with best practices in index calculations, which may be given the opportunity to consult themselves on

Sustainability

To a high extent To a moderate extent To a low extent Not at all

10

7 8

6

0 0 0 0 2 4 6 8

10 12

Lack of expert staff

Staff turover Limited technical

capabilities

Lack of funding Government change

None Other

Risk Factors

Risk Factors

35

the compilation of indices in practice, using the best practices for the collection of information they possess. Organize the exchange of experience between statistical offices in a particular country, to study practical skills in calculating indices.

- Participation in statistical business register events was highly informative for us. - To keep organizing regional meetings, additional to global events.

36

4.4 ONLINE SURVEY QUESTIONNAIRE – STAKEHOLDERS/PARTNERS

4.4.1 Introductory Text

This online survey is conducted in the context of the evaluation of the UNECE Project: Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 System of National Accounts.

The purpose of this survey is to gather the views of the stakeholders and partners who have invested time and resources to help the UNECE carry out its mandate to support the implementation of 2008 SNA and supporting statistics in the Eastern Europe, Caucasus and Central Asia region.

The survey, which contains 12 questions, will be conducted in English and take approximately 10 minutes to complete. All responses are confidential, and no individual can be identified from their responses. UNECE will not have access to the raw survey results, only to aggregated information. Should you have any questions or concerns, please contact the evaluator, Constance Graziadei at [email protected].

The survey will be available until 29 September 2023. Thank you in advance for your participation.

6.4.2 Questionnaire

1. Which organization do you represent? Please select all that apply.

• Drop down menu

2. Gender of respondent • Male • Female • Other: _________________ • Prefer not to respond

3.What is your relationship with the project?

• Donor • Expert • Oversight • Regional Partner • Sponsor • Other: _________________

4. To what extent did the project respond to the priorities and needs of the beneficiary countries?

• To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

5. How effective was the UNECE in drawing on established partnerships to achieve the results of the project?

• Very effective • Somewhat effective

37

• Neither effective nor ineffective • Somewhat ineffective • Very ineffective • Do not know

6. To what extent were the project's activities harmonized with activities, events or projects within your organization?

• To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

7. To what extent were alternative methods used to achieve the results more efficiently?

• To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

8. To what extent are the results of the project sustainable?

• To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

9. To what extent did the results of the activities and events organized by the UNECE align with the SDGs?

• To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

10. To what extent did the UNECE encourage the participation of women, persons with disabilities, minority groups and youths in the events and activities?

• To a high extent • To a moderate extent • To a low extent • Not at all • Do not know

11. Are there any lessons learned or good practices from the project that should be considered for future projects?

Text box

12. Do you have any comments or recommendations? Text box

38

4.5 INTERVIEW GUIDE – KEY STAKEHOLDERS

These questions are based on the evaluation matrix and will be used as a guideline only for interviews with international and regional partners. The questions will be tailored to each interview conducted. Follow up questions may be asked to explore emerging themes or discussions.

General

1. What is your relationship with the project and for how long have you been involved?

Relevance

2. To what extent did the project respond to the priorities and needs of the beneficiary countries? 3. To what extent did the activities and outputs from the project align with the project’s stated objectives?

a. Methodology and compilation techniques of 2008 System of National Accounts b. Price statistics c. Improving the coverage and accuracy of statistical business registers d. Expanding the use of household surveys for the estimates of the informal sector and remittances

Effectiveness

4. To what extend did the UNECE contribute to achieving the stated objectives? 5. How effective were the various activities used to achieve the project’s outcomes?

a. Regional workshops b. Participation in the UNECE Expert Group meetings c. Sharing of best practices

6. What were the challenges or obstacles to achieving the expected results? 7. What was the impact of the COVID-19 pandemic on planned activities? How was the project realigned to

help participant countries address these issues? 8. How effective was the UNECE in drawing on established partnerships to achieve the results of the project? 9. To what extent were the project activities coherent and harmonized with activities within your

organization?

Efficiency

10. Were the resources sufficient to achieve the desired results? 11. Were alternative methods used to achieve results more efficiently? Please describe.

Sustainability

12. To what extent are the outputs from the various modules sustainable? a. Methodology and compilation techniques for 2008 System of National Accounts b. Price statistics c. Improving the coverage and accuracy of statistical business registers d. Expanding the use of household surveys for the estimates of the informal sector and remittances

13. To what extent do the results of the project align with the SDGs?

Cross Cutting Questions

14. Was the project designed to ensure inclusion of women, persons with disabilities, minority groups and youth?

15. Did the UNECE include women, persons with disabilities, minority groups or youth? 16. How can gender and human rights perspectives be better included in the design and implementation of

future projects?

39

4.6 LIST OF PROJECT ACTIVITIES BY MODULE

4.6.1 Module 1 – National Accounts

YEAR ACTIVITY LOCATION

2017 Workshop on Consistency between National Accounts & Balance of Payments Statistics

Minsk, Belarus

2018 Workshop in compilation, dissemination and use of Supply and Use tables Meeting on the Group of Experts on National Accounts

Chisinau, Moldova Geneva, Switzerland

2020 Webinars of the Group of Experts on National Accounts Online

2021 Workshop on measuring informal economy Meeting on the Group of Experts on National Accounts

Online Online

2022 Workshop on Financial Accounts Meeting of the Group of Experts on National Accounts

Brussels, Belgium Online

2023 Meeting of the Group of Experts on National Accounts Geneva, Switzerland

4.6.2 Module 2 – Price Statistics

YEAR ACTIVITY LOCATION

2018 Meeting of the Group of Experts on Consumer Price Indices Geneva, Switzerland

2019 Regional Workshop on Consumer Price Indices for EECCA countries

Minsk, Belarus

2020 Webinars on producing the CPI under lockdown Online

2021 Meeting of the Group of Experts on Consumer Prices Indices Online

2023 Meeting of the Group of Experts on Consumer Prices Indices Geneva, Switzerland

40

4.6.3 Module 3 – Business Registers

YEAR ACTIVITY LOCATION

2017 Meeting of the Group of Experts on Business Registers Paris, France

2019 Meeting of the Group of Experts on Business Registers Geneva, Switzerland

2022 Workshop on Business Registers

Meeting of the Group Experts on Business Registers

Online

Online

4.6.4 Module 4 – Migration Statistics

YEAR ACTIVITY LOCATION

2017 Workshop on Migration Statistics Geneva, Switzerland

2018 Workshop on Migration Statistics

Harmonized survey module on international migration & remittances

Geneva, Switzerland

2019 Workshop on Migration Statistics

Testing of harmonized survey module on migration & remittances in countries

Geneva, Switzerland

2020 Analysis of the test results on the survey module

Workshop on Migration Statistics

Online

2021 Harmonized module for measuring & collecting information on migration & remittances after 2020

Workshop on Migration Statistics

Online

2022 Group of Experts on Migration Statistics Geneva, Switzerland

41

4.7 STAKEHOLDERS INVITED FOR IN-DEPTH INTERVIEW

ORGANIZATION FULL NAME TITLE RELATIONSHIP TO THE PROJECT

World Bank Mr. Maurice Nsabimana ECASTAT Task Team Leader Donor

ILO Ms. Valentina Stoevska Senior Statistician Group of Experts on CPI

EFTA Mr. Marius Andersen Deputy, EFTA Statistical Office Group of Experts on NA

European Union, Eurostat

Mr. John Verrinder Head, National Accounts, Methodology, Standards & Indicators

Group of Experts on NA

UNSD Mr. Herman Smith Chief of National Accounts Group of Experts on NA

Consultant Ms. Anna Prokhorova Consultant - Developer Harmonized module for measuring migration and remittances

UNECE Steering Group on National Accounts

Mr. Gerard Eding Director of National Accounts – Statistics Netherlands

Chair of Steering Committee

UNECE Steering Committee on Consumer Price Indices

Mr. Christopher Jenkins Assistant Deputy Director, Prices Division, Office for National Statistics, UK

Chair of Steering Committee

Armenia Ms. Lusya Khachartryan Head of Macroeconomics Indicators and National Accounts Division, Statistics Armenia

Group of Experts on National Accounts

United Kingdom Mr. Sanjiv Mahajan Head of Methods and Research Engagement, Office for National Statistics, UK

Steering Group on National Accounts

42

4.8 LIST OF DOCUMENTS REVIEWED

1. ECASTAT Project for 2017-2020 Implementation of 2008 SNA. Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 SNA – Project Proposal – Concept note 2016

2. ECASTAT Project no-cost extension for 2021-2023. Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 SNA – Project Proposal

3. Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 SNA https://www.statswiki.unece.org/display/EC/ECSTAT+Home

4. World Bank’s Implementation Completion and Results Report on a Small Grant to the United Nations Economic Commission for Europe (UNECE) for Enhancing Coherence and Integration of Economic and Social Statistics in Support of the Implementation of 2008 SNA (Draft-v1)

5. Statistical Commission. Report on the fortieth session (24-27 February 2009). https://unstats.un.org/unsd/statcom/doc09/Report-English.pdf

6. SDG Priorities. Supporting countries to achieve the SDGs https://unece.org.sdg-priorities

7. United Nations Geneva, 2021 Guide on producing CPI under lockdown https://unece.org/statistics/publications/buide-producing-cpi-under-lockdown

8. Generic Law on Official Statistics https://unece.org/DAM/stats/publications/2016/ECECESSTAT20163_E.pdf

9. United Nations Fundamental Principles of Official Statistics https://unsd/dnss/hb/E- fundamentatl%principles_A4-WEB.pdg

10. Interim Unaudited Financial Reports - June - December 2017 - January - June 2018 - July – December 2018 - January – May 2019 - June - December 2020 - January – June 2021 - July – December 2021 - January – June 2022

11. Reports from Meetings and Workshops

- Group of Experts on Business Registers (2017; 2019; and 2022) - Group of Experts on Consumer Price Indices (2018; 2021; and 2023) - Consumer Price Indices webinars (2020) - Group of Experts on Migration Statistics (2020; 2021; and 2022) - Group of Experts on National Accounts (2018; 2020; 2021; 2022; and 2023) - Workshop on Migration Statistics (2019) - Workshop on Consumer Price Indices (2019) - Workshop on Financial Accounts (2022) - Workshop on Measuring Informal Economy (2021) - Workshop on Migration Statistics (2017; 2018) - Workshop on Supply and Use Tables for EECCA and SEE countries (2018) - Workshop on the Implementation 2008 SNA: Consistency between national accounts and balance of

payments (2017) 12. Meeting Evaluation Results

- Workshops of Migration Statistics (2017; 2018; and 2019)

43

- Meeting on Migration Statistics (2021) - Meetings of the Group of Experts on National Accounts & webinars (2020; 2021; 2022; and 2023) - Regional Workshop on Consumer Price Indices 2019 - Meetings of the Group of Experts on Consumer Price Indices (2018; 2021 and 2023) - Meeting of the Group of Experts on Business Registers 2019

13. Semi-annual progress reports

- June – December 2017 - January – June 2018 - July – December 2018 - January – May 2019 - June – November 2019 - December 2019 – May 2020 - June 2020 – December 2020 - January 2021 – June 2021 - July 2021 – December 2021 - January 2022 – June 2022 - July – December 2022

14. Global Assessment of the National Statistical Systems of EECCA (beneficiary) countries, where available:

Armenia (2009), Azerbaijan (2017), Belarus (2020), Georgia (2019), Kazakhstan (2017), Kyrgyzstan (2011), Moldova (2013), Mongolia (2014), Tajikistan (2013), Turkmenistan (2014), Ukraine (2017), Uzbekistan (Joint review - 2018)

Report on enhancing national capacities to develop and implement energy efficiency standards for buildings in the United Nations Economic Commission for Europe region

The present document was developed based on the outcomes of the 2020-2022 project "Enhancing National Capacities to Develop and Implement Energy Efficiency Standards for Buildings in the UNECE Region" and contains results of evaluations from the training seminars and analysis of the impact of project activities on improving energy efficiency in buildings in the project’s beneficiary countries.

Languages and translations
English

GE.22-11559(E)

Economic Commission for Europe

Committee on Sustainable Energy

Group of Experts on Energy Efficiency

Nineth session

Geneva, 3-4 October 2022

Item 7 of the provisional agenda

Improving energy efficiency in buildings

Report on enhancing national capacities to develop and implement energy efficiency standards for buildings in the United Nations Economic Commission for Europe region

Note by the secretariat

Summary

During the period July 2020 to March 2022, the United Nations Economic

Commission for Europe implemented the project “Enhancing National Capacities to Develop

and Implement Energy Efficiency Standards for Buildings in the UNECE Region”.

The Group of Experts on Energy Efficiency at its eighth session (20-21 September

2021) requested (ECE/ENERGY/GE.6/2021/2) the results of project implementation be

reported at the ninth session of the Group of Experts.

The present document was developed in response to this request. It contains results

of evaluations from the training seminars and analysis of the impact of project activities on

improving energy efficiency in buildings in the project’s beneficiary countries.

United Nations ECE/ENERGY/GE.6/2022/3

Economic and Social Council Distr.: General

22 July 2022

Original: English

ECE/ENERGY/GE.6/2022/3

2

I. Introduction

1. The project “Enhancing National Capacities to Develop and Implement Energy

Efficiency Standards for Buildings in the UNECE Region” builds on previous activities of

the United Nations Economic Commission for Europe (ECE) in the area of energy efficiency

standards in buildings and aims to enhance the capacity of the ECE member States to develop

and implement such standards.

2. The project duration was from July 2020 to March 2022. The project implemented all

of the requested activities, namely:

(a) A study on gap analysis between the performance objectives set forth in the

Framework Guidelines for Energy Efficiency Standards in Buildings

(ECE/ENERGY/GE.6/2020/4) and current energy efficiency standards and their

implementation in the countries of South-Eastern and Eastern Europe, the Caucasus, Central

Asia, and in the Russian Federation has been conducted;1

(b) The findings and recommendations from this regional study were discussed at

a workshop for stakeholders from the energy and housing sectors to validate the gap analysis

on 9 April 2021;2

(c) Three national studies (covering Armenia, Kyrgyzstan and the Republic of

Moldova) with a more detailed gap analysis have also been developed;3

(d) The country-specific recommendations from those studies were discussed at a

workshop for stakeholders from the energy and housing sectors on 20 September 2021;4

(e) The project established a Collaborative Environment for Experts on Energy

Efficiency in Buildings in the UNECE region and updated the network of experts from public

and private sectors on energy efficiency in buildings;5

(f) In October and November 2021, national training seminars on high-

performance energy efficiency standards in buildings were organized in Armenia and

Kyrgyzstan.6 A national training seminar for the Republic of Moldova was conducted on 20

-21 January 2022;7

(g) Impact of project activities was evidenced by conducting an impact study on

how project countries use and implement best practices and guidelines provided by ECE

activities in their national or/and sub-national measures to address the issues of energy

efficiency in buildings.8

II. Trainings on high-performance energy efficiency standards in buildings in the selected project countries

3. One of the project activities was to conduct national training seminars on high-

performance energy efficiency standards in buildings in selected ECE member States. The

trainings were organized in Yerevan, Armenia on 25-26 October 2021; Bishkek, Kyrgyzstan

on 29-30 November 2021; and Chisinau, Republic of Moldova on 20-21 January 2022.

1 See: https://unece.org/sites/default/files/2021-06/Study_on_Gap_Analysis_07.06.2021.pdf

2 See: https://unece.org/sustainable-energy/events/online-workshop-energy-efficiency-standards-

buildings-and-their

3 See: https://unece.org/sustainable-energy/regional-advisory-services/gap-analysis-and-national-

studies

4 See: https://unece.org/sustainable-energy/events/workshop-regional-and-national-studies-gap-

analysis-between-performance

5 See: https://sedwiki.unece.org/display/SED/EEEB_db_Home

6 See: https://unece.org/sustainable-energy/regional-advisory-services/national-training-seminars-high-

performance-energy

7 See: https://unece.org/info/events/event/364307

8 See: https://unece.org/sustainable-energy/regional-advisory-services/impact-study

ECE/ENERGY/GE.6/2022/3

3

4. The objective of the trainings was to: (i) improve understanding of the energy

management and monitoring, reporting, and verification (MRV) in the buildings sector;

(ii) improve understanding of institutional, organizational, and legal aspects of energy

management and MRV at the global and European Union level; (iii) consolidate capacities

to organize municipal energy management as part of the national MRV system; (iv) provide

knowledge to use existing or organize the development of an Energy Management

Information System; (v) provide an understanding of the energy auditing process and

confidence in using multicriteria analysis tools; and (vi) enhance capacities to use and

understand the funding possibilities for energy efficiency measures in buildings.

5. The selection of training participants was based on the open invitation sent by

governmental institutions and the United Nations Development Programme (UNDP) Country

Offices of the three ECE member States. Based on the training evaluation,9 in Armenia most

of the participants were from public institutions; in Kyrgyzstan the majority were from

private companies and universities; and in the Republic of Moldova they were primarily from

public institutions and private companies. All together 180 participants followed the trainings

conducted both online and in-person. Out of them, in Armenia, 46 participants followed the

training. In Kyrgyzstan, there were 44 participants. In the Republic of Moldova, 90

participants followed the training.

6. Three training seminars were organized as a two-day event. At each training the

presentations were distributed into six sessions. The first day sessions were focused more on

energy audit procedures, energy management information systems and municipal energy

management. The second day sessions were oriented towards institutional, organizational,

and legal aspects of energy management and monitoring, reporting and verification of energy

efficiency measures, and economic evaluation and funding possibilities of energy efficiency

measures in buildings. The training seminar included classroom lectures (with in-person

participation and online connection) and classroom calculation exercises using various tools.

7. The main outcome of the training seminar was achieved: the participants gained

knowledge and expertise on high-performance energy efficiency standards in buildings. At

the end of each training seminar the participants were asked to fill in the questionnaire to

share their perspective on the delivered national training course. The trainees were asked to

provide their evaluation of organisational issues, the quality of training, and self-assessment

of their own understanding of topics covered by the training.

8. Organization of the training was evaluated as follows: 4.6 out of 5.0 in Armenia, 4.2

out of 5.0 in Kyrgyzstan, and 4.8 out of 5.0 in the Republic of Moldova. Venue of the training

seminar, the number and duration of breaks, and the time schedule of lectures were scored in

the countries as 4.7, 4.6, and 4.8 out of 5.0. The average score of the overall duration of the

training seminar was 4.3 out of 5.0.

9. Participants also scored (on the scale from 1.0 to 5.0) each session with various scores

as presented in the Table below. The overall average scores of all the sessions were 4.4 in

Armenia,4.0 in Kyrgyzstan, and 4.7 in the Republic of Moldova.

Table

Quality of sessions as assessed by session participants

Title of session Armenia Kyrgyzstan

Republic of

Moldova

Averag

e

Introduction to energy management and monitoring,

reporting and verification in the buildings sector

4.3 3.8 4.7 4.3

Institutional, organizational and legal aspects of

energy management and monitoring, reporting and

verification at the global and EU level

4.5 4.2 4.7 4.5

9 More information is available in the Final Report on National Training Seminars:

https://unece.org/sites/default/files/2022-03/Final%20Report.pdf

ECE/ENERGY/GE.6/2022/3

4

Title of session Armenia Kyrgyzstan

Republic of

Moldova

Averag

e

Municipal energy management as part of the national

monitoring, reporting and verification system

4.2 4.0 4.7 4.3

Energy management information system and

examples of energy management system tools

4.3 4.0 4.7 4.3

Energy audits procedure of buildings and

multicriteria analysis

4.6 4.0 4.7 4.4

Best practices of funding possibilities for energy

efficiency measures in buildings

4.2 4.1 4.8 4.4

Average 4.4 4.0 4.7 4.4

10. Similarly, the participants scored the overall satisfaction with the training seminar

(4.3) as well as the ratio between theoretical and practical part of the training seminar (4.4).

The set of questions related to participants’ improvement of understanding of the energy

efficiency topics presented during the training seminar, their enhanced capacities to perform

specific tasks related to energy efficiency, and improved capacities to deliver presentations

on any specific topic presented is scored as very good with average scores of 4.0, 3.8 and 3.9

for Armenia, Kyrgyzstan, and Republic of Moldova, respectively.

11. The participants of the training seminars indicated the need for more such events to

be organized regularly reflecting more local issues related to various field of energy

efficiency and energy management information systems implementation in the countries.

Some mentioned the need of such trainings focused on industrial energy management and

energy auditing of industrial facilities with practical examples on how to use metering

devices. Many participants commented that they would like to have more intensive energy

auditing trainings. Some participants suggested to include more videos and visualisations for

future trainings and provide more examples and more information on tools that they could

use in practice. Most of the participants were, in general, very satisfied and commented that

the trainings were well-organized and that the information provided was well structured.

III. Impact study on how member States could better use and implement best practices and guidelines to improve energy efficiency in buildings

12. The project was implemented with the goal of achieving significant change by

improving the knowledge of policymakers and experts from ECE member States on energy

efficiency standards, including on enforcement mechanisms and energy-efficient

technologies in buildings. The impact study analysed the project’s impact and how member

States could better use and implement the best practices and guidelines developed by ECE in

their national and/or sub-national measures to address the issues of energy efficiency in

buildings.

13. The study investigated the project’s impact on: (i) its gender perspective; (ii) overall

relevance, including its design and its specific activities; (iii) the extent to which member

States implemented the recommendations and best practices, accompanied by examples from

respondents’ countries; (iv) the extent to which the project provided guidance for

implementation of recommendations; (v) the extent to which the project increased

stakeholder capacity to implement the recommendations; (vi) the remaining challenges of

member States to implement the recommendations; and, (vii) the extent to which countries

implemented recommendations from the cross-country or regional perspective.

14. Both an online questionnaire and in-person interviews were conducted for the

purposes of the impact study. The analysis of responses delivered valuable results regarding

the relevance of the project, its impact, and suggestions for future projects.

ECE/ENERGY/GE.6/2022/3

5

15. The results of questionnaire revealed that project activities advocated for gender

equality. On a scale from 1.0 (very low) to 5.0 (very high), the average score was 3.7,

indicating that respondents perceived that gender equality was advocated. Furthermore,

participation of women in workshops serves as a good indicator of support in achieving the

gender equality. The online Workshop on Energy Efficiency Standards in Buildings and their

Implementation in the UNECE region, held on 9 April 2021, included 68 participants, of

which 24 were women (35 per cent). In the Workshop on Regional and National Studies on

a Gap Analysis between the Performance Objectives of the Framework Guidelines for Energy

Efficiency Standards in Buildings and Implementation of Current Building Energy

Efficiency Standards, which was held on 20 September 2021, 162 experts participated, of

which 34 were women (20 per cent). The Workshop on Best practices to Address the Issues

of Energy Efficiency in Buildings and their Implementation in UNECE member States on 11

March 2022 was attended by 64 participants, of which 28 were women (43 per cent). These

numbers show that participation of women in the workshops was lower than that of men.

Taking into consideration that women are a minority in the energy sector, the numbers are

not surprising. Together with the high average score on the survey question regarding gender

equality, the figures are appropriate to conclude that the project performed well in terms of

supporting the gender equality. Nevertheless, more efforts should be made to ensure equal

opportunities for participation of men and women in the future activities.

16. The impact analysis has shown that the project was relevant to achieving its objective

and has had a valuable impact on all its key aspects. The survey respondents indicated that

their country implemented the recommendations and best practices from regional and

national studies. The survey also revealed that, overall, respondents are moderately confident

about the future implementation of recommendations and best practices. The respondents

were quite confident that the implemented recommendations and best practices will sustain

in the future.

17. The interviews have shown that member States regarded the recommendations and

best practices developed by ECE as very helpful and that countries have undergone efforts to

implement them. All countries recognize the importance of energy efficiency in buildings

and the pressing need to implement the recommendations and best practices. In line with the

results from the survey, the countries have indicated several measures that have been

implemented at the national or local levels to improve energy efficiency standards in

buildings.

18. The key focus of the interviews was to gain insights into how the recommendations

provided guidance to countries on the topic of energy efficiency in buildings. Interviewees

indicated that a key aspect of providing guidance was addressed in the regional study through

the gap analysis, national studies, and national trainings. The latter presented the necessary

steps that countries need to take in order to improve energy efficiency in buildings. They

provided knowledge and expertise gained from countries that are more developed in the area

of energy efficiency in buildings, helped other countries to bring that knowledge back to their

national decision- and policy makers. Furthermore, the regional study’s gap analysis was

mentioned as an essential tool that showed a need to boost transition progress in the energy

sector and communicated the urgency that energy efficiency should be a focus of national

and local efforts. Sharing of best practices, including on data collection, monitoring and

verification, energy audits, and implementation of specific measures has provided valuable

guidance to the project stakeholders. Consequently, this provides a value added, especially

in the long term. Sharing of knowledge and recommendations, together with the success

factors, has proven to be a key activity of the project that provides guidance to member States.

19. The impact analysis looked at the materialized impact that the project had on member

States. Since the project has been completed very recently, the specific observable benefits

were expected to be limited. Nevertheless, respondents indicated several activities that have

been undertaken in their country to use and implement the recommendations. Overall, the

responses can be grouped into several main categories:

(a) The impact was on implementation of national laws and policies, meaning that

countries either revised existing laws or adopted new laws on energy efficiency in buildings.

Additionally, normative documents and national energy strategies were updated in the project

countries. For example, Albania adopted several by-laws implementing the 2016 Law on

ECE/ENERGY/GE.6/2022/3

6

Energy Performance in Buildings, including the methods for calculating and setting

minimum energy performance requirements and certification of buildings. Georgia is

working to harmonize its national legislation with European Union (EU) Directives.

Kyrgyzstan worked on the legal framework and started implementing practical steps to

address the requirements set in the relevant laws. The Republic of Moldova started to update

its energy efficiency laws based on the EU Directives;

(b) The impact was also on updating national norms and standards. Countries have

updated several normative documents and revised national standards on energy efficiency in

buildings. Specifically, the regional and national studies have focused the attention of

decision-makers on the gaps in neighbouring countries, motivating them to update the

standards to reflect the level of efficiency for both existing and new buildings in the countries.

For example, Armenia developed energy efficiency standards based on adjustments to the

EU standards, and those standards were fully adopted by January 2022. These standards will

be mandatory for all entities engaged in planning and construction work;

(c) The project led to improved measurements. Following the project

participation, countries improved their data collection and measurement efforts as well as

measured building energy performance in accordance with International Organization for

Standardization (ISO) standards. Armenia, for example, implemented ISO standards.

Specifically, the ISO standards were translated into the local language and registered as

national standards;

(d) The project countries increased the number of energy audits being conducted.

More audits have been performed following the project implementation. For example,

Kyrgyzstan certified 20 energy auditors to assess the energy efficiency in buildings and issue

the certifications. Construction companies are now required to engage energy auditors to

certify the newly constructed buildings. Even though a lot of progress has been made in all

countries, it is still a long process, and national systems are slow to absorb the changes;

(e) The impact was seen on implementation of energy management information

systems. Some experts indicated that their countries have implemented the energy

management information systems;

(f) The project contributed to awareness-raising. The training seminars conducted

in the selected countries increased awareness about the topic of energy efficiency in

buildings. Further, through awareness-raising campaigns and marketing, the public was

informed about the importance of energy efficiency. In the longer term, this translates into

the increased investments in retrofits or higher standards for newly constructed buildings.

Educating homeowners about the benefits of investing in energy efficient retrofits of existing

buildings or the construction of new buildings according to the high-performance standards

is crucial to saving energy in the long term.

20. The impact study showed that countries engaged in efforts to raise awareness about

the importance of energy saving and energy efficiency in buildings. On one hand, they

informed decision- and policy makers about the need to improve legislative framework and

update the norms and standards frequently. On the other hand, they also promote the need to

invest in energy efficiency in existing buildings through retrofits. For example, awareness-

raising activities in Armenia are implemented through extensive campaigns on energy

efficiency in buildings and through engaging influencers and other famous characters and

opinion makers. Also, through movies and banners, Armenia engaged in active efforts to

communicate the benefits of investing in energy efficiency in buildings, such as the

percentage of energy saved and the adhering cost savings. In Kyrgyzstan, active sharing of

knowledge and promotion of the importance of energy efficiency in buildings have been an

outcome that can be attributed to the project.

21. The study also revealed that countries are engaged in implementing and updating the

building codes. Yet, there is a room for improvement, especially since buildings codes are

not updated frequently enough. In Armenia, for example, the energy measurements of

building codes do not account for cooling or hot water. The process is also moving in the

direction to include cooling and lighting into the building codes. Also, other countries, like

North Macedonia and the Republic of Moldova indicated that work on building certification

is ongoing but that the process is not yet finalized. Interviews voiced the fact that

ECE/ENERGY/GE.6/2022/3

7

improvement of energy efficiency in buildings is a long process and that continuous efforts

are required for ensuring that efficiency gains are met.

22. The impact study looked at how exactly the project was helping the countries to

implement recommendations and best practices. One aspect was that the project, through the

workshops and trainings, provided benefits beyond the technical training of participants. The

project offered a platform for communication among experts. The interviewees emphasized

the benefit of having an opportunity to exchange information and engage in discussions

among experts within the framework of the project, especially after the COVID-19 pandemic.

Knowledgeable experts facilitated discussions on a detailed level during the project events.

23. Another objective of the impact study was to formulate recommendations for future

projects and how member States could better implement recommendations and best practices.

The analysis of the interviews indicated that stakeholders believe that projects should focus

on awareness raising and provision of information to the general public to educate about the

importance of energy efficiency in buildings. Additionally, more examples should be

provided, such as best practices on project implementation, public-private partnerships, and

practical examples on the implementation of measures and energy saving calculations.

Another recommendation was that projects should provide detailed information on energy

measurements in accordance with the ISO standards. Respondents suggested working on

climate data for monthly and hourly calculations. The analysis has shown that respondents

suggest both a regional focus and a country-specific focus of future projects. While the

regional focus is good for learning about experiences and best practices, the countries are

interested in recommendations that are specific to their national context and infrastructure.

  • Table Quality of sessions as assessed by session participants
French

GE.22-11559 (F) 100822 100822

Commission économique pour l’Europe

Comité de l’énergie durable

Groupe d’experts de l’efficacité énergétique

Neuvième session

Genève, 3 et 4 octobre 2022

Point 7 de l’ordre du jour provisoire

Améliorer l’efficacité énergétique des bâtiments

Rapport sur le renforcement des capacités nationales d’élaboration et d’application de normes relatives à l’efficacité énergétique des bâtiments dans la région de la Commission économique pour l’Europe

Note du secrétariat

Résumé

Entre juillet 2020 et mars 2022, la Commission économique pour l’Europe (CEE) a

mis à exécution le projet sur le renforcement des capacités nationales d’élaboration et

d’application de normes relatives à l’efficacité énergétique des bâtiments dans la région de

la CEE.

À sa huitième session (20 et 21 septembre 2021), le Groupe d’experts de l’efficacité

énergétique a demandé que les résultats de ce projet lui soient présentés à sa neuvième

session (voir le document ECE/ENERGY/GE.6/2021/2).

On trouvera dans le présent rapport, qui fait suite à cette demande, les résultats des

évaluations des séminaires de formation et une analyse de l’incidence des activités relatives

à l’amélioration de l’efficacité énergétique des bâtiments dans les pays bénéficiaires du

projet.

Nations Unies ECE/ENERGY/GE.6/2022/3

Conseil économique et social Distr. générale

22 juillet 2022

Français

Original : anglais

ECE/ENERGY/GE.6/2022/3

2 GE.22-11559

I. Introduction

1. Le projet sur le renforcement des capacités nationales d’élaboration et d’application

de normes relatives à l’efficacité énergétique des bâtiments dans la région de la CEE fait fond

sur les activités antérieures de la CEE à cet égard et vise à renforcer les capacités de ses États

membres.

2. Le projet a été mis en œuvre entre juillet 2020 et mars 2022. Toutes les activités

prévues ont été menées à bien, à savoir :

a) Une étude portant sur l’analyse des écarts entre les objectifs d’efficacité

énoncés dans les Orientations-cadres pour l’élaboration de normes sur l’efficacité

énergétique dans les bâtiments (ECE/ENERGY/GE.6/2020/4) et les normes d’efficacité

énergétique actuelles et leur application dans les pays d’Europe du Sud-Est, d’Europe

orientale, du Caucase et d’Asie centrale et dans la Fédération de Russie1 ;

b) L’examen, le 9 avril 2021, à l’occasion d’un atelier destiné aux parties

prenantes des secteurs de l’énergie et du logement, des conclusions et recommandations

issues de cette étude régionale, afin de valider l’analyse des écarts2 ;

c) Trois études nationales (portant sur l’Arménie, le Kirghizistan et la République

de Moldova) et une analyse des écarts plus détaillée3 ;

d) L’examen, le 20 septembre 2021, des recommandations découlant de ces

études pour chacun des trois pays, à l’occasion d’un atelier destiné aux parties prenantes des

secteurs de l’énergie et du logement4 ;

e) La mise en place d’un cadre collaboratif destiné aux experts en efficacité

énergétique des bâtiments dans la région de la CEE et l’amélioration du réseau d’experts des

secteurs public et privé en matière d’efficacité énergétique des bâtiments5 ;

f) L’organisation, en octobre et novembre 2021, en Arménie et au Kirghizistan,

de séminaires de formation nationaux sur les normes relatives aux bâtiments à haute

performance énergétique6. Un séminaire national de formation pour la République de

Moldova a été organisé les 20 et 21 janvier 20227 ;

g) L’évaluation de l’incidence des activités menées dans le cadre du projet grâce

à une étude sur la manière dont les pays participant au projet utilisent et appliquent, dans le

cadre de leurs mesures nationales ou infranationales visant à améliorer l’efficacité

énergétique des bâtiments, les meilleures pratiques et les directives élaborées par la CEE8.

II. Formations sur les normes relatives aux bâtiments à haute performance énergétique dans les pays sélectionnés pour le projet

3. Des séminaires de formation nationaux sur les normes relatives aux bâtiments à haute

performance énergétique dans certains États membres de la CEE ont été organisés dans le

cadre du projet à Erevan, en Arménie, les 25 et 26 octobre 2021, à Bichkek, au Kirghizistan,

les 29 et 30 novembre 2021, et à Chisinau, en République de Moldova, les 20 et 21 janvier

2022.

1 Voir : https://unece.org/sites/default/files/2021-06/Study_on_Gap_Analysis_07.06.2021.pdf.

2 Voir : https://unece.org/sustainable-energy/events/online-workshop-energy-efficiency-standards-

buildings-and-their.

3 Voir : https://unece.org/sustainable-energy/regional-advisory-services/gap-analysis-and-national-

studies.

4 Voir : https://unece.org/sustainable-energy/events/workshop-regional-and-national-studies-gap-

analysis-between-performance.

5 Voir : https://sedwiki.unece.org/display/SED/EEEB_db_Home.

6 Voir : https://unece.org/sustainable-energy/regional-advisory-services/national-training-seminars-

high-performance-energy.

7 Voir : https://unece.org/info/events/event/364307.

8 Voir : https://unece.org/sustainable-energy/regional-advisory-services/impact-study.

ECE/ENERGY/GE.6/2022/3

GE.22-11559 3

4. Ces formations avaient pour objectifs : i) de faire mieux comprendre la gestion de

l’énergie et du suivi, de la notification et de la vérification en la matière dans le secteur des

bâtiments ; ii) de faire mieux comprendre les aspects institutionnels, organisationnels et

juridiques de la gestion de l’énergie et du système de suivi, de notification et de vérification

aux niveaux mondial et de l’Union européenne ; iii) de renforcer les capacités en matière

d’organisation de la gestion de l’énergie au niveau municipal dans le cadre du système

national de suivi, de notification et de vérification ; iv) de transmettre des connaissances

permettant d’utiliser le système existant d’information sur la gestion de l’énergie ou

d’organiser la mise en place d’un tel système ; v) de faire comprendre la procédure d’audit

énergétique et de permettre d’utiliser en toute confiance les outils d’analyse multicritères ;

vi) de renforcer les capacités permettant de comprendre les possibilités de financement des

mesures permettant d’améliorer l’efficacité énergétique des bâtiments et de les mettre à

profit.

5. Les participants à ces formations ont été choisis sur la base des réponses à une

invitation générale adressée par les organismes publics et les bureaux de pays du Programme

des Nations Unies pour le développement (PNUD) des trois États membres de la CEE

concernés. Les données fournies lors de l’évaluation de ces formations9 montrent : qu’en

Arménie, la plupart des participants venaient d’organismes publics ; qu’au Kirghizistan, la

majorité des participants étaient issus d’entreprises privées et d’universités ; et qu’en

République de Moldova, les participants appartenaient principalement à des organismes

publics et à des entreprises privées. Au total, 180 participants ont suivi ces formations, qui

étaient dispensées à la fois en ligne et en présentiel. En Arménie, la formation a été suivie

par 46 participants, au Kirghizistan, par 44 participants, et 90 personnes ont participé à la

formation dispensée en République de Moldova.

6. Les trois séminaires de formation avaient une durée de deux jours. Chacun comportait

six séances. Les séances du premier jour étaient plutôt axées sur la procédure d’audit

énergétique, les systèmes d’information sur la gestion de l’énergie et la gestion de l’énergie

au niveau municipal. Celles du deuxième jour étaient axées sur les aspects institutionnels,

organisationnels et légaux de la gestion de l’énergie, du système de suivi, de notification et

de vérification des mesures d’efficacité énergétique, ainsi que de l’évaluation économique

des mesures d’efficacité énergétique des bâtiments et des possibilités de financement de telles

mesures. Les séminaires comprenaient des exposés (qu’il était possible de suivre en

présentiel ou en ligne) et des ateliers sur place permettant de se familiariser avec divers outils

de calcul.

7. Le principal objectif des séminaires de formation a été atteint, dans la mesure où les

participants ont acquis des connaissances et des compétences sur les normes relatives aux

bâtiments à haute performance énergétique. À la fin de chaque séminaire, les participants ont

été invités à donner leur avis en remplissant un questionnaire d’évaluation. Ils étaient invités

à se prononcer sur l’organisation et la qualité de la formation et à auto-évaluer leur

compréhension des sujets abordés.

8. L’organisation de la formation a été notée sur 5, comme suit : 4,6 en Arménie, 4,2 au

Kirghizistan et 4,8 en République de Moldova. Les locaux dans lesquels se déroulait la

formation, le nombre et la durée des pauses, et l’horaire des cours ont respectivement été

notés, pour ces trois pays 4,7, 4,6 et 4,8. La note moyenne pour l’évaluation de la durée

globale du séminaire était de 4,3.

9. Chaque séance a été évaluée sur une échelle de 1 à 5 (voir le tableau ci-dessous). Pour

l’ensemble des séances, la note moyenne était de 4,4 en Arménie, de 4 au Kirghizistan et de

4,7 en République de Moldova.

9 De plus amples informations à ce sujet peuvent être consultées dans le rapport final sur les séminaires

de formation nationaux (Final Report on National Training Seminars) à l’adresse suivante :

https://unece.org/sites/default/files/2022-03/Final%20Report.pdf.

ECE/ENERGY/GE.6/2022/3

4 GE.22-11559

Tableau

Évaluation de la qualité des séances par les participants

Intitulé de la séance Arménie Kirghizistan Rép.de Moldova Moyenne

Introduction à la gestion de l’énergie et au système de

suivi, de notification et de vérification dans le secteur

du logement 4,3 3,8 4,7 4,3

Aspects institutionnels, organisationnels et légaux

de la gestion de l’énergie et du système de suivi,

de notification et de vérification aux niveaux mondial

et de l’Union européenne 4,5 4,2 4,7 4,5

Gestion de l’énergie au niveau municipal dans le cadre

du système national de suivi, de notification et de

vérification 4,2 4,0 4,7 4,3

Système d’information sur la gestion de l’énergie et

exemples d’outils relatifs au système de gestion de

l’énergie 4,3 4,0 4,7 4,3

Procédure d’audit énergétique des bâtiments et analyse

multicritères 4,6 4,0 4,7 4,4

Meilleures pratiques en matière de possibilités de

financement des mesures d’efficacité énergétique

des bâtiments 4,2 4,1 4,8 4,4

Moyenne 4,4 4,0 4,7 4,4

10. Les participants ont aussi évalué leur niveau de satisfaction globale (4,3) et l’équilibre

entre la théorie et la pratique (4,4). Ils ont réagi très positivement à la série de questions

portant sur l’amélioration de leur compréhension des sujets relatifs à l’efficacité énergétique

présentés au cours du séminaire, leur capacité à effectuer des tâches concrètes liées à

l’efficacité énergétique, et leur capacité à faire des exposés sur les sujets présentés, les notes

moyennes étant respectivement de 4, de 3,8 et de 3,9 pour l’Arménie, le Kirghizistan et la

République de Moldova.

11. Les participants aux séminaires ont dit qu’il était nécessaire d’organiser régulièrement

d’autres formations de ce type et de prendre davantage en considération les questions locales

liées aux différents aspects de l’efficacité énergétique et de la mise en application des

systèmes d’information sur la gestion de l’énergie au niveau national. Certains participants

estimaient qu’il fallait consacrer des formations de ce type à la gestion de l’énergie dans

l’industrie et à l’audit énergétique des installations industrielles, et y donner des exemples

pratiques sur la façon d’utiliser les dispositifs de mesure. De nombreux participants

souhaitaient des formations plus approfondies sur l’audit énergétique. Certains ont proposé

que les formations suivantes soient plus riches en vidéos et en présentations visuelles, ainsi

qu’en exemples et en informations sur les outils pratiques. La plupart des participants étaient

globalement très satisfaits. Ils pensaient que les formations étaient bien organisées et que les

informations fournies étaient bien structurées.

III. Étude d’impact sur la manière dont les États membres pourraient mieux utiliser et appliquer les meilleures pratiques et les directives élaborées par la CEE pour améliorer l’efficacité énergétique des bâtiments

12. Le projet dont il est ici question visait à susciter d’importants changements en

améliorant les connaissances des décideurs et des experts des États membres de la CEE sur

les normes d’efficacité énergétique, notamment en ce qui concerne les mécanismes

d’application de ces normes et les technologies à haute performance énergétique dans les

bâtiments. L’étude d’impact a permis d’analyser l’incidence dudit projet et la manière dont

les États membres pourraient mieux utiliser et appliquer, dans le cadre de leurs mesures

nationales ou infranationales visant à améliorer l’efficacité énergétique des bâtiments, les

meilleures pratiques et les directives élaborées par la CEE.

ECE/ENERGY/GE.6/2022/3

GE.22-11559 5

13. L’étude a déterminé l’incidence du projet selon les critères suivants : i) sa prise en

compte des questions de genre ; ii) sa pertinence globale, notamment en ce qui concerne sa

structure et les activités menées ; iii) la mesure dans laquelle les États membres ont appliqué

les recommandations et les meilleures pratiques, avec des exemples dans les pays concernés ;

iv) la mesure dans laquelle le projet a donné des orientations utiles pour la mise en application

des recommandations ; v) la mesure dans laquelle le projet a permis d’accroître la capacité

des parties prenantes à appliquer les recommandations ; vi) les difficultés que les États

membres devront encore surmonter pour donner suite aux recommandations ; vii) la mesure

dans laquelle les pays ont appliqué les recommandations dans une perspective transnationale

ou régionale.

14. L’étude d’impact a été réalisée à partir d’un questionnaire en ligne et d’entretiens

individuels. L’analyse des réponses a permis de tirer des conclusions utiles concernant la

pertinence du projet et son incidence, et de formuler des suggestions pour les futurs projets.

15. Les réponses au questionnaire ont permis de montrer que les activités exécutées dans

le cadre du projet promouvaient l’égalité des genres. Sur une échelle allant de 1 (très faible)

à 5 (très élevé), la note moyenne était de 3,7, ce qui indique que les personnes ayant répondu

au questionnaire ont constaté que le projet promouvait l’égalité des genres. La participation

des femmes aux ateliers est en outre un bon indicateur de l’importance accordée à l’égalité

des genres. L’atelier en ligne sur les normes relatives à l’efficacité énergétique des bâtiments

et leur mise en application dans la région de la CEE, qui s’est tenu le 9 avril 2021, a réuni

68 participants, dont 24 femmes (35 %). L’atelier sur les études régionales et nationales

relatives à l’analyse des écarts entre les objectifs d’efficacité énoncés dans les Orientations-

cadres pour l’élaboration de normes sur l’efficacité énergétique dans les bâtiments et

l’application des normes actuelles relatives à l’efficacité énergétique dans les bâtiments, qui

s’est tenu le 20 septembre 2021, a réuni 162 experts, dont 34 femmes (20 %). L’atelier sur

les meilleures pratiques à adopter pour gérer les questions d’efficacité énergétique des

bâtiments et leur mise en application dans les États membres de la CEE, qui s’est tenu le

11 mars 2022, a réuni 64 participants, dont 28 femmes (43 %). Ces chiffres montrent que le

niveau de participation des femmes à ces ateliers était inférieur à celui des hommes. Si l’on

tient compte du fait que les femmes sont minoritaires dans le secteur de l’énergie, cela n’est

pas surprenant. Si l’on prend aussi en considération la note moyenne élevée donnée en

réponse à la question portant sur l’égalité des genres, on est en mesure de conclure que le

projet a bien répondu aux critères en matière de promotion de l’égalité des genres.

Néanmoins, davantage d’efforts devraient être faits pour assurer un niveau égal de

participation des hommes et des femmes aux activités futures.

16. L’étude d’impact a montré que le projet avait permis d’atteindre les objectifs visés et

eu des effets positifs en ce qui concerne ses principaux aspects. Les personnes ayant répondu

à l’enquête ont indiqué que leur pays appliquait les recommandations et les meilleures

pratiques issues des études régionales et nationales. L’enquête a également révélé, dans

l’ensemble, une confiance modérée quant à la mise en application future des

recommandations et des meilleures pratiques, mais une certaine confiance dans le fait que

celles qui étaient déjà appliquées continueraient à l’être à l’avenir.

17. Les entretiens ont permis de constater que les États membres jugeaient très utiles les

recommandations et les meilleures pratiques énoncées par la CEE, et que les pays s’étaient

employés à les appliquer. Tous les pays étaient conscients de l’importance de l’efficacité

énergétique des bâtiments et de la nécessité d’appliquer sans attendre ces recommandations

et ces meilleures pratiques. Comme le montraient les résultats de l’enquête, les pays ont

indiqué qu’ils avaient pris plusieurs mesures au niveau national ou local en vue d’améliorer

les normes relatives à l’efficacité énergétique des bâtiments.

18. L’objectif principal des entretiens était d’obtenir des informations sur l’utilité des

recommandations pour les pays dans le domaine de l’efficacité énergétique des bâtiments.

Les personnes interrogées ont indiqué que les éléments les plus utiles étaient l’analyse des

écarts de l’étude régionale, les études nationales et les formations nationales. Les formations

portaient sur les mesures que les pays devaient prendre pour améliorer l’efficacité

énergétique des bâtiments. Elles permettaient d’acquérir des connaissances et des

compétences fondées sur l’expérience des pays les plus performants dans ce domaine, et elles

aidaient les autres pays à transmettre ces connaissances aux décideurs et aux responsables au

ECE/ENERGY/GE.6/2022/3

6 GE.22-11559

niveau national. En outre, on estimait que l’analyse des écarts effectuée dans le cadre de

l’étude régionale était un outil essentiel, qui confirmait qu’il était nécessaire de dynamiser la

transition dans le secteur de l’énergie et de placer sans attendre l’efficacité énergétique au

centre des efforts déployés aux niveaux national et local. Les échanges d’informations sur les

meilleures pratiques, notamment en matière de collecte de données, de suivi et de

vérification, d’audits énergétiques et de mise en œuvre de mesures concrètes, ont fourni des

orientations utiles aux participants au projet. Il s’agit là d’un atout, surtout à long terme. Il a

été établi que les connaissances transmises, les recommandations et les informations sur les

facteurs de réussite étaient des aspects essentiels du projet, très utiles aux États membres.

19. L’étude d’impact a permis d’examiner l’incidence réelle du projet sur les États

membres. Le projet s’étant achevé très récemment, on s’attendait à ce que ses bénéfices

concrets observables soient peu nombreux. Toutefois, il a été fait état dans les réponses de

plusieurs activités menées dans les pays en application des recommandations.

Dans l’ensemble, les réponses peuvent être classées en plusieurs grandes catégories :

a) Le projet a eu une incidence sur la mise en application des lois et des politiques

nationales, ce qui signifie que les pays ont soit révisé les lois existantes, soit adopté de

nouvelles lois sur l’efficacité énergétique des bâtiments. En outre, des documents normatifs

et des stratégies nationales en matière d’énergie ont été actualisés dans les pays concernés

par le projet. Par exemple, l’Albanie a adopté plusieurs règlements d’application se

rapportant à la loi de 2016 sur la performance énergétique des bâtiments, notamment

concernant les méthodes de calcul et de fixation des prescriptions minimales en matière de

performance énergétique et la certification des bâtiments. La Géorgie s’emploie à harmoniser

sa législation nationale avec les directives de l’Union européenne. Le Kirghizistan a travaillé

sur son cadre juridique et commencé à appliquer des mesures visant à satisfaire aux

prescriptions fixées dans les lois pertinentes. La République de Moldova a commencé à

mettre à jour sa législation sur l’efficacité énergétique en se fondant sur les directives de

l’Union européenne ;

b) Le projet a également eu une incidence sur l’actualisation des normes et des

règles nationales. Les pays ont mis à jour plusieurs documents normatifs et révisé les normes

nationales portant sur l’efficacité énergétique des bâtiments. Plus précisément, les études

menées aux niveaux régional et national ont appelé l’attention des décideurs sur les écarts

entre leur pays et les pays voisins et cela les a incités à actualiser leurs normes en tenant

compte du niveau d’efficacité de leurs bâtiments existants et nouveaux. Par exemple,

l’Arménie a élaboré des normes relatives à l’efficacité énergétique fondées sur les

modifications apportées aux normes de l’Union européenne, et ces normes ont été adoptées

dans leur intégralité en janvier 2022. Elles seront obligatoires pour toutes les entités

participant à des travaux de planification et de construction ;

c) Le projet a permis des améliorations en matière de mesure. Après avoir

participé au projet, certains pays ont amélioré leurs activités de collecte de données et de

mesure et commencé à mesurer la performance énergétique des bâtiments conformément aux

normes de l’Organisation internationale de normalisation (ISO). L’Arménie, par exemple, a

appliqué les normes ISO. Plus précisément, ces normes ont été traduites en arménien et sont

devenues des normes nationales ;

d) Les pays participant au projet ont augmenté le nombre de leurs audits

énergétiques. Un plus grand nombre d’audits ont effectivement été réalisés après l’exécution

du projet. Par exemple, le Kirghizistan a certifié 20 inspecteurs chargés d’évaluer l’efficacité

énergétique des bâtiments et de délivrer des homologations. Les entreprises de construction

sont désormais tenues de recourir à leurs services pour homologuer les bâtiments neufs.

De nombreux progrès ont été réalisés dans tous les pays, mais il reste encore beaucoup à faire

et les systèmes nationaux intègrent lentement les changements ;

e) Le projet a eu une incidence sur la mise en application des systèmes

d’information sur la gestion de l’énergie. Certains experts ont indiqué que leur pays avait mis

en place de tels systèmes ;

f) Le projet a favorisé la sensibilisation. Les séminaires de formation organisés

dans les pays sélectionnés ont contribué à accroître la sensibilisation à la question de

l’efficacité énergétique des bâtiments. En outre, des campagnes de sensibilisation et des

ECE/ENERGY/GE.6/2022/3

GE.22-11559 7

activités de promotion ont ouvert les yeux du public sur l’importance de l’efficacité

énergétique. Ces progrès mènent à long terme à une augmentation des investissements dans

la modernisation ou à l’imposition de normes plus strictes à la construction des bâtiments.

Pour favoriser les économies d’énergie à long terme, il est essentiel d’expliquer aux

propriétaires les avantages qu’ils peuvent retirer d’investissements dans l’amélioration de

l’efficacité énergétique du bâti existant ou de la construction de nouveaux bâtiments

conformes aux normes de haute performance.

20. L’étude d’impact a montré que les pays s’efforçaient de faire comprendre

l’importance des économies d’énergie et de l’efficacité énergétique des bâtiments.

D’une part, les décideurs et les responsables ont été informés de la nécessité d’améliorer le

cadre législatif et de mettre fréquemment à jour les normes et les règles. D’autre part, la

nécessité d’investir dans l’efficacité énergétique des bâtiments existants en les modernisant

a également été mise en avant. L’Arménie, par exemple, mène des activités de sensibilisation

en vantant à grande échelle l’efficacité énergétique des bâtiments et en faisant appel à des

influenceurs et à d’autres personnalités et faiseurs d’opinion célèbres. En outre, ce pays a

produit des films et déployé des banderoles pour faire comprendre l’intérêt qu’il y avait à

investir dans l’efficacité énergétique des bâtiments, par exemple en mettant en avant la

moindre consommation d’énergie et les économies qui en découlent. Au Kirghizistan, le

projet a notamment permis de favoriser la promotion de l’efficacité énergétique des bâtiments

et le partage de connaissances à ce sujet.

21. L’étude a également permis de constater que les pays s’efforçaient de faire appliquer

les codes du bâtiment et de les mettre à jour. Des améliorations sont toutefois possibles,

notamment en ce qui concerne la fréquence de ces mises à jour. En Arménie, par exemple,

les dispositions portant sur la mesure de l’utilisation de l’énergie ne prennent pas en

considération le refroidissement ou l’eau chaude. La prise en compte du refroidissement et

de l’éclairage dans les codes du bâtiment est aujourd’hui au programme. D’autres pays,

comme la Macédoine du Nord et la République de Moldova, ont indiqué que des travaux sur

l’homologation des bâtiments étaient en cours mais que le processus n’était pas encore mené

à terme. Les entretiens ont fait ressortir que l’amélioration de l’efficacité énergétique des

bâtiments prenait du temps et que les efforts devaient être constants si l’on voulait obtenir

des gains en matière d’efficacité.

22. L’étude d’impact a permis d’examiner comment le projet avait aidé les pays à

appliquer les recommandations et les meilleures pratiques énoncées. L’un des aspects de cette

question est que les ateliers et les formations organisés dans le cadre du projet ont apporté

bien davantage qu’une simple formation technique. Ils ont constitué une plateforme propice

à la communication entre experts. Les personnes interrogées ont souligné que le projet avait

servi à échanger des informations et à débattre entre experts, surtout après la pandémie de

maladie à coronavirus 2019 (COVID-19). La participation d’experts compétents a permis de

mener des débats approfondis.

23. L’étude d’impact avait également pour objectif de formuler des recommandations

concernant les projets futurs et la manière dont les États membres pourraient appliquer plus

efficacement les recommandations et les meilleures pratiques énoncées. L’analyse des

entretiens a montré que les parties prenantes estimaient que les projets devraient porter plus

particulièrement sur les activités de sensibilisation et d’information destinées au grand public

afin de faire mieux comprendre l’importance de l’efficacité énergétique des bâtiments.

En outre, il faudrait donner davantage d’exemples, notamment sur les meilleures pratiques

en matière d’exécution de projets et de partenariats public-privé et, d’un point de vue

pratique, sur l’application de mesures dans ce domaine et le calcul des économies d’énergie.

Une autre recommandation portait sur le fait que les projets devaient être assortis

d’informations détaillées sur la mesure de la consommation d’énergie conformément aux

normes ISO. Il a été proposé que les données relatives aux conditions météorologiques soient

utilisées pour effectuer des calculs sur une base mensuelle et horaire. Il ressort de l’analyse

des entretiens que les futurs projets devraient s’appliquer à la fois au niveau régional et au

niveau national. La perspective régionale est efficace pour tirer des enseignements de

l’expérience et pour formuler les meilleures pratiques, mais les pays sont aussi intéressés par

des recommandations adaptées à leur contexte et à leurs infrastructures au niveau national.

  • Groupe d’experts de l’efficacité énergétique
  • Rapport sur le renforcement des capacités nationales d’élaboration et d’application de normes relatives à l’efficacité énergétique des bâtiments dans la région de la Commission économique pour l’Europe
    • Note du secrétariat
  • I. Introduction
  • II. Formations sur les normes relatives aux bâtiments à haute performance énergétique dans les pays sélectionnés pour le projet
  • Tableau Évaluation de la qualité des séances par les participants
    • III. Étude d’impact sur la manière dont les États membres pourraient mieux utiliser et appliquer les meilleures pratiques et les directives élaborées par la CEE pour améliorer l’efficacité énergétique des bâtiments
Russian

GE.22-11559 (R) 290722 090822

Европейская экономическая комиссия

Комитет по устойчивой энергетике

Группа экспертов по энергоэффективности

Девятая сессия

Женева, 3–4 октября 2022 года

Пункт 7 предварительной повестки дня

Повышение энергоэффективности зданий

Доклад об укреплении национального потенциала в области разработки и применения стандартов энергоэффективности зданий в регионе Европейской экономической комиссии Организации Объединенных Наций

Записка секретариата

Резюме

В период с июля 2020 года по март 2022 года Европейская экономическая

комиссия Организации Объединенных Наций осуществляла проект «Укрепление

национального потенциала в области разработки и применения стандартов

энергоэффективности зданий в регионе ЕЭК ООН».

Группа экспертов по энергоэффективности на своей восьмой сессии

(20–21 сентября 2021 года) поручила секретариату (ECE/ENERGY/GE.6/2021/2)

представить на девятой сессии Группы экспертов доклад о результатах осуществления

проекта.

Настоящий документ был подготовлен во исполнение этого поручения. В нем

содержатся результаты оценок учебных семинаров и анализа воздействия

деятельности по проекту на повышение энергоэффективности зданий в странах ⸺

участницах проекта.

Организация Объединенных Наций ECE/ENERGY/GE.6/2022/3

Экономический

и Социальный Совет

Distr.: General

22 July 2022

Russian

Original: English

ECE/ENERGY/GE.6/2022/3

2 GE.22-11559

I. Введение

1. Проект «Укрепление национального потенциала в области разработки и

применения стандартов энергоэффективности зданий в регионе ЕЭК ООН» опирается

на результаты предыдущей деятельности Европейской экономической комиссии

Организации Объединенных Наций (ЕЭК) в области стандартов энергоэффективности

зданий и направлен на укрепление потенциала государств ⸺ членов ЕЭК по

разработке и применению таких стандартов.

2. Проект осуществлялся с июля 2020 года по март 2022 года. В рамках проекта

были осуществлены все предложенные виды деятельности, а именно:

a) проведено исследование, посвященное анализу расхождений между

целевыми показателями, установленными в рамочных руководящих указаниях по

стандартам энергоэффективности зданий (ECE/ENERGY/GE.6/2020/4) и

действующими стандартами энергоэффективности, а также их применению в странах

Юго-Восточной, Восточной Европы, Кавказа, Центральной Азии и Российской

Федерации1;

b) выводы и рекомендации этого регионального исследования прошли

обсуждение на состоявшемся 9 апреля 2021 года рабочем совещании

заинтересованных сторон из энергетического и жилищного секторов с целью

подтверждения итогов анализа расхождений2;

c) также были подготовлены три национальных исследования

(охватывающие Армению, Кыргызстан и Республику Молдова) с более подробным

анализом расхождений3;

d) на состоявшемся 20 сентября 2021 года рабочем совещании для

заинтересованных сторон из энергетического и жилищного секторов состоялось

обсуждение рекомендаций, касающиеся конкретных стран, охваченных этими

исследованиями4;

e) в рамках проекта было создано пространство для сотрудничества

экспертов по энергоэффективности зданий в регионе ЕЭК и обновлена база данных

сети экспертов по энергоэффективности зданий государственного и частного

секторов5;

f) в октябре и ноябре 2021 года в Армении и Кыргызстане были

организованы национальные учебные семинары по высокоэффективным стандартам

энергоэффективности зданий6. Национальный учебный семинар для Республики

Молдова был проведен 20–21 января 2022 года7;

g) эффект деятельности по проекту был продемонстрирован в исследовании

воздействия по вопросу о том, как страны, участвующие в проекте, используют и

применяют передовой опыт, а также выполняют рекомендации, вносимые в рамках

деятельности ЕЭК, в принимаемых ими на национальном и/или субнациональном

уровнях мерах по обеспечению энергоэффективности зданий8.

1 См.: https://unece.org/sites/default/files/2021-06/Study_on_Gap_Analysis_07.06.2021.pdf.

2 См.: https://unece.org/sustainable-energy/events/online-workshop-energy-efficiency-standards-

buildings-and-their.

3 См.: https://unece.org/sustainable-energy/regional-advisory-services/gap-analysis-and-national-

studies.

4 См.: https://unece.org/sustainable-energy/events/workshop-regional-and-national-studies-gap-

analysis-between-performance.

5 См.: https://sedwiki.unece.org/display/SED/EEEB_db_Home.

6 См.: https://unece.org/sustainable-energy/regional-advisory-services/national-training-seminars-high-

performance-energy.

7 См.: https://unece.org/info/events/event/364307.

8 См.: https://unece.org/sustainable-energy/regional-advisory-services/impact-study.

ECE/ENERGY/GE.6/2022/3

GE.22-11559 3

II. Учебные семинары по высокорезультативным стандартам энергоэффективности зданий в отдельных странах ⸺ участницах проекта

3. Одним из направлений деятельности по проекту явилось проведение

национальных учебных семинаров по высокоэффективным стандартам

энергоэффективности зданий в отдельных государствах ⸺ членах ЕЭК. Учебные

семинары были организованы в Ереване, Армения, 25–26 октября 2021 года; Бишкеке,

Кыргызстан, 29–30 ноября 2021 года; и Кишиневе, Республика Молдова, 20–21 января

2022 года.

4. Проведение учебных семинаров преследовали следующие цели: i) улучшение

понимания по вопросам управления энергопотреблением, а также мониторинга,

отражения в отчетности и проверки (МОП) в секторе зданий; ii) улучшение понимания

институциональных, организационных и правовых аспектов управления

энергопотреблением и МОП на глобальном уровне и на уровне Европейского cоюза;

iii) консолидация потенциала для организации управления энергопотреблением на

муниципальном уровне как части национальной системы МОП; iv) распространение

знаний об использовании существующей информационной системы управления

энергопотреблением или организация ее разработки; v) обеспечение понимания

процесса аудита энергопотребления и целесообразности использования средств

многокритериального анализа; и vi) укрепление потенциала для использования и

понимания возможностей для финансирования мер по повышению

энергоэффективности зданий.

5. Отбор участников учебных семинаров проводился на основе открытого

приглашения, направленного правительственными учреждениями и страновыми

отделениями Программы развития ООН (ПРООН) в трех государствах ⸺ членах ЕЭК.

Согласно результатам оценки учебных семинаров9 в Армении большинство

участников представляли государственные учреждения; в Кыргызстане ⸺ частные

компании и университеты; а в Республике Молдова ⸺ в основном государственные

учреждения и частные компаний. В общей сложности в учебных семинарах,

состоявшихся в онлайновом и очном форматах, приняли участие 180 человек. Из их

числа в Армении прошли обучение 46 участников, в Кыргызстане ⸺ 44 участника,

в Республике Молдова ⸺ 90 участников.

6. Три учебных семинара прошли в течение двух дней. На каждом учебном

семинаре рассмотрение тем проходило в рамках шести учебных занятий. Занятия

первого дня были посвящены процедурам аудита энергопотребления,

информационным системам управления энергопотреблением и управлению

энергопотреблением на муниципальном уровне. Занятия второго дня были посвящены

институциональным, организационным и правовым аспектам управления

энергопотреблением, а также мониторингу, отражению в отчетности и проверке мер

по обеспечению энергоэффективности, экономической оценке и возможностям

финансирования мер по обеспечению энергоэффективности зданий. Каждый учебный

семинар включал в себя аудиторные лекции (с очным участием и онлайн-

подключением), а также выполнение расчетных задач в аудитории с использованием

различных инструментов.

7. Основным итогом учебного семинара является приобретение его участниками

знаний и опыта по высокоэффективным стандартам энергоэффективности зданий.

В конце каждого учебного семинара участников просили заполнить вопросник и

поделиться своими мнениями о проведенном на национальном уровне курсе обучения.

Слушателей просили дать свою оценку организации, качеству обучения, а также

самостоятельно оценить свое понимания тем, которые были затронуты в ходе

обучения.

9 Дополнительную информацию см. в Final Report on National Training Seminars:

https://unece.org/sites/default/files/2022-03/Final%20Report.pdf.

ECE/ENERGY/GE.6/2022/3

4 GE.22-11559

8. Организация обучения была оценена следующим образом: 4,6 баллов из 5,0

в Армении, 4,2 балла из 5,0 в Кыргызстане и 4,8 балла из 5,0 в Республике Молдова.

Место проведения учебного семинара, количество и продолжительность перерывов,

расписание лекций были оценены в этих странах как 4,7, 4,6 и 4,8 баллов

из 5,0 соответственно. Средняя оценка общей продолжительности учебного семинара

составила 4,3 балла из 5,0.

9. Участники также дали (по шкале от 1,0 до 5,0) разную бальную оценку каждому

учебному занятию, итоги которой приведены в таблице ниже. Общий средний балл по

всем учебным занятиям в целом составил в Армении ⸺ 4,4, Кыргызстане ⸺ 4,0

и Республике Молдова ⸺ 4,7 балла.

Таблица

Качество учебных занятий по оценке их участников

Название учебного занятия: Армения Кыргызстан

Республика

Молдова

средний

балл

Введение в управление энергопотреблением,

а также мониторинг, отражение в отчетности

и проверка в секторе зданий 4,3 3,8 4,7 4,3

Институциональные, организационные и

правовые аспекты управления

энергопотреблением, а также мониторинг,

отражение в отчетности и проверка на

глобальном уровне и уровне ЕС 4,5 4,2 4,7 4,5

Управление энергопотреблением на

муниципальном уровне как часть

национальной системы мониторинга,

отражения в отчетности и проверки 4,2 4,0 4,7 4,3

Информационная система управления

энергопотреблением и примеры инструментов,

используемых в системе управления

энергопотреблением 4,3 4,0 4,7 4,3

Процедура аудита энергопотребления зданий

и многокритериальный анализ 4,6 4,0 4,7 4,4

Передовой опыт по выявлению возможностей

финансирования мер по повышению

энергоэффективности зданий 4,2 4,1 4,8 4,4

Средний балл 4,4 4,0 4,7 4,4

10. Участники дали аналогичную бальную оценку своей общей удовлетворенности

учебным семинаром (4,3), а также соотношению между теоретической и практической

частью учебного семинара (4,4). В рамках набора вопросов, относящихся к улучшению

понимания участниками тем в области энергоэффективности, рассмотренных в ходе

учебного семинара, укреплению их потенциала для выполнения конкретных задач,

связанных с энергоэффективностью, и улучшению их потенциала, необходимого для

изложения любой конкретной рассмотренной темы, были вынесены весьма

удовлетворительные оценки со средними баллами, равными 4,0, 3,8 и 3,9 для Армении,

Кыргызстана и Республики Молдова соответственно.

11. Участники учебных семинаров отметили необходимость проводить большее

количество таких мероприятий на регулярной основе и рассматривать в ходе их

проведения большее количество возникающих на местном уровне вопросов,

относящихся к разным областям обеспечения энергоэффективности, внедрению

информационных систем управления энергопотреблением в странах. Некоторые

участники отметили необходимость проведения подобных учебных семинаров,

ориентированных на управление энергопотреблением в промышленности и аудит

энергопотреблением на промышленных объектах с рассмотрением примеров из

практики использования приборов учета. Многие участники отметили, что хотели бы

участвовать в более интенсивных учебных семинарах по энергетическому аудиту.

ECE/ENERGY/GE.6/2022/3

GE.22-11559 5

Некоторые участники предложили использовать больше количество видеороликов и

наглядных пособий при проведении будущих учебных семинаров, а также

рассматривать больше примеров и информации об инструментах, которые они могут

использовать на практике. В целом большинство участников остались весьма

удовлетворены и положительно оценили хорошую организацию учебных семинаров,

а также правильную подачу информации.

III. Исследование воздействия по вопросу о том, как государства-члены могли бы лучше использовать и применять передовой опыт и руководящие принципы для повышения энергоэффективности зданий

12. Проект имел своей целью достижение значительных изменений путем

повышения знаний на уровне директивных органов и экспертов в государствах ⸺

членах ЕЭК о стандартах энергоэффективности, в том числе о механизмах

правоприменения и энергоэффективных технологиях для зданий. Было проведено

исследование его воздействия с анализом полученного от проекта эффекта и вопроса

о том, как государства-члены могли бы лучше использовать и применять передовой

опыт и руководящие принципы, разработанные в рамках деятельности ЕЭК, в своих

национальных и/или субнациональных мерах по решению вопросов обеспечения

энергоэффективности зданий.

13. В ходе исследования изучалось влияние проекта с точки зрения: i) учета

гендерных аспектов; ii) общей востребованности, в том числе его концепции и

конкретных видов деятельности; iii) степени, в которой государства-члены выполнили

его рекомендации и применили передовой опыт на примерах, представленных

странами-респондентами; iv) степени, в которой в рамках проекта были

предоставлены руководящие указания по выполнению рекомендаций; v) степени,

в которой проект позволил увеличить потенциал заинтересованных сторон,

необходимый для выполнения рекомендаций; vi) нерешенных проблем государств-

членов по выполнению рекомендаций; и vii) степени выполнения странами

рекомендаций, затрагивающих межстрановые или региональные аспекты.

14. Для целей исследования проведена работа с онлайновым вопросником и были

организованы очные собеседования. Анализ полученных ответов позволил извлечь

ценную информацию, касающуюся востребованности проекта, полученного от него

эффекта и предложений для будущих проектов.

15. Результаты работы с вопросником показали, что предусмотренные проектом

виды деятельности способствовали пропаганде гендерного равенства. По шкале

от 1,0 (предельно низкий балл) до 5,0 (предельно высокий балл) соответствующий

средний балл составил 3,7; из этого показателя следует, что респонденты уверены в

проведении работы по пропаганде гендерного равенства. Кроме того, хорошим

показателем поддержки в достижении гендерного равенства служит участие женщин

в рабочих совещаниях. В состоявшемся 9 апреля 2021 года в онлайновом режиме

рабочем совещании по стандартам энергоэффективности зданий и их применению в

регионе ЕЭК ООН, приняли участие 68 человек, из них 24 женщины (35 %). В рабочем

совещании по региональным и национальным исследованиям, посвященным

анализу расхождений между целевыми показателями рамочных руководящих

указаний по стандартам энергоэффективности зданий и применяемыми

действующими стандартами энергоэффективности зданий, которое состоялось

20 сентября 2021 года, приняли участие 162 эксперта, в том числе 34 женщины (20 %).

В проведенном 11 марта 2022 года рабочем совещании по передовому опыту решения

вопросов обеспечения энергоэффективности зданий и его применению в

государствах ⸺ членах ЕЭК приняли участие 64 человека, из них 28 женщин (43 %).

Эти цифровые данные показывают, что участие женщин в рабочих совещаниях было

ниже, чем мужчин. Вместе с тем они не вызывают удивления с учетом того, что в

секторе энергетики на долю женщин приходится меньшинство работников. Наряду с

высоким средним баллом в ответах на вопросы, касающиеся гендерного равенства, эти

ECE/ENERGY/GE.6/2022/3

6 GE.22-11559

цифры позволяют сделать вывод о том, что проект показал хорошие результаты в

плане поддержки гендерного равенства. Тем не менее для обеспечения равных

возможностей для участия мужчин и женщин в будущих мероприятиях необходимо

приложить дополнительные усилия.

16. Анализ полученного эффекта показал, что проект сыграл свою роль в

достижении поставленных перед ним целей и оказал ценное воздействие по всем его

ключевым аспектам. Участвовавшие в опросе респонденты указали, что в их странах

выполняются рекомендации и применяется передовой опыт, предложенные в ходе

региональных и национальных исследований. Опрос также показал, что в целом

респонденты испытывают умеренную уверенность в выполнении рекомендаций и

применении передового опыта в будущем. Респонденты проявили свою полную

уверенность в том, что выполненные рекомендации и нашедший применение

передовой опыт в будущем будут играть позитивную роль.

17. Собеседования показали, что государства-члены считают разработанные ЕЭК

рекомендации и накопленный передовой опыт весьма полезными и что страны

прилагают усилия по их применению. Все страны признают важность

энергоэффективности зданий и настоятельную необходимость выполнения

рекомендаций, а также применения передового опыта. Результаты опроса

коррелируют с указанными странами мерами, которые были реализованы на

национальном или местном уровнях для улучшения стандартов энергоэффективности

зданий.

18. Основное внимание в ходе собеседований было уделено выяснению того, каким

образом страны руководствуются рекомендациями по тематике энергоэффективности

зданий. Опрошенные указали, что ключевые аспекты предлагаемых рекомендаций

были рассмотрены в региональном исследовании по анализу расхождений,

национальных исследованиях и в ходе национальных учебных семинаров. На этих

семинарах рассматривались шаги, которые странам необходимо предпринять для

повышения энергоэффективности зданий. На них была предоставлена возможность

почерпнуть знания и опыт, накопленные в странах с более высоким уровнем развития

в области энергоэффективности зданий, а также помочь другим странам донести эти

знания до лиц, принимающих решения и директивных органов своих стран. Кроме

того, региональное исследование анализа расхождений было упомянуто как

важнейший инструмент, который продемонстрировал необходимость ускорения

прогресса в секторе энергопотребления в переходный период и показал, что

энергоэффективность должна находиться в центре усилий, предпринимаемых на

национальном и местном уровнях. Обмен передовым опытом, в том числе по вопросам

сбора данных, мониторинга и проверки, проведения энергетического аудита и

реализации конкретных мер, явился для участвовавших в проекте заинтересованных

сторон источником ценной информации, которой следует руководствоваться. Из этого

следует, что применяемый подход вносит позитивный вклад, особенно в долгосрочной

перспективе. Ключевым направлением деятельности по проекту стал обмен знаниями

и рекомендациями, а также информацией о факторах успеха, которыми государства-

члены могут воспользоваться в качестве руководства в своей работе.

19. В ходе анализа воздействия рассматривались ощутимые выгоды проекта для

государств-членов. Поскольку проект был завершен в самое последнее время,

ожидалось, что конкретные заметные выгоды будут ограниченными. Тем не менее

респонденты указали на ряд проведенных в их странах видов деятельности по

использованию и выполнению рекомендаций. В целом ответы можно разделить на

несколько основных категорий:

a) был получен эффект на уровне национальных законов и политики,

т. е. страны либо пересмотрели действующие законодательные акты, либо приняли

новые законы по вопросу об энергоэффективности зданий. Кроме того, в странах ⸺

участницах проекта были обновлены нормативные документы и национальные

энергетические стратегии. Например, Албания приняла несколько подзаконных актов

по выполнению Закона об энергоэффективности зданий 2016 года, в том числе по

методам расчета и установления минимальных требований к энергоэффективности и

сертификации зданий. Грузия работает над гармонизацией своего национального

ECE/ENERGY/GE.6/2022/3

GE.22-11559 7

законодательства с директивами Европейского союза (ЕС). Кыргызстан провел работу

над нормативно-правовой базой и начал осуществлять практические шаги по

выполнению требований, установленных в соответствующих законах. Республика

Молдова приступила к обновлению своих посвященных энергоэффективности

законодательных актов на основе директив ЕС;

b) кроме того, получен эффект в плане обновления национальных норм и

стандартов. Страны обновили ряд нормативных документов и пересмотрели

национальные стандарты по энергоэффективности зданий. В частности, региональные

и национальные исследования привлекли внимание директивных органов к

расхождениям с соответствующими документами соседних стран, мотивируя их

обновить стандарты, чтобы отразить уровень эффективности как для существующих,

так и для новых зданий в этих странах. Так, например, Армения разработала свои

стандарты энергоэффективности с учетом стандартов ЕС, и эти стандарты были

полностью приняты к январю 2022 года. Эти стандарты будут обязательными для всех

организаций, занимающихся планированием и строительными работами;

c) проект привел к улучшению измерений. По итогам участия в проекте

страны активизировали свои усилия по сбору данных и проведению измерений,

а также предприняли конкретные шаги по измерению показателей энергопотребления

зданий в соответствии со стандартами Международной организации по

стандартизации (ИСО). Например, стандарты ИСО были приняты Арменией.

В частности, стандарты ИСО были переведены на язык страны и зарегистрированы как

национальные стандарты;

d) в странах ⸺ участницах проекта увеличилось количество проводимых

энергетических аудитов. По итогам реализации проекта количество аудитов

увеличилось. Например, Кыргызстан сертифицировал 20 энергетических аудиторов,

осуществляющих оценку энергоэффективности зданий и выдачу сертификатов.

В настоящее время строительные компании обязаны привлекать энергетических

аудиторов для сертификации вновь возведенных зданий. Несмотря на то, что во всех

странах был достигнут большой прогресс, данный процесс имеет длительный

характер, а национальные системы медленно воспринимают изменения;

e) проект сказался и на внедрении информационных систем в области

управления энергопотреблением. Некоторые эксперты сообщили, что в их странах

внедрены информационные системы управления энергопотреблением;

f) проект внес вклад в повышение осведомленности. Проведенные в

отдельных странах учебные семинары позволили повысили осведомленность по теме

энергоэффективности зданий. Кроме того, в рамках кампаний по повышению

осведомленности и маркетинговых мероприятий общественность была

проинформирована о важности энергоэффективности. В долгосрочной перспективе

эти усилия находят выражение в увеличении инвестиций в капитальный ремонт или

повышении уровня стандартов для вновь возводимых зданий. Информирование

домовладельцев о преимуществах инвестирования в энергоэффективную

модернизацию существующих зданий или строительство новых зданий в соответствии

с высокоэффективными стандартами имеет решающее значение для экономии энергии

в долгосрочной перспективе.

20. Исследование воздействия показало, что в странах предпринимаются усилия по

повышению осведомленности о важности энергосбережения и энергоэффективности

зданий. С одной стороны, осуществляется информирование лиц, принимающих

решения, и директивных органов о необходимости совершенствования

законодательной базы и регулярного обновления норм и стандартов. С другой

стороны, пропагандируется необходимость инвестирования в энергоэффективность

существующих зданий путем их модернизации. Например, деятельность по

повышению осведомленности в Армении осуществляется путем проведения широких

кампаний по повышению энергоэффективности зданий, а также привлечения к ним

лиц, пользующихся влиянием и авторитетом, и лидеров общественного мнения. Кроме

того, посредством использования фильмов и баннеров Армения предпринимает

активные усилия по информированию о преимуществах инвестирования в

ECE/ENERGY/GE.6/2022/3

8 GE.22-11559

энергоэффективность зданий, в частности основной акцент в этой связи сделан на

информировании о выражаемой в процентных долях возможной экономии энергии и

соответствующей экономии затрат. К результатам осуществления проекта в

Кыргызстане можно отнести активный обмен знаниями и пропаганду важности

энергоэффективности зданий.

21. Исследование также показало, что страны занимаются принятием и

обновлением строительных норм и правил. Тем не менее еще не исчерпаны все

возможности для улучшения положения дел, особенно в силу недостаточно

регулярного проведения обновления строительных норм и правил. Например,

предусмотренные строительными нормами и правилами Армении измерения

энергопотребления не распространяются на кондиционирование воздуха или

подаваемую горячую воду. Вместе с тем процесс в направлении включения в

строительные нормы и правила кондиционирования и освещения уже начат. Кроме

того, другие страны, в частности Северная Македония и Республика Молдова, указали,

что работа по сертификации зданий продолжается, но этот процесс еще не завершен.

В ходе собеседований отмечалось, что повышение энергоэффективности зданий

представляет собой длительный процесс и что для обеспечения повышения

эффективности необходимы постоянные усилия.

22. В исследовании воздействия рассмотрен вопрос о том, как именно проект помог

странам выполнять рекомендации и применять передовой опыт. Один из аспектов

заключается в том то, что в рамках рабочих совещаний и учебных семинаров проект

принес пользу не только в плане технической подготовки участников. Проект

позволил создать платформу для общения между экспертами. Участники

собеседования подчеркнули полезность предоставленной в рамках проекта

возможности для экспертов обмениваться информацией и участвовать в дискуссиях,

особенно после завершения пандемии COVID-19. Участие компетентных экспертов в

проведенных по линии проекта мероприятиях способствовало проведению дискуссий

на соответствующем уровне детализации.

23. Еще одной целью исследования воздействия являлась подготовка

рекомендаций для будущих проектов и определение способов для более эффективного

выполнения государствами-членами рекомендаций, а также применения передового

опыта. Анализ итогов собеседований показал, что, по мнению заинтересованных

сторон, проекты должны быть направлены на повышение осведомленности и

информирование широкой общественности о важности обеспечения

энергоэффективности зданий. Кроме того, следует приводить больше примеров,

в частности примеров передового опыта, касающихся реализации проектов,

партнерств между государственным и частным секторами, а также примеров из

практики по проведению мероприятий и расчетам показателей энергосбережения. Еще

одна рекомендация касается подробного информирования в рамках проекта об

измерениях энергопотребления по стандартам ИСО. Респонденты предложили

включить в ежемесячные и почасовые расчеты климатические данные. Анализ

показал, что респонденты предлагают как региональную направленность будущих

проектов, так и направленность на конкретную страну. Хотя региональная

направленность полезна для изучения накопленного опыта и передовых практик,

страны заинтересованы в рекомендациях, в которых бы учитывалась специфика их

национальных условий и инфраструктуры.

  • Группа экспертов по энергоэффективности
  • Доклад об укреплении национального потенциала в области разработки и применения стандартов энергоэффективности зданий в регионе Европейской экономической комиссии Организации Объединенных Наций
    • Записка секретариата
  • I. Введение
  • II. Учебные семинары по высокорезультативным стандартам энергоэффективности зданий в отдельных странах ⸺ участницах проекта
    • Таблица Качество учебных занятий по оценке их участников
  • III. Исследование воздействия по вопросу о том, как государства-члены могли бы лучше использовать и применять передовой опыт и руководящие принципы для повышения энергоэффективности зданий