Skip to main content

Azerbaijan

EVAL_Trade_UNDA2023W_MR_2024

Global Initiative towards post-Covid-19 resurgence of the MSME sector (UNDA project 2023W) Period of Review: 2020 - 2023   Date of Evaluation Report: 30/04/2024 

Languages and translations
English

MANAGEMENT RESPONSE

Evaluation Title: Global Initiative towards post-Covid-19 resurgence of the MSME sector (UNDA project 2023W)

Period of Review: 2020 - 2023

Date of Evaluation Report:

30/04/2024

Approved by:

Chief, PMU Nicolas Dath-Baron

Date Signature 05/06/2024

Cleared by:

Director of Divisions: Elisabeth Türk, Economic Cooperation and Trade Division Dario Liguti, Sustainable Energy Division

18/07/2024

Prepared by:

UNECE Project Managers: Hana Daoudi Oleg Dzioubinski

05/06/2024

1

2

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

Recommendation 1:

1. (i) UNCTAD should further leverage the experience gained through the project to map out how components of the Entrepreneurship Policy Framework relate to and can be best positioned to support MSMEs in their recovery from different types of crises and (ii) DESA and the Regional Commissions should add their analysis of how their work can contribute towards the objective.

2. UNCTAD, DESA and the Regional Commissions could build on the project to map their areas of intervention in support of the MSME sector, as well as capacities and knowledge on the implementation of the EPF components, including in a crisis context, and opportunities for broadening the uptake of EPF components at the regional level based on national needs. UNCTAD, DESA and the Regional

Accepted Through existing and future UNDA projects and subject to the availability of funds and the potential for synergies withs UNCTAD, DESA and the remaining regional commissions, UNECE will focus its actions on three key areas:

Integration of evidence-based sectoral insights, focused on the impact of trade disruptions and fluctuations in national, regional and global sustainable development for guiding the development of resilience strategies for MSMEs within the EPF.

Capacity-building: UNECE can help MSMEs implement the strategies outlined in the EPF through training on new technologies, sustainable trade practices to mitigate the effects of crises and capitalize on new market opportunities as they arise.

Strengthening broad-based networks and partnerships, which bring together national and local governments; UNECE sustainable trade expert communities, enterprise

UNECE Economic Cooperation and Trade Division and Sustainable Energy Division

2 years Yes

3

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

Commissions could further consider identifying areas of joint interventions that would trigger complementarities and synergies between the agencies. This could involve collaborating on the development of capacities of all national and sub-national actors, i.e. Governments, MSMEs, other partners (e.g. Chamber of Commerce, Business Incubators, etc.) in line with United Nations Sustainable Development Cooperation Frameworks (SDCF) in respective countries; strengthening or contributing to facilitate access to finance for the MSMEs (including seed money, grants, access to credit, etc.); improving the coordination of MSMEs related policies across ministries; increasing interventions at the local level, such as by supporting NGOs or MSMEs outside of the main cities. The application of a human rights-based approach, gender responsiveness and

support organizations working with UNECE and MSMEs to support the practical application of the EPF.

4

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

inclusion of other vulnerable groups (LNOB) should be ensured.

Recommendation 2:

UNCTAD should continue building on the momentum generated by the project to continue fostering knowledge exchanges and promoting the Entrepreneurship Policy Framework.

UNCTAD, DESA and the Regional Commissions should identify means to more meaningfully continue to share good practices and lessons learned on the demand-driven support they provide to the MSME sector, including on areas such as green/circular economy; innovation policies and ecosystems; MSME formalization and strengthening MSME capacities for the empowerment and leadership of women, youth and groups in vulnerable situations. UNCTAD and the Regional Commissions could also

Accepted Through existing and future UNDA projects and subject to the availability of funds and the potential for synergies with UNCTAD, DESA and the remaining regional commissions, UNECE’s actions can be framed around the following activities:

Facilitating knowledge exchange by participating in, and possibly co-hosting national and regional events for discussing emerging issues and sharing national and regional experiences.

Promoting best practices by leveraging its extensive network and expertise to compile and disseminate case studies and success stories from its member states.

Supporting national, regional and sector- specific initiatives that align with the EPF, including contributing to the development of policies and programs that foster innovation, the transition to a circular

UNECE Economic Cooperation and Trade Division and Sustainable Energy Division

2 years Yes

5

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

propose organizing regional events and/or a global conference to promote entrepreneurship policy and further advance the goals and impact of the Surge project. Furthermore, learnings from the experiences with the Surge about external partnerships should encourage UNCTAD to promote the EPF and entrepreneurship development to other UN agencies also engaged in this area (e.g. ILO, FAO, UN Women, etc.), to development banks, or to related initiatives such as the recent Global Accelerator on Jobs and Social Protection. UNCTAD could also consider joining and supporting events that promote the achievement of SDG 8.3 as an avenue to further promote the EPF.

economy, and enable MSMEs to integrate sustainability practices throughout their supply chains.

Recommendation 3:

The DA-PMT should develop a clear framework for assessing the costs and benefits of implementing a global or

6

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

inter-regional project versus regional projects.

As a global crisis can affect regions and countries differently, global or joint projects should be developed only when there are clear benefits of joint implementation. Thus, a global or inter-regional vs. regional response would not be a priori decision, but a result of a clear assessment. A specific guideline or framework should be developed for this. Some of the assessment criteria could include the need or demand from member States for an integrated response; the range of common versus entity-specific activities and added value of complementary interventions (i.e. in terms of geographical coverage, reach of the target population, capacity, and/or coverage of multiple reinforcing technical areas, or networks and partnerships); capability to execute joint interventions (e.g. in terms of time, resources, logistics, and flexibility to

7

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

pivot interventions without bureaucratic procedures); coordination costs (which increase with the number of participating UN entities); project inception modalities and governance and knowledge management requirements to facilitate collaboration and knowledge sharing; scaling and sustainability plans; expected results of spreading resources versus concentrating on fewer countries; and so forth. The time taken to conduct such assessments will increase project coherence and effectiveness and facilitate the identification of the financial and human resources needed for project implementation. The network of DA Focal Points is a key existing asset for this assessment.

Recommendation 4:

Implementing UN entities should ensure that they have a comprehensive Results Framework for the entire project as well as an

Accepted This guidance already exists for UNECE projects, whether funded from XB or UNDA. UNECE divisions will continue to apply and enrich in future projects.

UNECE Economic Cooperation and Trade Division and Sustainable Energy Division

8

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

adequate monitoring plan, with indicators that are designed to support the ongoing monitoring.

The results framework should have one objective and ideally have one outcome per cluster/workstream. The objective should state the intended goal of the project, describe the overall achievement targeted by the project, involving a process of change aimed at meeting the needs of identified beneficiaries, and reflect the overall funding available to the project. Each objective should include reference to the project’s beneficiaries and its substantive focus. The objective should not attempt to explain the ways in which the project intends to achieve the objective (i.e. it should not include the word ‘through’ or describe the internal work of the UN using verbs such as ‘support’, ‘facilitate’ or ‘contribute’). The outcomes (OCs) should describe the changes that are expected to occur as a result of the completion of outputs.

9

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

The OCs should be achievable within the project’s timeframe and budget, and should be specific enough to be measured by the associated indicators of achievement. The indicators of achievement (IAs) should provide measures for monitoring progress towards achieving the OCs and reporting on them after completion of the project. Every indicator needs to provide clearly defined baselines, units of measurement and targets, detailing the quantity, quality and timing of expected results. The monitoring, evaluation, accountability, and learning (MEAL) system should also be designed to capture HRBA, gender and LNOB aspects. In phased interventions or during project execution, any changes in the logframe if/when pivoting activities should be clearly explained to the wider team.

Recommendation 5: Accepted This guidance already exists for UNECE projects, whether funded from XB or

UNECE Economic Cooperation and

10

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

Implementing UN entities should ensure that sufficient resources are allocated to project coordination, technical collaboration, and partnership building.

The absorption capacity of implementing entities can be challenged by crisis response projects which add to the planned programme of work. This is further compounded by projects that come with an extensive UN partnership and a global scope. Sufficient resources should be dedicated to global coordination and to building global partnerships with strategic stakeholders (e.g., UN organisations engaged in supporting the MSME sector; development banks). Capacities should also be directed to supporting technical collaboration and the staff implementing interventions, including towards synergy or liaison with the UNCTs. When designing the project, implementing UN entities should consider featuring

UNDA. UNECE divisions will continue to apply and enrich in future projects.

Trade Division and Sustainable Energy Division

11

Evaluation Recommendation (a) Management

response (b) Accepted, partially

accepted or rejected

Management plan Actions to be taken, and/or

comments about partial acceptance or rejection (c)

Responsible unit (d)

Timeframe (e)

Resources required

(Y or N) (f)

coordination and partnership- building in the Theory of Change or logframe of the project. Tools to support continuous connections and knowledge exchange, and to ensure institutional memory should be part of the response package, such as a project website, SharePoint space for all team members, and a Yammer network or Teams channel. The integration of cross-cutting aspects (HRBA, gender responsiveness, LNOB) also requires expertise with sufficient and dedicated time and resources. Guidance could be development to project managers on how to do this.

EVAL_Trade_UNDA2023W_ToR_2024

Evaluation Terms of Reference  Evaluation of the United Nations Development Account 12th tranche “Global Initiative towards post-Covid-19 resurgence of the MSME sector” (2023W)  

Languages and translations
English

Annex 7: Evaluation Terms of Reference Evaluation of the United Nations Development Account 12th tranche “Global Initiative towards post-Covid-19 resurgence of the MSME sector” (2023W)

TERMS OF REFERENCE

A. BACKGROUND A1. ABOUT THE DEVELOPMENT ACCOUNT

The Development Account (DA) is a mechanism to fund capacity development projects of the 10 economic and social entities of the United Nations Secretariat, namely: the Department of Economic and Social Affairs (DESA), the Economic Commission for Africa (ECA), the Economic Commission for Europe (ECE), the Economic and Social Commission for Asia and the Pacific (ESCAP), the Economic and Social Commission for Western Asia (ESCWA), the Economic Commission for Latin America and the Caribbean (ECLAC), the United Nations Conference on Trade and Development (UNCTAD), the United Nations Environment Project (UNEP), the United Nations Human Settlements Project (UN-Habitat) and the United Nations Office on Drugs and Crime (UNODC). The DA provides capacity development support to developing countries in their implementation of the 2030 Agenda for Sustainable Development, as well as recommendations and decisions made in intergovernmental processes and relevant governing bodies. The DA-funded projects build on the mandates, individual technical capacities and comparative advantages of the respective implementing entities, while providing those mostly non-resident entities with the ability to operationalize their knowledge and know-how to deliver capacity development support at regional, sub-regional and country levels. The Under-Secretary-General (USG) for Economic and Social Affairs is designated as the Project Manager of the Development Account with responsibility for overall coordination, programming, monitoring and evaluation, as well as for reporting to the intergovernmental bodies. The Project Manager is supported by the DA Steering Committee, who advises him/her on strategic policy and project-support matters.166 The Project Manager is also supported by the DA Project Management Team (DA-PMT) located within the Capacity Development Programme Management Office (CDPMO) of DESA, which assists with all aspects of the management of the DA, in particular with regard to programming, monitoring, evaluation and reporting. DA-PMT also liaises with the DA Focal Points in the implementing entities, who are most often the head of the entity’s unit responsible for project planning, project management, capacity development or technical cooperation, on all aspects of the management of DA-funded projects. In response to the COVID-19 crisis, the Development Account has funded five short-term joint projects to help developing countries alleviate the socio-economic impact of the pandemic, including the project on Global initiative towards post-COVID-19 resurgence of the MSME sector (2023W).

A2. ABOUT THE PROJECT The COVID-19 crisis is plunging the global economy into a deep recession and micro, small and medium enterprises – which play a major role in emerging economies – are amongst the hardest hit. Trapped in economic stagnation due to large-scale lockdowns, millions of MSMEs have become the most vulnerable to COVID-19 within the private sector. Compared with large firms, small businesses have fewer resources and lower capacities to cope with the abrupt economic shocks economies are currently facing. With more than two-thirds of the global population employed by MSMEs, the unprecedented outbreak of the pandemic has vividly shown how tightly their activities are woven into the economic and social fabric of the world, as well as their critical role in social and economic resurgence. The objective of the project is to develop and implement capacity-building tools for governments and micro, small and medium enterprises (MSMEs) to facilitate the resurgence and strengthen the resilience of MSMEs in developing countries and economies in transition. These capacity-building tools will seek to mitigate the

economic and social impact of the global COVID-19 crisis and to facilitate the contribution of MSMEs to the SDGs implementation. The project was designed based on the request for assistance for MSMEs from more than 50 Member States, including countries in Latin America and the Caribbean, Asia and the Pacific, Africa, Europe and the Arab regions, as well as intergovernmental demands and resolutions on COVID-19. The project is jointly implemented by UNCTAD, DESA, ECA, ECE, ECLAC, ESCAP and ESCWA, and builds upon the comparative advantages of the participating agencies to provide immediate advice, capacity-building and support to governments and MSMEs during the ongoing global pandemic. The project is structured in five clusters that address the most critical areas of the MSME recovery. Broadly, the roles and lead entities for each of the clusters and workstreams are as presented in the following table:

Table 1. Project clusters and leads Project cluster/workstream Lead agency

Overall coordination UNCTAD

Project cluster/workstream 1 Entrepreneurship and business skills promotion

UNCTAD

Project cluster/workstream 2 Business facilitation/formalization

UNCTAD/DESA

Project cluster/workstream 3 Access to finance/financial literacy

ESCAP

Project cluster/workstream 4 Access to technology and innovation

UNECA

Project cluster/workstream 5 Access to markets

UNECE

The beneficiary countries cover different geographical regions, as shown in Annex 2. The expected outcomes, indicators of achievement, and outputs are presented in the project results framework (Annex 3). The project was developed and implemented under three phases. A new set of outputs was designed or added at each of the three phases of the project. Under the three-phase approach, the project budget was approved by phase. In 2021, when the phase 3 budget was discussed, the Development Account faced a funding gap. To bridge the gap, in November of the same year, the five joint projects were requested to reduce their proposed phase 3 budget by 1 million USD, which led to the curtailment of certain planned activities. For this project, the budget was reduced by $310,000. Overall, a total of $4,490,500 was allocated under this project. Concretely, UNCTAD received $2,671,000, ECLAC received $134,000, ESCAP received $240,000, ECA received $467,000, ECE received $448,500, ESCWA received $370,000 and DESA received $160,000. The project started its implementation in May 2020 and was scheduled to conclude on 31 March 2022, but received approval in February 2022 for an extension until 30 June 2022.

B. EVALUATION OBJECTIVES AND SCOPE B1. EVALUATION PURPOSE, OBJECTIVES AND SCOPE

The present evaluation will constitute a terminal evaluation of the Project. Terminal evaluations are mandatory for all DA-funded projects with a value above $1 million. The evaluation will be largely guided by the UN Development Account Project Evaluation Guidelines, issued in October 2019 and the evaluation policies of the implementing entities, in particular, UNCTAD, which leads the evaluation. The main purpose of the evaluation will be to support accountability for results, and to enable learning. This terminal evaluation of the project has the following specific objectives:

• Assess the results and establish the link between achievements and activities of the intervention;

• Assess the response delivery and external coordination167, including the extent of gender, human rights and disability mainstreaming; and • Identify good practices and lessons learned from the project that could feed into and enhance the implementation of related interventions.

The primary intended users of the assessment are the management of the implementing entities. The evaluation will also provide accountability to project beneficiaries and member States. Furthermore, the evaluation will form a key input to the programme-level evaluation of the DA’s response to COVID-19 to be initiated by the CDPMO/DESA. The programme-level evaluation will entail: a synthesis of the terminal evaluations of five COVID-19 joint DA projects, including this project; a review of relevant 10th and 11th tranche DA projects; and a programme-level assessment. The primary audiences of the programme-level evaluation will include the DA Steering Committee, the DA-Programme Management Team (DA-PMT), and the management of the implementing entities. The results of the programme-level evaluation will also be presented to the General Assembly, through the biennial progress report on the implementation of the DA. The evaluation will cover the duration of the project from May 2020 to 30 June 2022, covering all phases, clusters and activities.

B2. EVALUATION CRITERIA AND QUESTIONS The evaluation will assess the Project’s performance against the main criteria of relevance, efficiency, effectiveness, coherence, sustainability, gender, human rights and disability. In particular, the evaluation is expected to address a number of questions under the following criteria168:

Table 2: Evaluation criteria and tentative questions Relevance 1. To what extent was the project designed to target the new

needs and priorities of participating countries as a result of COVID- 19?

Relevance 2. To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g. COVID-19 Socio-Economic Response Plan)?

Efficiency 3. How well coordinated was the response among the entities implementing the joint project?

Efficiency 4. How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

Effectiveness 5. To what extent did the programme (Development Account) and project governance and management structures and processes enabled, or hindered, the effective implementation of the joint project and the achievement of its results?

Effectiveness 6. To what extent has the project contributed to the expected outcomes as enunciated in the project document?

Effectiveness 7. How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

Effectiveness 8. What innovative approach or tool, if any, did the response use, and what were the outcomes and lessons learned from its application?

Sustainability 9. What measures were adopted to ensure that the outcomes of the response would continue after the project ended?

Coherence 10. To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

Coherence 11. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID-19 in delivering socio-economic support to Member States?

Gender, human rights and disability

12. To what extent were gender, human rights and disability perspectives integrated into the design and implementation of the project? What results can be identified from these actions?

C. EVALUATION APPROACH AND METHODOLOGY The evaluation will be a transparent and participatory process involving the Project’s implementing entities and key stakeholders. It will be conducted based on gender and human rights principles and adhere to the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation. The evaluation will apply a mixed-method design, including a combination of quantitative and qualitative analysis to inform findings.It is anticipated that travel of the evaluation team may take place in support of elaborating case study/ies, as well as to meet key project stakeholders in Geneva (UNCTAD and ECE). The selection of potential case study/ies and travel requirements will be developed as part of the inception report. Following a preliminary documentation review and a limited number of inception meetings with the core project team, the Evaluation Team will develop an inception report for the evaluation, which will include the finalized overall scope and focus of the evaluation, evaluation questions and methodology, including information on data sources and collection, sampling, key indicators, stakeholder mapping, selection of case study/ies, survey design, and the evaluation timeline. The tentative methodology for the evaluation is presented in Table 4.

Table 3: Tentative methodology for the assessment

a. A desk review of Project documents, including documents/data related to: o Project-level planning, implementation and results achievement, including but not limited to:

▪ Concept note, Phase 2 project proposal, and Phase 3 budget and outputs ▪ Progress report for Phases 1 and 2 (both financial and substantive/narrative report) ▪ Final report (both financial and substantive/narrative report) ▪ Meeting minutes, including the minutes of the bi-weekly/monthly DA network meetings ▪ Monitoring reports ▪ Information on non-DA resources, financial and in-kind, brought in by the participating entities ▪ Information on resources, financial and in-kind, contributed by partners/donors (including information requested under the “supplementary funding” section in the progress reports, which is often incomplete) ▪ Beneficiary/user feedback collected, including, but not limited to, workshop survey results, user feedback on publications, advisory services, guidelines, methodology documents, etc. ▪ Requests for assistance/services received ▪ List of activities completed and details about each activity, including but not limited to:

▪ Agenda, participant lists (name, title, division/unit, organization, country, gender, email address), report and any outcomes document, for each workshop/meeting ▪ Description of each advisory service, beneficiaries (including contact details of the contact persons) and any outputs/deliverables produced ▪ List and description of tool(s), research papers, policy briefs, studies published and information on how each product was disseminated and/or used, list of recipients/users of the product (e.g., dissemination lists)

▪ Documentation related to broader projects or sub-projects of the participating entities of which the Project or its component(s) has constituted an integral part or which are linked to and/or build upon/succeed the work undertaken as part of the Project ▪ Documents and literature related to the Project context ▪ Relevant web and social media metrics related to the outputs of the project;

o Project strategic documents, including but not limited to: ▪ General Assembly's Resolution on Global Solidarity to fight the coronavirus disease 2019 (COVID-19) (A/RES/74/270);

▪ Secretary General's report on "Shared responsibility, global solidarity: Responding to the socio-economic impacts of COVID-19"; ▪ UN Framework for the immediate socio-economic response to COVID-19; ▪ 2021 Programme budget and mandate of implementing entities; ▪ COVID-19 Socio-Economic Response Plan of participating countries.

b. Questionnaires/surveys (in appropriate languages in addition to English) to relevant stakeholders in countries participating in a sample of project activities; c. Telephone, online or in-person interviews with key stakeholders, including but not limited to:

o Project Coordination Team and project focal points of implementing entities o DA-PMT o DA focal points in participating entities o Sample of UN Resident Coordinators/Country Teams, as appropriate o Sample of key global partners o Sample of country-level stakeholders (mainly stakeholders from key beneficiary countries)

d. Case Study/ies, which may include for example a detailed examination of a particular intervention, or of project activities at a regional or national level.

In addition to assessing the mainstreaming of gender, human rights and disability perspectives in the design, implementation and monitoring of the Project (evaluation question 12), the evaluation will integrate these perspectives in the management of the evaluation, data collection and analysis, as well as the development of the evaluation report. Gender balance will be given full consideration in the composition of the Evaluation Reference Group, elaborated in Section D1 (Evaluation management), and the Evaluation Team. Data collected and analyzed in the course of the evaluation will be disaggregated by gender to the extent possible and whenever appropriate, and the evaluation findings, conclusions and recommendations to be presented in the evaluation report will reflect a gender analysis. The evaluation will be carried out according to the UNEG ethical principles and standards.169 The evaluators should demonstrate behavioural independence, impartiality, credibility, honesty, integrity and accountability in conducting the evaluation/assessment to avoid biasing the findings. The evaluators must also address in the design and conduct of the evaluation procedures to safeguard the rights and confidentiality of information providers. The Evaluation Manager will be responsible for ensuring that the Evaluation Team conducts the work assignments without any undue interference from those who were responsible for the implementation of the Project.

D. ORGANIZATION OF THE EVALUATION D1. EVALUATION MANAGEMENT

The independent final project evaluation will be managed/coordinated by UNCTAD’s Independent Evaluation Unit, with the support of an Evaluation Advisory Committee (EAC) that comprises a representative each of the evaluation units of the partner entities (DESA, ECA, ECE, ECLAC, ESCAP, ESCWA) and the Evaluation Officer with the CDPMO/DESA. The EAC primarily serves a quality assurance function and facilitates support to the Evaluation Team as necessary. An Evaluation Reference Group (ERG) consisting of a representative from each UN partner entity (e.g., the DA Focal Point for each entity) and the DA-PMT will review and contribute inputs to key steps in this evaluation such as the TOR and draft final report. Both the EAC and the ERG commit to submitting substantive comments on a timely basis, and comments will be invited on a ‘non-objection’ basis (no response = agree) so that the process is not delayed for an unnecessarily long time. An independent Evaluation Team will be convoked to undertake this assignment. The Evaluation Team (ET) is responsible for conducting the evaluation, applying the methodology as appropriate and for producing the evaluation report. All team members, including the Team Leader, will participate in briefing and debriefing meetings, discussions, and will contribute to the evaluation with written inputs for the draft and final report. A selected number of the evaluation team members will participate in the mission travel(s) if applicable. The ET and the Evaluation Manager will agree on the outline of the report, in consultation with

the EAC early in the evaluation process. The ET will develop its own evaluation tools and framework, within the available timeframe and resources. The team is fully responsible for its report, which may not reflect the views of any of the implementing entities of the project. The evaluation report is subject to quality control by the Evaluation Advisory Committee and clearance by the Evaluation Manager, as set out above. The Team Leader guides and coordinates the team member(s) in their specific work, discusses their findings, conclusions and recommendations and prepares the draft and the final report, consolidating the inputs from the team member(s) with his/her own. The members of the evaluation team should possess a mix of evaluation skills and technical or sectoral/thematic knowledge relevant to the evaluation. In putting together the team, adequate linguistic, geographic and gender representation will also be key considerations. The Evaluation Team will be provided full access to all project reports, documentation, and stakeholder lists and contact information. The Project Coordination Team are required to submit to the evaluation manager project documentation, including data and information residing with the other participating entities, in the last month of the project if possible, if not, immediately following the completion of the project, as well as support the evaluation process, including through facilitating the evaluators’ access to the project’s beneficiaries and other key stakeholders. The roles and responsibilities in the evaluation process are described below:

Evaluation Manager (UNCTAD) will: • Prepare the draft evaluation TOR and revise/finalize based on inputs received • Prepare the TOR for each member of the Evaluation Team (Team Leader, Team Member and Expert(s)) • Recruit and manage the Evaluation Team • Backstop the evaluation process, including supporting the development and administration of surveys, support outreach of the evaluation team to project stakeholders, and access to secondary data listed in Table 3. • Oversee/provide quality assurance to the evaluation and the development of the evaluation report • Facilitate the work of the Evaluation Advisory Committee and the Evaluation Reference Group • Be responsible for clearance of the evaluation report • Support the development of a management response to the evaluation report, including an implementation plan • Organise a virtual workshop on evaluation findings and lessons learned.

Evaluation Advisory Committee comprises a representative each of the evaluation units of the partner entities (ESCWA, ECA, ECE, ECLAC, ESCAP, DESA) and the Evaluation Officer with the CDPMO/DESA. The EAC primarily serves a support and quality assurance function. Specific responsibilities of the EAC include:

• Review and approve the evaluation TOR; • Advise on the selection of the evaluation consultant(s) to ensure that the selection is based on the required skills and qualifications; • Support facilitating access from their respective entities to relevant project documentation and stakeholders; • Review and comment on the inception and evaluation reports; and • Monitor and conduct periodic follow-ups on the implementation of evaluation recommendations addressed to the parties within their entities.

Evaluation Reference Group, consisting of a representative from each UN partner entity (e.g., the DA Focal Point for each entity) and the DA-PMT,170 will review and contribute inputs to key steps in this evaluation such as the TOR and draft final report. The ERG’s key function is to enhance the relevance, credibility and transparency of the evaluation process. Specific responsibilities include:

• To review the draft evaluation ToR and provide substantive feedback; • To facilitate access from their respective entities to relevant project documentation and stakeholders;

• To review the draft evaluation report and provide substantive feedback, including coordinating feedback from other sections, units and offices from headquarters and from the field to ensure quality and completeness; • To participate in the validation meeting of the final evaluation report; • To play a key role in disseminating the findings of the evaluation and implementation of the management response.

Project Coordination Team will: • Facilitate the Evaluation Team’s access to relevant Project documentation and stakeholders, including through:

o Collecting and compiling requested data and information from the participating entities, as requested by the Evaluation Manager o Providing an updated list of stakeholders, and facilitating access to the sample of stakeholders that the Evaluation Team may wish to interview o Facilitating the administration of questionnaires to workshop participants in the participating countries o Ensure the cooperation and contribution of the relevant staff of the implementing entities to the evaluation process, as requested

• Lead the preparation of a response to the recommendations directed to the participating entities, including an implementation plan

DA-PMT will: • Participate in the Evaluation Reference Group • Provide guidance on the allocation of the evaluation budget • Organize a virtual meeting with DA focal points to discuss the key lessons from this evaluation as well as from other COVID-19 joint project evaluations and how to incorporate them in future programming, implementation, monitoring and evaluation of DA-funded projects.

D2. EVALUATION TIME FRAME

The evaluation will be conducted from December 2022 to August 2023. The evaluation process will involve five phases with the tentative timelines as below in Table 5 (the timelines may be adjusted should any exigencies arise):

Table 5: Evaluation phases and tentative timelines Phase Timelines

1. Preparation

August 2022 – December 2022 • Preparation and finalization of evaluation TOR • Establishment of the Evaluation Reference Group • Recruitment of the Evaluation Team • Prepare package of documents required by the Evaluation Team

2. Inception

December 2022 -March 2023 • Preliminary documentation review and preparation of inception report by the Evaluation Team, including development of data collection instruments (questionnaires/ surveys, interview guides) • Brief visit to Geneva (3 days) to meet with UNCTAD Evaluation Unit and key project stakeholders (UNCTAD and ECE) • Draft inception report due: 10 February 2023

• Evaluation Manager review and inception report revision by Evaluation Team: 13 - 22 February 2023 • Reviews by Project Coordination Team, project focal points of implementing entities, and Evaluation Advisory Committee (in parallel): 23 February – 7 March 2023 • Draft final inception report due: 14 March 2023 • Final inception report approved: 20 March 2023

3. Data collection and analysis March- May 2023 • Desk review of remaining Project documents, including requesting additional documentation • Online surveys of stakeholders • Interviews with stakeholders • Data analysis and triangulation

4. Report preparation and reviews May – July 2023 • Data analysis and triangulation • First draft evaluation report due: 16 June 2023 • Evaluation Manager review and report revision by the Evaluation Team: 19 – 28 June 2023 • Reviews by Project Coordination Team, project focal points of implementing entities, Evaluation Advisory Committee and Evaluation Reference Group (in parallel): 29 June – 11 July 2023 • Revised draft evaluation report due: 18 July 2023 • Final evaluation report with annexes: 25 July 2023

5. Dissemination and follow-up August 2023 and onwards • Presentation to the Project Coordination Team, project teams of implementing entities and development and approval of a management response, including an implementation plan for recommendations • Virtual workshop on evaluation findings, lessons learned and follow-up with the DA Focal Points: April 2023

D3. EVALUATION TEAM DELIVERABLES The Evaluation Team will be composed of a team of three consultants (evaluators), namely Team Leader, Team Member and a Gender and Human Rights (HRGE) Expert who also plays the role of Team Member. The two Team Members will report functionally to the Team Leader. The Team Leader will report to the Evaluation Manager. Each of the Evaluation Team has a set of deliverables as described below: Deliverables for Team Leader and Team Member

o Initial review of key Project documents (preliminary document review) o Preparation of an inception report with a finalized evaluation scope and focus, evaluation questions and methodology, including information on data sources, sampling and key indicators, stakeholder mapping/analysis, selection of case study/ies, as well as survey design

o Desk review of remaining Project documents o Data collection and analysis based on the finalized methodology o Preparation of an evidence matrix presenting a summary of evidence collected through each data collection method by evaluation question o Development of a draft evaluation report, based on the template presented in Annex 1, for review by the Evaluation Manager, Project Coordination Team, project focal points of implementing entities, the EAC and the ERG o Revision/finalization of the evaluation report, including all annexes, based on comments received o Preparation of a 3-page summary of the evaluation report and a presentation (PPT) on key findings, conclusions and recommendations o Presentation of evaluation report and discussions with relevant stakeholders such as Project Coordination Team, project teams of implementing entities, DA focal points of participating entities and DA-PMT.

Deliverables for Gender and Human Rights Expert/Team Member

o Initial review of key Project documents (preliminary document review), including identifying gender equality, human rights and disability inclusion dimensions and issues for consideration; o Preparation of an inception report with a finalized evaluation scope and focus, evaluation questions and methodology, including information on data sources, sampling and key indicators, stakeholder mapping/analysis, selection of case study/ies, as well as survey design. Where applicable, gender equality, human rights and disability inclusion considerations will be integrated in the evaluation scope of analysis; evaluation criteria and questions design; methods and tools, and data analysis techniques; o Desk review of remaining Project documents; o Data collection and analysis based on the finalized methodology which would be gender sensitive; o Preparation of an evidence matrix presenting a summary of evidence collected through each data collection method by evaluation question; o Development of a draft evaluation report, based on the template presented in Annex 1 of the Terms of Reference for the evaluation, for review by the Evaluation Manager, Project Coordination Team, project focal points of implementation entities, the EAC and the ERG. The analysis of gender equality, human rights and disability inclusion dimensions and issues should be integrated in the report as an independent section and to the extent possible, these issues should be mainstreamed throughout the report, including in the evaluation findings, conclusions and recommendations. o Revision/finalization of the evaluation report, including all annexes, based on comments received; o Preparation of a 3-page summary of the evaluation report and a presentation (PPT) on key findings, conclusions and recommendations. o Presentation of evaluation report and discussions with relevant stakeholders such as Project Coordination Team, project teams of implementing entities, DA focal points of participating entities and DA-PMT.

E. COMMUNICATIONS AND DISSEMINATION PLAN: The results from the evaluation including key lessons learned, best practices and recommendations will be shared widely with participating entities, partners and stakeholders, and member States. In particular, the following modes of communication could be used:

e. A workshop with all relevant stakeholders to present the key findings, recommendations and lessons learned. The evaluation report will be presented at a workshop attended by the implementing entities, the DA-PMT and other relevant stakeholders for discussion and validation. The implementing

entities will be given the opportunity to present their management response, including an implementation plan for the recommendations; f. A separate virtual meeting will be held with the DA focal points to discuss the key lessons from the evaluation as well as from other COVID-19 joint project evaluations and how to incorporate them in future programming, implementation, monitoring and evaluation of DA-funded projects and projects. g. A copy of the final evaluation report will be published on UNCTAD’s website and the websites of the partner implementing entities, as appropriate; and h. The key findings from the evaluation report will also form a key input to the programme-level evaluation of the DA’s response to COVID-19 to be initiated by the CDPMO/DESA. i. Other communication briefs and products will be produced as appropriate.

EVAL_Trade_UNDA2023W_EvalReport_April2024

United Nations Development Account Terminal Evaluation of Project 2023W “Global Initiative towards post-Covid-19 resurgence  of the MSME sector” (2020-2023) Report completed: April 2024 Evaluation conducted by: Patrick Breard, Ariane Agnes Corradi, Jude Kallick 1 

Languages and translations
English

1

United Nations Development Account

Terminal Evaluation of Project 2023W

“Global Initiative towards post-Covid-19 resurgence of the MSME sector”

(2020-2023)

Report completed: April 2024

Evaluation conducted by: Patrick Breard, Ariane Agnes Corradi, Jude Kallick

2

Evaluators: Patrick Breard, PhD, Team Leader

Ariane Agnes Corradi, PhD, Team Member Jude Kallick, MA, Team Member

Evaluation Manager: Nishta Keeble, Chief, Independent Evaluation Unit, Office of the Secretary-

General, UNCTAD

Contact: Thomas Callaghan, Associate Programme Management Officer, Independent Evaluation Unit, Office of the Secretary-General, UNCTAD. Tel: +41 22 917 5295 Email: [email protected] This report was commissioned by UNCTAD. The findings, conclusions and recommendations of this report are

those of the external evaluator and do not necessarily reflect the views of the UNCTAD.

3

Table of Contents

Lists of figures, tables, and boxes.............................................................................................................................................. 4

List of Acronyms and Abbreviations ........................................................................................................................................ 5

Executive summary ....................................................................................................................................................................... 6

1. Introduction .................................................................................................................................................................................11

2. Description of the Project........................................................................................................................................................11

2.1 Background ...........................................................................................................................................................................11

2.2 Project objectives and expected accomplishments ................................................................................................ 12

2.3 Project strategies and key activities .............................................................................................................................13

2.4 Beneficiaries and target countries................................................................................................................................. 14

2.5 Key partners and other key stakeholders .................................................................................................................. 16

2.6 Resources ........................................................................................................................................................................... 16

2.7 Link to the Sustainable Development Goals (SDGs) ................................................................................................ 17

2.8 Innovative elements ......................................................................................................................................................... 17

3. Evaluation objectives, scope, and questions .................................................................................................................... 17

3.1 Purpose and objectives .................................................................................................................................................... 17

3.2 Evaluation scope, criteria, and questions .................................................................................................................. 18

4. Methodology ............................................................................................................................................................................. 19

5. Findings ....................................................................................................................................................................................... 21

5.1 Relevance ............................................................................................................................................................................. 21

5.2 Coherence .......................................................................................................................................................................... 24

5.3 Efficiency ............................................................................................................................................................................ 29

5.4 Effectiveness ...................................................................................................................................................................... 32

5.4 Sustainability .................................................................................................................................................................... 59

5.5 Gender, Human Rights, and Leave No One Behind ................................................................................................ 64

6. Conclusions ............................................................................................................................................................................... 73

7. Recommendations .................................................................................................................................................................. 76

Annexes .......................................................................................................................................................................................... 78

Annex 1: Project’s Theory of Change ................................................................................................................................. 79

Annex 2: Evaluation matrix.................................................................................................................................................. 80

Annex 3: Data collection instruments .............................................................................................................................. 86

Annex 4: List of documents reviewed ............................................................................................................................... 97

Annex 5: List of individuals interviewed .......................................................................................................................... 99

Annex 6: Evaluation surveys ............................................................................................................................................. 100

Annex 7: Evaluation Terms of Reference ........................................................................................................................ 134

4

Lists of figures, tables, and boxes List of Figures Figure 1: Overview of project activities 13 Figure 2: Allocation of project funding 16 Figure 3: Perceived relevance of the project according to the survey 22 Figure 4: Project outputs per implementing UN entity and phase 27 Figure 5: Number of project outcomes covered by the implementing UN entities 28 Figure 6: Perceived complementarity of the project with other interventions 29 Figure 7: Surge Project outputs delivered per implementing entity and implementation phase 37 Figure 8: DEPAR number of users 38 Figure 9: Effectiveness results of outcome 1.A 38 Figure 10: Effectiveness results of outcome 1.B 40 Figure 11: Effectiveness results of outcome 3 45 Figure 12: Effectiveness results of outcome 4 47 Figure 13: Effectiveness results of outcome 5 48 Figure 14: Outputs per project outcome and implementing entity 53 Figure 15: Project contributions to formulate and implement enabling policies to MSMEs 54 Figure 16: Contribution of the project to improve the resilience and competitiveness of MSMEs in post-. COVID-19 resurgence in a sustainable way 60 Figure 17: Contribution of the project to better outcomes for women and youth entrepreneurs and other vulnerable groups in a sustainable way 61 Figure 18: Contribution of the project to formulate and implement enabling policies for MSME post-COVID-19 resurgence in a sustainable way 61 Figure 19: Contribution of the project to improve the resilience and competitiveness of MSMEs in post-COVID-19 resurgence in a sustainable way 62 Figure 20: Perceived accessibility of project activities for women, youth, people with disabilities or other vulnerable groups 69 Figure 21: Perceived contribution of the project to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups 70 List of Tables Table 1: Project’s beneficiary countries 15 Table 2: Project clusters and lead and participating entities 16 Table 3: Project's implementation phases 17 Table 4: Evaluation questions 18 Table 5: Groups of informants and number of consultations 19 Table 6: Survey recipients and respondents 20 Table 7: Planned and delivered outputs per project phase 37 Table 8: Level of achievement of the Surge project indicators 50 List of Boxes Box 1: ECE’s guidelines and best practices – from a general approach to supporting country-specific responses 55 Box 2: UNCTAD Cross-border Trade and Gender Initiative 68

5

List of Acronyms and Abbreviations

ADB Asian Development Bank

AfDB African Development Bank

DA Development Account

DESA Department of Economic and Social Affairs

EAC Evaluation Advisory Committee

ECA Economic Commission for Africa

ECE Economic Commission for Europe

ECLAC Economic Commission for Latin America and the Caribbean

EMPRETEC Emprendedores (entrepreneurs) and tecnología (technology)

ERG Evaluation Reference Group

ESCAP Economic and Social Commission for Asia and the Pacific

ESCWA Economic and Social Commission for Western Asia

ET Evaluation Team

GDP Gross Domestic Product

GHRP Global Humanitarian Response Plan

HIC High-income Country

HRBA Human Rights-Based Approach

ICT Information and Communication Technology ILO International Labour Organization

IMF International Monetary Fund

LDC Least Developed Country

LIC Low-income Country

LLDC Landlocked Developing Country

LNOB Leaving No One Behind

MIC Middle-income Country

MNE Multinational Enterprise

MPTF Multi-Partner Trust Fund

MSME Micro-, Small and Medium Enterprises

PMT Project Management Team

PRODOC Project Document

RC Resident Coordinator

SDG Sustainable Development Goal

SIDS Small Island Developing States

SPRP Strategic Preparedness and Response Plan

SRO-SA Subregional office for Southern Africa

TCS Technical Cooperation Section

TOC Theory of Change TOR Terms of Reference

UN United Nations

UNCTAD United Nations Conference on Trade and Development

UNDP United Nations Development Programme

UNIDO United Nations Industrial Development Organization

UNOSSC United Nations Office of South-South Cooperation

UNSD United Nations Statistics Division

WHO World Health Organization

WTO World Trade Organization

6

Executive summary Project overview

The project “Global Initiative towards post-Covid-19 resurgence of the MSME sector” aimed at enhancing the resilience of Micro, Small, and Medium Enterprises (MSMEs) in developing countries and economies in transition, especially in the wake of the global COVID-19 crisis. The primary objective of the project was to develop and implement capacity-building tools for both governments and MSMEs. These tools were designed to mitigate the economic and social repercussions of the pandemic, focusing on MSMEs’ contribution to the SDGs. Operationally, the project was structured according to the Entrepreneurship Policy Framework (EPF) of UNCTAD. The EPF aims to support developing country policymakers in the design of initiatives, measures and institutions to promote entrepreneurship. The EPF features 6 key components: 1: Formulating a National Entrepreneurship Strategy; 2: Optimizing the Regulatory Environment; 3: Enhancing Entrepreneurship Education and Skills Development; 4: Facilitating Technology Exchange and Innovation; 5: Improving Access to Finance; and 6: Promoting Awareness and Networking. Building on these components1. The project was operated under five clusters, addressing critical areas of MSME recovery. These clusters included mobilizing entrepreneurial ecosystems, simplifying business registration processes, improving access to finance and financial literacy, increasing access to technology and innovation, and enhancing access to markets. The activities were designed to create an enabling entrepreneurship ecosystem by improving regulatory environments, providing access to innovation, technology, finance, and markets, and reaching out to vulnerable groups such as women and informal workers.

The project commenced in May 2020. The initial completion date was planned for March 2022 but this was extended to June 2022. It underwent three phases involving project startup and approval, initial project delivery and design of scaling up initiatives, and the launch of new activities based on previous results. Beneficiaries of this project included governments, policymakers, MSMEs, business associations, support services, and relevant technical staff in institutions and ministries. The project provided support to 96 beneficiary countries, employing various interventions such as advisory services, training, workshops, seminars, and online platforms. Jointly implemented by UNCTAD, DESA, and the UN regional commissions for Africa (ECA), Europe (ECE), Latin America and the Caribbean (ECLAC), Asia and the Pacific (ESCAP), and the Arab region (ESCWA), the project received financial allocations totalling $4,490,500, with each participating agency implementing its allocated component and managing day-to-day project operations. UNCTAD served as the lead coordinating entity. Key partners in this initiative included UN Development Partners, UN Resident Coordinators, UN Country Teams, UNITAR, UNDP, UNOSSC and ITC. Moreover, the project collaborated with EMPRETEC Centres and other stakeholders such as business incubators and support services. In alignment with the post-2015 development agenda, the project aimed to contribute directly to specific SDGs, particularly SDG 4 (Quality Education), SDG 8 (Decent Work and Economic Growth), and SDG 9 (Industry, Innovation, and Infrastructure).

Evaluation purpose and scope

This evaluation analysed the relevance, coherence, efficiency, effectiveness, and sustainability of the project and its mainstreaming of gender, human rights and disability considerations. The evaluation was carried out between February 2023 and September 2023 following a structured process of data collection and analysis, which included desk review, key informant interviews and online surveys. The evaluation conclusions and recommendations are intended to inform the work of UNCTAD and other UN implementing entities as well as the Development Account Programme Management Team (DA-PMT), which will conduct a synthesis from this evaluation and those of the other four DA Covid-19 projects.

1 According to the Project Document for Phase 2, interventions were selected based on several criteria including to target capacity building areas that are proven to have key impacts on entrepreneurship and MSMEs promotion as based on the formulation and implementation of UNCTAD’s Entrepreneurship Policy Framework (EPF): enabling ecosystem and facilitation of entrepreneurship competencies/motivation, access to finance, technology and markets, networking.

7

Conclusions

Relevance

The project responded to the needs of Governments and MSMEs. Interventions contributed to and were informed by needs and impact assessments. Recipients of project interventions were often involved in or contributed to the design and delivery of project activities and outputs. On various occasions, project outputs were tailored to the needs of target users and beneficiaries, being countries or vulnerable groups. The relevance of the interventions was somewhat mitigated by the mandates and capacities of the implementing UN entities. The latter are equipped for supporting medium- and long- term policy changes rather than immediate crisis responses. Other challenges included the restrictive Covid-19 measures and the short time frame for implementation of the project. Nonetheless, on the whole, the implementing UN entities exercised adaptive management to ensure high relevance of their interventions. The phased approach, in particular, provided the opportunity to introduce new activities not foreseen at an earlier stage.

Coherence

The project was anchored in the mandates and comparative advantages of the implementing UN entities. Selected interventions followed previous work and specific requests from member States and were often coordinated with or complementary to existing activities. At conception, the target of the project was to address specific constraints faced by MSMEs. It was designed around the Entrepreneurship Policy Framework to show the complementarities across interventions that each implementing UN entity would deliver in response to regional priorities and capacities. Some collaborations between implementing UN entities were pursued and complementarities established, but on a limited level. A few synergies in the form of joint outputs were integrated in the 3rd phase of the project. By design, technical task forces created through the project were expected to bridge the implementing UN entities. However, this proved difficult to launch and sustain, owing to several constraints, including a highly ambitious objective given the global crisis and competing priorities within each agency. Nevertheless, the coordination and complementarities established by the project among the implementing entities contributed to expanded dissemination and outreach of outputs, avoided overlaps between them, and allowed significant knowledge exchanges aligned with and supportive of a “One UN” system. On the other hand, collaboration of the implementing UN entities with other UN organisations, including UNCTs, was limited. Despite this shortcoming, at national level, target recipients of project outputs found the interventions complementary to those of other UN agencies.

Efficiency

Project coordination was complex and demanding, involving seven UN entities and the target delivery of around 100 outputs, of which some were composed of many activities. Nevertheless, the coordination of the project was found to be efficient overall given the circumstances and resources available. The coordination of the design phase of the response was actively supported by the DA-PMT. Coordination of project implementation was led by UNCTAD in collaboration with a Steering Committee that met on a regular basis, most often bi-monthly. UNCTAD’s Budget and Project Finance Section (BPFS) produced the project’s financial monitoring table every month, which included the expenditure data of UNCTAD as well as the other implementing entities. A monitoring dashboard was created by UNCTAD to facilitate the tracking of UNCTAD’s project delivery. Data from some of the other implementing UN entities was added, when provided, at two reporting points during the lifetime of the project.

The project would have benefited from the allocation of more resources towards the coordination and monitoring of overall implementation. Across the implementing UN entities, the project was considered complex and hard to follow outside of the interventions under their direct control. Some staff perceived that there were too many participants in the coordination meetings, that the project lacked a Theory of Change, and that it was primarily a compilation of activities rather than a cohesive and mutually reinforcing bundle of interventions supporting in depth any given country. Some staff in the UN Regional Commissions questioned the efficiency cost of a global or inter-regional response versus regional responses. The project was implemented in three phases, which were supportive of adaptive management and perceived by staff as the most rational approach considering the circumstances. However, it was also pointed out that clearer visibility from the onset on the funding available for future phases would have facilitated planning and the search for synergies.

8

Effectiveness

The Development Account programme, as a funding modality, was quick to respond and adapt to the COVID- 19 crisis. The prompt availability of resources and rapid orientation toward a global response contributed to the rapid launch of a wide range of interventions. However, questions remain as to its adequacy for addressing the immediate needs of a crisis in comparison with other UN agencies and mechanisms with a mandate for humanitarian response. The governance of this project, with DA-PMT’s oversight, the management structure with UNCTAD in the lead coordinating role, and each implementing entity leading the operational implementation at the regional level, ensured the global and regional scope of project delivery. At the operational level, implementing entities relied more on partnerships with local and regional institutions rather than coordination within the larger project management structure. The project governance and management mechanisms evolved in tandem with the COVID-19 crisis and the project’s emerging needs. The result was a high delivery rate but with limited inter-agency collaboration in the process. Yet, despite limited collaboration, this independent evaluation identified that the project was effective in producing identifiable results at the outcome level.

Overall, the project delivered 85% of the planned outputs. Phase 3, particularly, delivered the highest number of outputs but had the lowest outcome effectiveness rate, mostly because of budget cuts and some outputs still in progress at the time of this evaluation. There is evidence of the effective improvement of national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship and MSME promotion (outcome 1.A) [such as the adoption of a revised entrepreneurship strategy by South Africa, Uganda and Seychelles]; improved resilience and competitiveness of MSMEs (outcome 1.B) [for example with 64.5% of participants opening a new business after attending Empretec-based training]; facilitated MSME registration and formalization through e-platforms (outcome 2) in El Salvador, Benin, Cameroon and Mali; improved access to finance, including upskilling in financial literacy (outcome 3) [for example with 97% of the MSMEs in Latin America that attended trainings reporting improved financial literacy in accounting and reporting and also improved capacity to manage financial resources]; increased MSME access to innovation and technology (outcome 4) [including through capacity development on green technologies for SMEs in Southern Africa]; and increased access to local, regional and international markets through digitalization and non-tariff measures (outcome 5) [for instance with case studies on competition in Thailand, South Africa and Brazil, followed by webinars and the creation of an online course on SMEs and competition policy, primarily for government officials].

In relation to the project indicators, the assessment of their effectiveness was based on limited information, proxies, and expert judgement since they often lacked SMART criteria. It was possible to infer that 85% of them showed some level of progress, either through objective measures (e.g., sales increase) or proxies (e.g., post-training satisfaction assessments and survey responses). The high effectiveness results can be partly attributed to the conceptual framework provided by the EPF, as well as the cluster-based approach with each cluster addressing specific regional constraints and responding to demand from member States. In addition to clustering about 100 outputs from seven implementing entities into 5+1 outcomes, the EPF provided a coherent thread to relate outputs that cut across outcomes. Further analysis would be needed, however, to clearly identify which of those outputs are the most suitable or adaptable for a crisis response.

The project improved the capacity of policymakers in designing and implementing policies supportive of MSMEs, especially in terms of contributing to a country’s government responses, country-specific studies, technical assistance, training courses, and the development of digital tools and regional interventions. This improved capacity is reflected in beneficiaries’ reports of how they have been considering the needs of MSMEs, including those led by women and youth, in their daily work. These results could become more transformative and sustainable with more institutional support to networks of policymakers and communities of practice for mutual and regional learning on supportive policies for MSMEs, as well as by mainstreaming gender and Leaving No One Behind (LNOB) approaches in government officials’ capacity-building.

Sustainability

The project developed approaches, tools and capacities that are being transferred at multiple levels. At the policy level, the addition of more functionalities to e-government tools, expanding training courses to more countries, and the establishment of agreements to implement the project recommendations are all strong evidence of the sustainability of the project’s achievements. At the behavioural level, beneficiaries are applying new knowledge and skills to their daily work, be it by including MSMEs’ issues in policy making or by improving MSMEs’ management, performance, outputs and, by extension, resilience. Challenges to this transfer of

9

knowledge relate to the lack of financial resources, need for follow-up mentoring, and lack of a favourable business environment.

Gender, human rights, and disability

The overall project design was well aligned with the two key UN documents that provide clear direction for the integration of a human rights-based approach, which include addressing the aspect of gender equality and Leaving No One Behind. Although these aspects were well-articulated in the project documents, strong follow-through and documentation of their implementation was limited to a few project components. For those components that did include these aspects, gender was more commonly integrated and youth to a lesser degree. There was very little evidence of the outreach, participation, or impact of project activities for people with disabilities or other vulnerable groups. The UNCTAD component on Women and Cross-Border Trade targeting five countries in Africa was a good practice in how to design and implement this type of intervention with a strong reflection of the needs of some of the most vulnerable women across border areas.

Recommendations Recommendation 1: (i) UNCTAD should further leverage the experience gained through the project to map out how components of the Entrepreneurship Policy Framework relate to and can be best positioned to support MSMEs in their recovery from different types of crises and (ii) DESA and the Regional Commissions should add their analysis of how their work can contribute towards the objective. UNCTAD, DESA and the Regional Commissions could build on the project to map their areas of intervention in support of the MSME sector, as well as capacities and knowledge on the implementation of the EPF components, including in a crisis context, and opportunities for broadening the uptake of EPF components at the regional level based on national needs. UNCTAD, DESA and the Regional Commissions could further consider identifying areas of joint interventions that would trigger complementarities and synergies between the agencies. This could involve collaborating on the development of capacities of all national and sub-national actors, i.e. Governments, MSMEs, other partners (e.g. Chamber of Commerce, Business Incubators, etc.) in line with United Nations Sustainable Development Cooperation Frameworks (SDCF) in respective countries; strengthening or contributing to facilitate access to finance for the MSMEs (including seed money, grants, access to credit, etc.); improving the coordination of MSMEs related policies across ministries; increasing interventions at the local level, such as by supporting NGOs or MSMEs outside of the main cities. The application of a human rights- based approach, gender responsiveness and inclusion of other vulnerable groups (LNOB) should be ensured. Recommendation 2: UNCTAD should continue building on the momentum generated by the project to continue fostering knowledge exchanges and promoting the Entrepreneurship Policy Framework. UNCTAD, DESA and the Regional Commissions should identify means to more meaningfully continue to share good practices and lessons learned on the demand-driven support they provide to the MSME sector, including on areas such as green/circular economy; innovation policies and ecosystems; MSME formalization and strengthening MSME capacities for the empowerment and leadership of women, youth and groups in vulnerable situations. UNCTAD and the Regional Commissions could also propose organizing regional events and/or a global conference to promote entrepreneurship policy and further advance the goals and impact of the Surge project. Furthermore, learnings from the experiences with the Surge about external partnerships should encourage UNCTAD to promote the EPF and entrepreneurship development to other UN agencies also engaged in this area (e.g. ILO, FAO, UN Women, etc.), to development banks, or to related initiatives such as the recent Global Accelerator on Jobs and Social Protection. UNCTAD could also consider joining and supporting events that promote the achievement of SDG 8.3 as an avenue to further promote the EPF. Recommendation 3: The DA-PMT should develop a clear framework for assessing the costs and benefits of implementing a global or inter-regional project versus regional projects. As a global crisis can affect regions and countries differently, global or joint projects should be developed only when there are clear benefits of joint implementation. Thus, a global or inter-regional vs. regional response would not be a priori decision, but a result of a clear assessment. A specific guideline or framework should be developed for this. Some of the assessment criteria could include the need or demand from member States for an integrated response; the range of common versus entity-specific activities and added value of complementary interventions (i.e. in terms of geographical coverage, reach of the target population, capacity, and/or coverage of multiple reinforcing technical areas, or networks and partnerships); capability to execute joint interventions (e.g. in terms of time, resources, logistics, and flexibility to pivot interventions without bureaucratic procedures);

10

coordination costs (which increase with the number of participating UN entities); project inception modalities and governance and knowledge management requirements to facilitate collaboration and knowledge sharing; scaling and sustainability plans; expected results of spreading resources versus concentrating on fewer countries; and so forth. The time taken to conduct such assessments will increase project coherence and effectiveness and facilitate the identification of the financial and human resources needed for project implementation. The network of DA Focal Points is a key existing asset for this assessment. Recommendation 4: Implementing UN entities should ensure that they have a comprehensive Results Framework for the entire project as well as an adequate monitoring plan, with indicators that are designed to support the ongoing monitoring. The results framework should have one objective and ideally have one outcome per cluster/workstream. The objective should state the intended goal of the project, describe the overall achievement targeted by the project, involving a process of change aimed at meeting the needs of identified beneficiaries, and reflect the overall funding available to the project. Each objective should include reference to the project’s beneficiaries and its substantive focus. The objective should not attempt to explain the ways in which the project intends to achieve the objective (i.e. it should not include the word ‘through’ or describe the internal work of the UN using verbs such as ‘support’, ‘facilitate’ or ‘contribute’). The outcomes (OCs) should describe the changes that are expected to occur as a result of the completion of outputs. The OCs should be achievable within the project’s timeframe and budget, and should be specific enough to be measured by the associated indicators of achievement. The indicators of achievement (IAs) should provide measures for monitoring progress towards achieving the OCs and reporting on them after completion of the project. Every indicator needs to provide clearly defined baselines, units of measurement and targets, detailing the quantity, quality and timing of expected results. The monitoring, evaluation, accountability, and learning (MEAL) system should also be designed to capture HRBA, gender and LNOB aspects. In phased interventions or during project execution, any changes in the logframe if/when pivoting activities should be clearly explained to the wider team. Recommendation 5: Implementing UN entities should ensure that sufficient resources are allocated to project coordination, technical collaboration, and partnership building. The absorption capacity of implementing entities can be challenged by crisis response projects which add to the planned programme of work. This is further compounded by projects that come with an extensive UN partnership and a global scope. Sufficient resources should be dedicated to global coordination and to building global partnerships with strategic stakeholders (e.g., UN organisations engaged in supporting the MSME sector; development banks). Capacities should also be directed to supporting technical collaboration and the staff implementing interventions, including towards synergy or liaison with the UNCTs. When designing the project, implementing UN entities should consider featuring coordination and partnership-building in the Theory of Change or logframe of the project. Tools to support continuous connections and knowledge exchange, and to ensure institutional memory should be part of the response package, such as a project website, SharePoint space for all team members, and a Yammer network or Teams channel. The integration of cross-cutting aspects (HRBA, gender responsiveness, LNOB) also requires expertise with sufficient and dedicated time and resources. Guidance could be development to project managers on how to do this.

11

1. Introduction

1. This Evaluation Report presents the independent evaluation of the United Nations Development Account Project 2023W – “Global Initiative towards post-Covid-19 resurgence of the MSME sector” (the MSME Surge project or project). The objective of the project was to develop and implement capacity-building tools for governments and micro, small and medium enterprises (MSMEs) to facilitate their resurgence from the COVID-19 crisis and strengthen the resilience of MSMEs in developing countries and economies in transition. The project was implemented by UNCTAD, UN DESA and the UN regional commissions for Africa (ECA), Europe (ECE), Latin America and the Caribbean (ECLAC), Asia and the Pacific (ESCAP), and the Arab region (ESCWA). Project implementation started in May 2020 and ran until June 2022.

2. The final evaluation of the MSME Surge project aims to assess the results of the project and establish the link between the achievements and activities of the intervention; to assess the response delivery and external coordination of the project, including the extent of gender, human rights and disability mainstreaming; and to identify good practices and lessons learned from the project that could feed into and enhance the implementation of related interventions.

3. The evaluation started in February 2023 and concluded in September 2023. The evaluation conclusions and recommendations are intended to inform the work of UNCTAD and the implementing UN entities, as well as the UN Development Account Programme Management Team (DA-PMT).

2. Description of the Project

2.1 Background

4. The first diagnosed cases of ‘viral pneumonia’ became public on 31 December 2019. On 30 January 2020, WHO declared the coronavirus outbreak a public health emergency of international concern (i.e., WHO’s highest level of alarm) and soon after it launched a COVID-19 Strategic Response and Preparedness Plan (SPRP) and asked the UN Secretary-General to activate the UN crisis management policy2. COVID-19 was declared a “pandemic” on 11 March 2020. On 19 March 2020, the SG report “Shared Responsibility, Global solidarity: Responding to the socio-economic impacts of COVID-19”3 launched a call for action with a focus on people – women, youth, low-wage workers, small and medium enterprises, the informal sector and on vulnerable groups who were already at risk. The UN published the Global Humanitarian Response Plan (GHRP) for COVID-194 on 25 March 2020, initially calling for USD 2.01 billion. On 2 April 2020, the General Assembly adopted the Resolution on Global Solidarity to fight the coronavirus disease 2019 (COVID-19)5. On 5 April the COVID-19 Response and Recovery Multi-Partner Trust Fund (MPTF) was established to channel funds for a system-wide immediate socio-economic response to COVID-19. The UN Framework for the immediate socio-economic response to COVID-196 further set out the structure for the United Nations’ urgent socio-economic support to countries and societies in the face of COVID-19. The Framework presented five streams of work connected by a strong environmental sustainability and gender equality imperative to build back better. One pillar of the Framework focused on protecting jobs, supporting small and medium-sized enterprises, and informal sector workers through economic response and recovery programmes.

5. From its onset, the advent of COVID-19 confronted governments, the development and the humanitarian community and society with a deteriorating socio-economic context. COVID-19 affected poverty, trade and employment levels, and put at risk the achievement of the SDGs. The World Bank estimated the COVID-19 pandemic to have pushed an additional 119 million to 124 million people into extreme poverty

2 The UN Crisis Management Team brings together 23 UN entities to coordinate a comprehensive whole-of-UN response. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/interactive-timeline/#event-47. 3 https://www.un.org/sites/un2.un.org/files/sg_report_socio-economic_impact_of_covid19.pdf 4 UN OCHA. 2020. Global Humanitarian Response Plan for COVID-19. Geneva. 5 United Nations. 2020. Resolution adopted by the General Assembly on 2 April 2020. A/RES/74/270. General Assembly. New York. Link: https://undocs.org/en/A/RES/74/270. 6 UN. 2020. A UN framework for the immediate socio-economic response to COVID-19. April 2020. New York.

12

in 20207. The IMF estimated a 3.5 percent contraction in global GDP in 20208. UNCTAD reported on the use of trade policy instruments with trade-restrictive effects in over 180 countries, most of them in developing countries9. ILO estimated that 8.8 per cent of global working hours were lost in 2020 relative to the fourth quarter of 2019, equivalent to 255 million full-time jobs10. At the beginning of the pandemic, most of these job losses affected women, and, by August 2020, the participation of women in the labour force was over 15 per cent lower than the baseline – male participation was about 10 per cent lower11. Annual unmet SDG financing needs before COVID-19 were USD 2.5 trillion. The pandemic created additional needs in 2020 for USD 1 trillion in COVID-19 spending while the drop in external private resources for developing countries was USD 700 million12.

6. The COVID-19 crisis plunged the global economy into a deep recession and micro, small and medium enterprises were amongst the hardest hit, with decreased liquidity or cash flow availability, decreased demand for products and services, and decreased supply of inputs13. According to UNCTAD’s analysis of the World Bank data in selected countries, on average 13% of small-size firms were temporarily closed due to COVID-19 pandemic, almost 75% of SMEs experienced a decrease in demand for their products and services, over two-thirds of SMEs experienced decrease in supply of their inputs14. A survey implemented by UNECA and IEC Ltd of African MSMEs at the start of Covid-19 found that four-fifths of the survey respondents indicated being significantly affected by the current COVID-19 crisis (rating the effect as highly severe or severe). MSMEs make up 90 per cent of the economic engine of developing countries and are a major source of employment and self-employment, including for vulnerable groups, such as women, elderly and youth. Trapped in economic stagnation due to large-scale lockdowns, millions of MSMEs became the most vulnerable to COVID-19 within the private sector. At the sectoral level, demand and supply stagnation was seen in many industries. MSMEs are major players in non-essential services,15 which became the most affected sectors by lockdown and other isolation measures. In addition, many micro and small businesses are in the informal economy, which was hit first and hardest by the pandemic.

2.2 Project objectives and expected accomplishments

7. The main objective of the project was to develop and implement capacity-building tools for governments and MSMEs to facilitate the resurgence and strengthen the resilience of MSMEs in developing countries and economies in transition. These capacity-building tools sought to mitigate the economic and social impact of the global COVID-19 crisis and to facilitate the contribution of MSMEs to SDG implementation.

8. The initiative brought together UNCTAD, UN DESA and the UN regional commissions for Africa (ECA), Europe (ECE), Latin America and the Caribbean (ECLAC), Asia and the Pacific (ESCAP), and the Arab region (ESCWA) with funding provided by the UN Development Account (DA). The project expected to ensure both global reach and regional presence, international cooperation, and exchange of knowledge and good practices from all over the world. The project also aimed to establish a coherent approach towards MSME resurgence reflecting the holistic and comprehensive nature of efforts needed to ensure and facilitate MSMEs’ green, resilient and inclusive recovery.

7 https://blogs.worldbank.org/opendata/updated-estimates-impact-covid-19-global-poverty-looking-back-2020- and-outlook-2021 8 IMF. 2021. World Economic Outlook Update. January 2021. Washington. 9 UNCTAD. 2021. Impact of the COVID-19 pandemic on trade and development: Recovering, but unevenly - Situation as at 31 March 2021. https://unctad.org/programme/covid-19-response/impact-on-trade-and-development- 2021#aTradePolicy 10 ILO. 2021. ILO Monitor: COVID-19 and the world of work. Seventh edition. 25 January 2021. Geneva. 11 UNCTAD. 2021. Impact of the COVID-19 pandemic on trade and development: Recovering, but unevenly - Situation as at 31 March 2021. https://unctad.org/programme/covid-19-response/impact-on-trade-and-development- 2021#aTradePolicy 12 OECD. 2020. Global Outlook on Financing for Sustainable Development 2021. Paris. 13 UNCTAD. 2021. Impact of the COVID-19 pandemic on trade and development: Recovering, but unevenly - Situation as at 31 March 2021. https://unctad.org/programme/covid-19-response/impact-on-trade-and-development- 2021#aTradePolicy 14 UNCTAD. 2021. Global Initiative towards post-Covid-19 resurgence of the MSME sector. MSME surge project - Addendum Phase 3 Project Proposal. Geneva. 15 The exact definition of these services varies by country, but they are generally recreational businesses, such as accommodation, catering, entertainment and tourism.

13

2.3 Project strategies and key activities

9. The project followed the conceptual approach of clustering interventions under the overall umbrella of the Entrepreneurship Policy Framework (EPF)16 of UNCTAD. The five clusters addressed the most critical areas of MSME recovery:

1- Mobilize entrepreneurial ecosystem and strengthen business skills 2- Simplify business registration and facilitate formalization 3- Improve access to finance/financial literacy 4- Increase access to technology and innovation 5- Enhance access to markets

10. Activities were to be delivered through coordinated and complementary efforts of participating agencies covering key interrelated components of an enabling entrepreneurship ecosystem, such as improving a regulatory entrepreneurship environment and mobilizing entrepreneurship potential, enhancing MSMEs’ access to innovation and technology, finance and markets, including at a policy level and at MSME level, and reaching out to most affected target groups, including women and informal workers. The project also included a number of activities to facilitate the green and sustainable recovery of MSMEs.

11. The project started its implementation in May 2020 and was scheduled to conclude on 31 March 2022, but received approval in February 2022 for an extension until 30 June 2022. The project was developed and implemented under three phases:

• Phase 1 (May-June 2020): The start-up of the project was approved by the DA Steering Committee based on a concept note presenting an overview of the project intended to be implemented over an 18-month period (to December 2021), the intended outcomes for Phases 1 and 2, and the detailed budget for phase 1 only.

• Phase 2 (July-December 2020): The Project Document submitted for approval of Phase 2 was expected to cover Phases 2 and 3, with the expectation that it would be revised in September 2020 to include the finalized Phase 3 activities and budget, and the project’s monitoring and evaluation plan, for Phase 3 approval.

• Phase 3 (initially January-December 2021, extended twice, first to March 2022 and then to June 2022): The addendum to the Project Document focused interventions on scaling up the implementation of the online outputs developed in Phases 1 and 2 and launching new activities in selected countries based on demand and results in the previous phases.

12. Over its 3 phases the project implemented more than 290 activities (Figure 1). Activities were sometimes multi-faceted, for example with reports being used to feed training materials, conferences, or webinars. Activities involved varying levels of effort, from one-time webinars to reports or online platforms requiring several months of development.

Figure 1: Overview of project activities (indicative17).

16 Entrepreneurship Policy Framework and Implementation Guidance | UNCTAD 17 Some activities were described by the UN entities with interchangeable terms, such as webinars and trainings, while some other activities were not necessarily described, such as disseminating a report through a workshop. The evaluation reconstructed the number of activities based on a review and interpretation of the project logframe.

14

Source: Independent evaluation, 2023.

13. The Project Document did not formulate a Theory of Change (TOC) but presented a results framework that the evaluation used jointly with inputs provided by informants to reconstruct a simplified (draft) TOC at evaluation (Annex 1).

14. According to the Project Document (Phase 2), the monitoring arrangements planned for the project included:

● Bi-monthly online meetings of the Project Steering Committee: Representatives from all seven entities were sought to monitor project delivery to ensure smooth and steady implementation. Representatives were also expected to meet on an as-need basis for important decisions that affected all seven entities.

● Inputs and feedback from the Project Design Team: Representatives from UNCTAD, DESA, ECA, ECE, ECLAC, ESCAP and ESCWA, who participated in the project design and its adjustments as needed, in accordance with the COVID-19 situation, were considered to be a key part of the project to evaluate and act upon stakeholder commentary.

● Task Forces: Task Forces comprised of experts from the partner entities were to be established to provide ongoing coordination of work of the five clusters.

● Annual progress reports: Annual or phase-based progress reports were to be provided to the DA- PMT on all aspects of project implementation.

● Success stories: based on request and template provided by the DA-PMT.

2.4 Beneficiaries and target countries

15. The main direct beneficiaries targeted by the project included: Governments, policy makers, Micro-, small and medium enterprises (MSMEs), Business or industry associations, Business support services, Commercial Registry Offices, Technical/experts staff in relevant institutions and ministries, and Multinational Enterprises (MNE).

7

101

29

90

40

17

8

2

0 20 40 60 80 100 120

Online systems and platforms (e-Registration systems, knowldge hubs, websites, search engine, etc.)

Trainings (Workshops, online trainings, online toolboxes, etc.)

Webinars (Global, regional, national; series or unique events)

Reports (Books, Assessments, Policy Reports, Guidelines, Case Studies, Maps, etc.)

Online training tools (e-Learning programmes, course materials, self-training, etc.)

Conferences (Meetings, roundtables, inter-governmental discussions, online events, policy dialogues, etc.)

Technical assistance (Entrepreneurship strategies, national policies, etc.)

Networks (Community of Practice, Policy Network)

The project conducted more than 290 activities

15

16. The beneficiary countries covered different geographical regions. The project compiled a list of 96 beneficiary countries in the final report, as shown in Table 1. These countries were supported by the Surge project, to varying degrees, either through physical/in-person activities or through online modalities. Some of the forms of interventions (in-person or online) included advisory services, trainings, workshops, seminars, conferences and events, platforms and systems. In many cases, implementing entities also provided direct technical assistance to countries through policy advice or in the form of assessment reports, research studies, or tailored guidelines18. Project partners organized online seminars targeting specific countries and also regional or global audiences in many cases.

Table 1: Project’s beneficiary countries.

Africa

1. Algeria 2. Angola 3. Benin 4. Botswana

5. Cameroon 6. Egypt 7. Eswatini 8. Ethiopia

9. Ghana 10. Kenya 11. Lesotho 12. Malawi

13. Mauritius 14. Mozambique 15. Namibia 16. Nigeria

17. Seychelles 18. South Africa 19. Tanzania 20. The Gambia

21. Uganda 22. Zambia 23. Zimbabwe

Americas

24. Argentina 25. Bahamas 26. Barbados 27. Belize

28. Bolivia 29. Brazil 30. Chile 31. Colombia

32. Costa Rica 33. Cuba 34. Dominica 35. Dominican Republic

36. Ecuador 37. El Salvador 38. Grenada 39. Guatemala

40. Guyana 41. Jamaica 42. Mexico 43. Nicaragua

44. Panama 45. Paraguay 46. Peru 47. Saint Kitts and Nevis

48. Saint Lucia 49. Trinidad and Tobago 50. Uruguay 51. Venezuela

Asia

52. Afghanistan 53. Armenia 54. Azerbaijan 55. Bangladesh

56. Bhutan 57. Brunei Darussalam 58. Cambodia 59. China

60. Georgia 61. India 62. Indonesia 63. Islamic Republic of Iran

64. Jordan 65. Kazakhstan 66. Kyrgyzstan 67. Lebanon

68. Lao PDR 69. Maldives 70. Malaysia 71. Mongolia

72. Myanmar 73. Nepal 74. Pakistan 75. Palestine

76. Philippines 77. Saudi Arabia 78. Singapore 79. Sri Lanka

80. Tajikistan 81. Thailand 82. Türkiye 83. United Arab Emirates

84. Viet Nam

Europe

85. Albania 86. Belarus 87. Bosnia and Herzegovina 88. Montenegro

89. North Macedonia 90. Republic of Moldova 91. Romania 92. Russia

93. Serbia

Oceania

94. Fiji 95. Papua New Guinea 96. Tuvalu

18 Some of the countries that received direct technical assistance include for example Armenia, Bangladesh, Benin, Bosnia and Herzegovina, Botswana, Brazil, Cameroon, El Salvador, Georgia, Kazakhstan, Kenya, Kyrgyzstan, Malawi, Mali, North Macedonia, Republic of Moldova, Serbia, Tajikistan, Tanzania, Thailand, The Gambia, Ukraine, Zambia.

16

2.5 Key partners and other key stakeholders

17. Jointly implemented by UNCTAD, DESA, ECA, ECE, ECLAC, ESCAP and ESCWA, the project built upon the comparative advantages and mandates of the participating agencies to provide immediate advice, capacity-building and support to governments and MSMEs during the ongoing global pandemic. Broadly, the roles and lead entities, based on the mandates and expertise of the participating agencies, for each of the five project clusters and workstreams were as presented in table 2.

Table 2: Project clusters and lead and participating entities. Source: Project document Phase 2.

Project cluster/workstream UNCTAD DESA ECA ECE ECLAC ESCAP ESCWA

Overall coordination19 O Project cluster/workstream 1 Entrepreneurship and business skills promotion

O O O O

Project cluster/workstream 2 Business facilitation/formalization

O O O

Project cluster/workstream 3 Access to finance/financial literacy

O O O

Project cluster/workstream 4 Access to technology and innovation

O

Project cluster/workstream 5 Access to markets

O O O O O O

O: Lead institution O: Participating institution

18. UNCTAD was the lead project coordinating entity20. Day to day management of the project was entrusted to each implementing agency for its own component. Each participating agency managed and implemented its component with funds allocated to it. Within UNCTAD, project oversight was ensured by UNCTAD’s Enterprise Branch within the Division on Investment and Enterprise.

19. Key project partners included the UN Development Partners with the UN Resident Coordinators (UNRCs), UN Country Teams (UNCTs), UNITAR, UNDP, UNOSSC, ITC. The project was also implemented with the EMPRETEC Centres21. Other key stakeholders included technical/expert staff in relevant institutions, ministries, and business incubators and support services.

2.6 Resources

20. Overall, the United Nations Development Account (UNDA) allocated a total of $4,490,500 under this project. Concretely, UNCTAD received $2,671,000, ECLAC received $134,000, ESCAP received $240,000, ECA received $467,000, ECE received $448,500, ESCWA received $370,000 and DESA received $160,000 (Figure 2).

Figure 2: Allocation of project funding.

19 Phase 1 of the project was co-led by UNCTAD and ESCWA. 20 UNCTAD and ESCWA co-led phase 1 of the project. 21 EMPRETEC is a flagship capacity-building programme of UNCTAD for the promotion of entrepreneurship and micro, small and medium-sized enterprises (MSMEs) to facilitate sustainable development and inclusive growth.

17

Source: Final report.

21. The project budget was approved and distributed by phase (Table 3). The budget for phase 1 was $325,000 and the budget for phase 2 was $1,310,000. Budget for phase 3 was reduced in November 2011 from a provisional $3,165,500 to $2,855,500 (see section 5.3).

Table 3: Project's implementation phases

Phases Implementation period Effective budget allocations

I May-June 2020 $325,000

II July-December 2020 $1,310,000 III January 2021-June 2022 $2,855,500

2.7 Link to the Sustainable Development Goals (SDGs)

22. The project aimed to contribute to the post-2015 development agenda by making direct contributions to the sustainable development goals (SDGs), in particular goal 4 on “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” (target 4.4); goal 8 on “Promote development- oriented policies that support productive activities, decent job creation, entrepreneurship, creativity and innovation, and encourage the formalization and growth of micro-, small- and medium-sized enterprises, including through access to financial services” (target 8.3); and goal 9 on “Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation” (targets 9.3 and 9.5).

2.8 Innovative elements

23. The project demonstrated several innovative elements. At the institutional level, the project relied on increased consultative processes and a phased approach which enabled adaptive management. At the programmatic and technical levels, the project contributed to the introduction of MSMEs into policy discussions in response to the pandemic crisis, and e-government services. At the operational level, the project developed innovative approaches and tools to respond to country needs, such as online delivery of trainings and knowledge sharing events.

3. Evaluation objectives, scope, and questions

3.1 Purpose and objectives

$160,000

$370,000

$448,500

$467,000

$240,000

$134,000

$2,671,000

$0 $2,000,000 $4,000,000

DESA

ESCWA

ECE

ECA

ESCAP

ECLAC

UNCTAD

3.56%

8.24%

9.99%

10.40%

5.34%

2.98%

59.48%

DESA

ESCWA

ECE

ECA

ESCAP

ECLAC

UNCTAD

0% 20% 40% 60% 80%

18

24. As per the TOR (Annex 7), the final evaluation of the MSME Surge project has the following specific objectives:

● Assess the results and establish the link between achievements and activities of the intervention;

● Assess the response delivery and external coordination22, including the extent of gender, human rights and disability mainstreaming; and

● Identify good practices and lessons learned from the project that could feed into and enhance the implementation of related interventions.

3.2 Evaluation scope, criteria, and questions

25. The evaluation covers the duration of the project from April 2020 to June 2022. The evaluation focuses on the countries covered by the project. The evaluation analysed unintended positive (or negative) outcomes beyond the 96 countries identified by the project as beneficiary countries. The evaluation conclusions and recommendations are intended to inform the work of UNCTAD and other UN implementing entities as well as the Development Account Programme Management Team (DA-PMT), which will conduct a synthesis from this evaluation and those of the other four Covid-19 surge projects. The evaluation took a retrospective and forward-looking approach. It was summative but also formative aiming to support UNCTAD and UN partners in their next steps.

26. The evaluation addressed the questions in table 4. The evaluation questions were unpacked into an evaluation matrix with suggested measures (Annex 2) and guided data collection methods (Annex 3).

Table 4: Evaluation questions.

CRITERIA KEY QUESTIONS

RELEVANCE

1. To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID-19?

2. To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g., COVID-19 Socio-Economic Response Plan)?

COHERENCE

3. To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

4. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID-19 in delivering socio-economic support to Member States?

EFFICIENCY

5. How well coordinated was the process for the response among the entities implementing the joint project?

6. How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

EFFECTIVENESS

7. To what extent did the programme (Development Account) and project governance and management structures and processes enable, or hinder, the effective implementation of the joint project and the achievement of its results?

8. To what extent has the project contributed to the expected outcomes as enunciated in the project document?

9. How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

10. What innovative approaches or tools, if any, did the response use, and what were the outcomes and lessons learned from their application?

22 The OIOS COVID-19 response evaluation protocol identifies the following three cross-cutting focus areas: 1) response delivery; 2) external coordination (or “Delivering as one”); and 3) business continuity. “Response delivery” is further defined as consisting of delivery of: 1) the existing mandate needed to implement previously mandated activities in the new environment created by the pandemic; and 2) the COVID-19 specific response (health and non-health) needed to address the pandemic specifically. See OIOS (October 2020), “COVID-19 Response Evaluation Protocol”, para 3-4.

19

SUSTAINABILITY 11. What measures were adopted to ensure that outcomes of the response would

continue after the project ended? GENDER, HUMAN RIGHTS AND DISABILITY

12. To what extent were perspectives on gender equality, protection of human rights and reaching underserved groups integrated into design and implementation of the project? What results can be identified from these actions?

4. Methodology

27. The evaluation collected and analysed data from a range of sources to deepen understanding and triangulate the assessment. The following data collection instruments were used:

• Desk review: Study of secondary resources (Annex 4) as per the project management process and logframe in order to validate achievements, including UN GA resolutions, UN strategies and policies, documents/data related to project-level planning, implementation and results achievement (including post-training assessments, whenever data allowed23), project outputs and monitoring reports, and external reports (Annex 2).

• Interviews and focus groups: Interviews were conducted with a selected number of staff, partners, and stakeholders (Annex 5). Contact details of UN staff and partners were provided by UNCTAD (38 target informants), ECLAC (10), UNECE (10), UNECA (8), DESA (5), ESCAP (5), ESCWA (3). All prospective informants were invited to inform the evaluation, with at least one follow-up reminder message sent in absence of any response. The evaluation interviewed 58 informants. The canvas for semi-structured interviews was tailored to ensure specific relevancy to the selected stakeholders. To optimize time, a few interviews took the form of focus groups. Consultations were conducted virtually with Zoom or Teams. The following consultations per stakeholder group were conducted (Table 5):

Table 5: Groups of informants and number of consultations.

Informant groups Number of informants

Project Coordination Team 10 Project focal points of implementing entities (i.e. members of Task Forces) 12

DA-Project Management Team and DA focal points in participating entities 8

UN and Development Partners (UNCT members, UNDP) 3

Governments (Ministries, national institutions) and policy makers 6

MSMEs, Business or industry associations, Commercial Registry Offices 8

Technical/experts staff in relevant institutions, including EMPRETEC Centres 11

Total 58

• Survey: The evaluation carried out six external surveys (Annex 6) to inform the assessment of the relevance, coherence, and effectiveness of the project and its contribution to outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups. The surveys targeted participants to project activities and beneficiaries. The lists of contacts were provided by UNCTAD, DESA, ESCAP, ECLAC, ECA, and ECE (Table 6)24. The sampling strategy was convenience sampling. In addition, in consultation with UNCTAD Independent Evaluation Unit, the evaluation used purposive sampling to exclude from the lists provided by the UN entities (i) the staff from the same UN entities who were part of those lists as having attended or contributed to the events; (ii) participants to Trainings of Trainers workshops as their role was to train project beneficiaries; (iii) participants to one-time only webinar sessions as having had very limited exposure to the project.

23 Not all post-assessment data could be used by the evaluation team, because some were responded by a very small number of training participants, in relation to the total number of participants in a given training. 24 ESCWA was requested to provide a list of survey recipients but did not share one.

20

The surveys were composed of questions that were common to all six questionnaires and some specific to each of the 5+1 project outcomes25. The questionnaires were made available in English. They were translated in Spanish for outcomes 3 and 5 as target recipients were primarily in the Latin America and Caribbean region. The surveys were anonymous and remained open for 2 weeks, from Thursday 20 July to Friday 4 August. Two reminder messages were sent to increase the response rate. The surveys were launched to a combined list of 1454 persons from at least 120 countries (information on the localisation of survey recipients was not available for all recipients), with 89 messages bouncing back. Altogether, the surveys compiled feedback from 133 respondents. Responses were received from at least 53 countries (some participants did not indicate their location). Responses were received from at least 62 males and 42 females (some participants did not indicate their gender). All survey questions were optional. Questionnaires partially completed were kept in the batch of results when they contained information that was judged credible and meaningful. Statistics were calculated on the basis of the number of valid responses per question and not on the basis of the overall number of respondents to the surveys. The overall response rate to the email surveys is circa 9.7%. The surveys presented the opinion of those who responded but not of the entire list of recipients of the questionnaires nor of all the beneficiaries of the project.

Table 6: Survey recipients and respondents.

Outcome Email addresses Bounced back Valid email Valid responses Percentage

1A 574 38 536 52 9.70% 1B 258 24 234 40 17.09% 2 21 0 21 4 19.05% 3 219 4 215 11 5.12% 4 98 1 97 13 13.40% 5 284 22 262 13 4.96%

Total 1454 89 1365 133 9.74%

• Cross-cutting thematic analysis on human rights, gender and inclusion: As an integral part of the above-mentioned methods of data collection, the evaluation team explored the extent to which gender equality, human rights and leaving no one behind perspectives were considered in the design and implementation of the project. The UN framework for the immediate socio-economic response to COVID-1926 guided the assessment as it outlines how the responses to the pandemic should be consistent with international human rights standards, ensure gender equality and special measures to protect the most vulnerable and marginalized groups, such as women, youth, and migrants.

28. The evaluation used a combination of complementary tools for analysis of the data collected.

● Qualitative analysis of data, secondary resources and interviews/focus groups. The evaluation used ATLAS.ti for coding qualitative inputs.

● Quantitative analysis including systematizing post-training assessments, indexing and cross- tabulations of survey data, and quantitative coding of survey qualitative inputs.

29. The evaluation confronted several constraints or limitations that were mitigated as follows:

● Remote data collection: The evaluation did not include country visits that would allow for face- to-face interviews and direct observations, which may have limited the collection of evidence

25 The project outcome 1 was subdivided into outcome 1A and 1B. 26 A UN framework for the immediate socio-economic response to COVID-19, April 2020 https://unsdg.un.org/sites/default/files/2020-04/UN-framework-for-the-immediate-socio-economic-response-to- COVID-19.pdf

21

at the outcome level. Mitigation: Use of complementary data collection methods to enrich and triangulate findings.

● Limited availability of informants: The interviews (individual and group) are a key instrument for this evaluation. Given turnover and time since the project was completed, some country staff were not available for interviews as they, for example, had retired or left the organization, did not recall exactly what happened, or had too much work or competing demands for their time. Mitigation: Support was requested from the Project Manager for securing interviews.

● Lack of generalisation of survey results: Convenience sampling cannot ascertain that survey recipients were statistically representative of the entire community of project beneficiaries. Furthermore, reminder messages mitigated but did not prevent any non-response bias. Respondents also required to have internet access. The surveys presented the opinion of those who responded but not of all the beneficiaries of the project. Surveys were used as one among other sources of findings.

● Limited evaluability of some indicators: Because some indicators do not comply with the SMART criteria and are elaborated in a broad way, it was difficult to identify the evidence that answers to them. Additionally, they were associated with the work of specific implementing entities, rather than the project as a whole. This posed an additional challenge, especially when the entity associated with a certain indicator was not clear. Mitigation: Targeted consultation about these indicators, corresponding evidence directed at the responsible entity, and an inductive approach to associate results achieved within an outcome to specific indicators.

● Too short a time between end of project and the evaluation to observe impact: Since most of the project’s activities take time to show impact, this evaluation reports on indications of possible future impact on beneficiaries, governments, and partners. Mitigation: Self-reported survey for beneficiaries, and data triangulation generated this evidence.

5. Findings

30. This section presents the evaluation findings per evaluation criteria and question.

5.1 Relevance

To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID-19?

31. The project was designed to respond to demands of Member States to mitigate the effects of the COVID-19 pandemic. The project was based on ownership by beneficiary countries and relevance towards national and sub-national sustainable development priorities. A demand-driven approach was mainstreamed throughout the project lifecycle. The project was designed based on the request to UN Secretariat entities for assistance for MSMEs from Member States, as well as intergovernmental demands and resolutions on COVID- 19. The Project Document for phase 2 reported requests for support received from more than 70 countries across the 5+1 project outcomes. For example, the Governments of El Salvador, Benin, Cameroon and Mali, requested UNCTAD to create online services that streamlined and simplified business registration. The Governments of Malawi, Tanzania and Zambia, requested UNCTAD to scale-up the format of the cross-border trade trainings piloted in these countries and to replicate them at other borders in the same countries. Georgia and North Macedonia expressed interest for ECE to develop country-specific reports based on the findings of the regional Guidelines and Best Practices for MSMEs in delivering energy efficient products and in providing renewable energy equipment. Several staff from the implementing UN entities also mentioned that in-country interventions were conditioned by the receipt of a Letter of Agreement from Member States. In a few cases, informants further illustrated the demand-based approach of the project by indicating that some initial plans for collaboration were parked sometimes due to changing country needs and priorities.

32. The project conducted or contributed to needs and impact assessments to inform interventions. Across all regions the project collaborated with Member States and national stakeholders to conduct surveys, needs assessments, or impact studies that informed project design and implementation as well as national responses. In South-east and Central Asia, and the Pacific, countries such as Samoa, Bangladesh, Nepal,

22

Cambodia and Kazakhstan participated in ESCAP in-depth country assessments. In the Southern African region, ECA carried out a survey across its 11 member states to assess the impact of Covid-19 and the war in Ukraine on MSMEs in association with national business organizations, The survey gauged policies needed to foster regional integration and technology and innovation in post Covid-19 recovery strategies. ECE produced a rapid impact assessment series on end-to-end supply chain activities to support long-term development efforts for five beneficiary countries and presented them for intergovernmental discussion. ECLAC ran diagnostics on competition policy and consumer protection, and on SME support policies, to identify needs for project activities or products in target countries, including the needs of women and vulnerable groups. In Kenya, DESA and the Kenya Micro and Small Enterprises Authority assessed how MSMEs were affected by the pandemic. The survey results were disseminated to stakeholders, who were brought together to participate in a validation workshop to discuss most needed support systems and formulate policy advice to be shared with the Ministry of Cooperatives and MSME Development. In several cases informants also recalled that needs assessments are part of the project design methodologies implemented by UN entities. For example, the first step of the EPF is to assess the status of the national entrepreneurial ecosystem. According to the evaluation survey, the vast majority of respondents reported that project activities have responded to their priorities and were relevant to their work (Figure 3).

Figure 3: Perceived relevance of the project according to the survey (n=133 respondents).

Source: Independent evaluation survey, 2023.

33. Stakeholders, including policymakers, MSME entrepreneurs and development partners, participated in project design and implementation. On various occasions, evaluation informants stressed that the involvement of Member States and stakeholders in designing and implementing interventions was also a factor contributing to addressing their needs and priorities. In Kenya for example, in addition to ensuring buy-in from the Government before implementing cross-border trade trainings, UNCTAD relied on national institutions to facilitate project delivery on the ground. UNCTAD asked the Government to identify training participants and the Government requested local associations to identify participants and to help with logistics. A representative of the Ministry of Trade of Kenya travelled to the border to introduce the workshops. This was the first instance of high-level government support to the project. Several Empretec centres also mentioned that the identification of workshop participants was done in collaboration with government agencies and departments. In Zimbabwe, the Empretec centre selected primarily businesswomen and youth whose businesses were declining or faltering. Another example involved the creation of training materials on Core Indicators for Sustainability and SDG Reporting (GCI). A series of trainings of trainers (TOT) were organized by UNCTAD and participants’ feedback was used to revise the materials before training entrepreneurs.

34. Project activities and products were tailored to Member States and target beneficiaries. Project activities and products were often tailored to the needs of target recipients. Many regional reports were localized through national studies. For instance, ECE developed guidelines and best practices for MSMEs to

41%

44%

11%

1% 1%

1% 1%

The project activities responded to my priorities and were relevant to my work

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

23

assure resiliency and progress towards a circular economy in sustainable resource management and critical raw material supply chain solutions, as well as guidelines and best practices for MSMEs in delivering energy- efficient products and in providing renewable energy equipment27. Four countries of the ECE region were chosen as pilot countries for which the guidelines were customized, before spanning to other countries. In Tanzania, the Empretec centre delivered a training on entrepreneurship skills development that was based on the UNCTAD-EMPRETEC methodology but was adjusted to participants with limited literacy skills. UNECA delivered an online training course on the role of technology and innovation for MSMEs under Phase 2 and feedback from the training was used to develop a second course on role of green technologies in MSME development under Phase 3. For many activities and products, UN entities also ensured the translation of materials in official UN and local languages to facilitate access and outreach at global, regional and national levels. In the early months of the response, ECE partnered with UNDP (which had a Zoom license) to deliver workshops with simultaneous interpretation as this could not be accommodated by ECE online platforms.

35. Implementing UN entities adapted to maximize the relevance of interventions aimed for longer term change. While some Surge project outputs were found to be more crisis-response driven and with a shorter path to outcomes, such as the implementation of e-registration systems, other activities such as the design of a national entrepreneurship strategy were referred to as yielding an impact after a longer period of time. This is typical of DA projects and other socio-economic UN initiatives. Several interviewees were of the opinion that the UN socio-economic pillar did not generally have many crisis response products. Despite the achievements of the project (see section on Effectiveness), the capacity of the UN to respond to the needs and priorities of participating countries as a result of COVID-19 depended on institutional factors that were not always under their direct control. It was further indicated that at the onset of the pandemic, countries needed immediate financial support to help their companies and MSMEs to stay alive. Several survey respondents reported that complementing capacity-building activities for entrepreneurs with grants would have been beneficial28. Although the EPF and cluster-based approach proved an appropriate conceptual framework to articulate the response (see section on Coherence), the latter could have benefited from some prior analysis of the relief-recovery-development pathway to facilitate the identification of appropriate crisis mitigating interventions. Implementing UN entities therefore used adaptive management to ensure adequacy of the services in addressing demand-orientation and longer-term outcomes.

To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g., COVID-19 Socio-Economic Response Plan)?

36. The project was aligned with the COVID-19 Socio-Economic Response Plans (SERPs) in many participating countries, but the design occurred before the SERPs were finalized. The project was part of the general strategic plan of the United Nations called “A UN framework for the immediate socio-economic response to COVID-19”29 and specifically contributed to Pillar 3 “Economic Response and Recovery: Protecting Jobs, Small and Medium-Sized Enterprises, and Informal Sector Workers” of this system-wide framework30. This UN framework also served to guide the COVID-19 Socio-Economic Response Plans (SERPs) which were developed by UN Country Teams and Member States and stakeholders. However, while the concept note of the project was released in April 2020 and the Project Document for phase 2 in June 2020, many SERPs were published later, i.e. in or after August 202031. The design of Phase 3 of the project was finalized in early 2021 but most outputs were a continuation of previous work. Several informants also mentioned that the implementing UN entities have limited country presence, reducing the capability to mainstream the project in the SERPs. Despite these limitations, the evaluation reviewed a purposeful sample of 10 SERPs32 showing their systematic coverage of pillar 3 of the UN framework and some frequent references

27 UN Development Account project | ECE 28 See for example: Training and grant support for potential entrepreneurs and start-ups in the Sea of Azov region, Ukraine (ilo.org) 29 https://unsdg.un.org/sites/default/files/2020-04/UN-Framework-for-the-immediate-socio-economic-response-to- COVID-19.pdf 30 The UN framework built upon the UN General Assembly’s resolution on “Global solidarity to fight the coronavirus disease 2019 (COVID-19) (A/RES/74/270)” and on the Secretary-General’s report on “Shared responsibility, global solidarity: Responding to the socio-economic impacts of COVID-19”. 31 UNSDG | Resources Library 32 The evaluation selected 2 SERPs per region and countries where the project implemented interventions, i.e. Argentina, Bangladesh, Bhutan, BiH, Cameroon, Jordan, Moldova, Peru, Thailand, and Zimbabwe.

24

to or links with the outcome areas of the project, with a prevalence of connections with outcomes 4, 3 and 1A. The evaluation also found references sometimes to the project outputs. For example, the Socio-Economic Response and Recovery Plan for the Republic of Moldova (June 2020)33 was informed by the ECE’s survey- based assessment of the COVID-19-induced economic crisis and changes in non-tariff measures to contain the pandemic on MSMEs (May 2020) and by the Assessment on the impact of the crisis on female-owned enterprises (June 2020).

37. Independently of the SERPs, implementing UN entities considered the COVID-19 socio-economic responses of participating countries during project design and implementation. As noted in the earlier section, the project delivered many regional and national needs assessments and impact studies, which were a source for aligning project activities with the COVID-19 socio-economic responses of the participating countries. For example, UNECA organised, at the request of the Government of Mauritius and in association with the SADC Business Council, a regional seminar on the impact of Covid-19 on MSMEs in Southern Africa and the recommendations were intended to support the development of a roadmap for technical assistance at a regional level. In Latin America, ECLAC analysed the policies implemented by governments to face the MSME crisis generated by COVID-19. Experiences in countries having MSME fostering institutions were prioritized. Good practices were systematized and shared among public institutions in charge of MSMEs policies. Virtual meetings were organized to share implementation experiences about measures to help MSMEs in order to enhance the coordination of public policies for economic recovery. In the Gambia and Kenya, DESA ensured that the design and implementation of project activities were aligned with the national agendas to contribute to the relevance of the interventions and strengthen ownership by the countries. In the Gambia, work was carried out with the Ministry of Trade, Industry, Regional Integration and Employment (MOTIE), UNDP and the UNRCO. Project implementation served to strengthen the capacity of policy makers and other stakeholders to implement new policies and to pursue coherence among the already formulated policies in the country. Several evaluation informants indicated the project was relevant for bringing MSMEs to the policy debate where they were often absent.

5.2 Coherence

To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

38. The project was anchored in the mandates, comparative advantages, and programme budgets of the implementing UN entities, paving the way to complementarities with previous interventions. The initial concept note and Project Document presented a detailed review of the respective mandates and comparative advantages of the implementing UN entities as a rationale for their contribution to the project34. Among these

33 MDA_Socioeconomic-Response-Plan_2020.pdf (un.org) 34 Many factors were put forward to stress the institutional coherence and rationale for the project. Some highlights include: (i) UNCTAD is a focal point in the UN on entrepreneurship and MSME policy as formally mandated by two United Nations General Assembly resolutions on Entrepreneurship for Sustainable Development. UNGA resolutions /RES/71/221 and A/RES/73/225 both call on “the United Nations system, and in particular the United Nations Conference on Trade and Development, to continue to provide support to and assist member States, at their request, to identify, formulate, implement and assess coherent policy measures on entrepreneurship and the promotion of micro-, small and medium-sized enterprises”. UNCTAD also provides technical assistance and capacity-building support to a number of developing economies, including advice and training to policy makers to inform and guide the development and implementation of national entrepreneurship policies based on UNCTAD’s Entrepreneurship Policy Framework (EPF), which was endorsed by member states through a series of intergovernmental discussions and formally launched at the Ministerial Conference UNCTAD XIV in 2012. (ii) DESA brings relevant expertise from implementing the project “Enhancing national capacities for enhancing potentials of MSMEs in achieving SDGs in developing countries (MSME project)” funded by the UN Peace and Development Fund (PDF). (iii) ESCWA has developed and launched an SME information portal for the region with an aim to support entrepreneurs and small businesses access information needed to support their business. (iv) ECE has conducted rapid impact assessment of the COVID-19 pandemic on end-to-end supply chain activities, with a view to supporting national and regional recovery and long-term development efforts. (v) The ECA sub-regional office for Southern Africa leverages its mandate and implementation of a UNDA 13th tranche project for integrating a component to build capacities of MSMEs in Southern Africa to harness technological applications to address the impact of the pandemic. (vi) ECLAC has a long experience in studying the MSME performance in Latin America and in supporting public institutions in developing, implementing, and evaluating MSME fostering policies. (vii) ESCAP responds to ESCAP Resolution 70/5: Strengthening regional cooperation and capacity for enhanced trade and investment in support of sustainable development, and has planned to work with existing initiatives and organizations to support MSMEs in the region.

25

enabling institutional factors, references were primarily made to supporting policy makers on MSME development as well as to building capacities of MSMEs and entrepreneurs. Significant evidence was found showing that the project was consistent with and sometimes a continuation of previous activities. UNCTAD for instance relied on its network of Empretec Centres to deliver activities on the capacity-building of entrepreneurs. In collaboration with centres and experienced Empretec trainers, UNCTAD delivered on-line trainings and developed demand-driven, customized tools to assist the centres in coping with short-term and medium-term pandemic-induced challenges to their business operations. UNCTAD also built on earlier contacts to support the establishment of e-Registration platforms in El Salvador and Benin, which was accelerated by the pandemic. ESCWA grounded its proposed contribution to the 1st phase of the project in previous meetings and surveys. ECLAC delivered courses that were the result of prior studies and a long preparatory phase of getting to know the issues of SMEs in e-commerce. ESCAP’s “Bangladesh Startup Ecosystem Assessment Report”35 delivered by the Trade, Investment and Innovation Division followed earlier work from another division.

39. Interventions were often coordinated with and complementary to other internal activities that were underway in the implementing UN entities. Within each implementing UN entity, internal complementarity was sought with other projects. ECE for example used the impact assessments on the novel COVID-19 pandemic on end-to-end supply chain activities, conducted under UNDA 10 “Strengthening the national capacities of selected ECE countries for evidence-based regulatory and procedural trade policies to achieve SDGs”, to further draw two national assessments in Armenia and Moldova focusing on female-owned enterprises. DESA leveraged its expertise and understanding of country needs gained with the project “Enhancing national capacities for enhancing potentials of MSMEs in achieving SDGs in developing countries (MSME project)”36 funded by the UN Peace and Development Fund (PDF), DESA complemented the PDF Kenya and The Gambia to improve contributions of MSMEs to social and economic resurgence after the pandemic and strengthen their roles as enduring institutions for jobs and entrepreneurship to reach those further behind. ESCAP took advantage of the Asia-Pacific MSME Policymakers Network established with the project to present and launch the 2nd edition of the “Policy Guidebook for (M)SME Development in Asia and the Pacific”37, and its accompanying “Online Resource Centre for MSME Development”. Several knowledge products from ESCAP were also co-financed by the project and other sources. For instance, the publication on MSME Access to Finance: The Role of Digital Payments38 was funded from the Regular Programme of Technical Cooperation for the studies on Cambodia and Nepal, and from the Government of Canada (through Global Affairs Canada, in the context of ESCAP’s Catalysing Women’s Entrepreneurship Programme), for the studies on Bangladesh and Samoa. The UNECA Sub-regional office for Southern Africa leveraged the project to build the foundations of a UNDA 12th Tranche project on innovation and technology for SMEs, which had been approved. ESCWA presented the DEPAR portal39 and MSME toolbox40 at the first Arab SME Summit41, organized in Jordan from 30 October to 1 November 2022 by another ESCWA project.

40. The Surge project increased internal collaborations between departments in the implementing UN entities. Evaluation informants often shared the perspective that the project has contributed to increased communication between internal departments that were used to working separately. Several project outputs were developed on the basis of internal cross-collaborations between teams. In UNCTAD for example, the SME Entrepreneurship Branch and the Empretec centres collaborated with the Trade and Gender team, the Competition Branch, and the Accounting team. Staff from these sections participated in coordination meetings, exchanged tools and new materials, and were consulted to identify beneficiaries. The UNCTAD Business registration team also commended the project for providing an opportunity to have an open channel of communication with the Entrepreneurship section. This contributed to sharing experiences and good practices and for these colleagues to be better positioned to promote e-Registration. In ECE, project activities helped to break down silos and foster collaboration between different teams working in the area of sustainable trade, innovation, circular economy, rational use of natural resources, PPPs and women’s empowerment. In ESCWA, the work on the MSME toolbox involved 7 divisions, reflective of the 7 foci of the toolbox. Each specific team

35 Bangladesh startup ecosystem assessment report | ESCAP (unescap.org) 36 UNPDF | Enhancing national capacities for unleashing full potentials | United Nations 37 Policy guidebook for MSME development in Asia and the Pacific, 2nd edition | ESCAP (unescap.org) 38 ESCAP. 2022. MSME Access to Finance: The Role of Digital Payments, MSME Financing Series No.7. Bangkok. Available at: https://www.unescap.org/kp/2022/msme-financing-series-role-digital-payments. 39 Communities | DEPAR (unescwa.org) 40 MSME Toolbox Dashboard | DEPAR (unescwa.org) 41 Arab SMEs Summit | DEPAR (unescwa.org)

26

helped with research, data collection, and finalising materials. Weekly meetings were organized with the coordinators under each division. The teams used Trello to see what others were doing. In ECLAC, the International Trade and Integration Division and the Production, Productivity and Management Division jointly developed a course promoting digital economy and digital trade. In ECA, the ECA subregional office for Southern Africa (SRO-SA) and the ECA Digital Center of Excellence (DCE) collaborated to jointly produce a study on the “Role of Digitalisation in Strengthening Capacities of Micro, Small and Medium-size Enterprises (MSMEs) in Southern Africa”42. The ECA Subregional Office for Southern Africa collaborated with the ECA Africa Institute for Economic Development and Planning (IDEP) to deliver two training courses on role of technology and innovation for MSMEs. In some cases, informants mentioned that the Surge coordination meetings organized by UNCTAD helped identify other units from the same implementing UN entity involved in the project. Despite such examples, other staff also stressed that the overall emergency context meant there was little time to pursue internal synergies as teams concentrated most often on activities under their direct responsibility and on achieving immediate results.

41. The project enabled some level of technical collaboration and complementarities between implementing UN entities. According to the concept note, the project was conceived to propose a coherent approach towards MSME resurgence reflecting the holistic and comprehensive nature of efforts needed to ensure and facilitate the MSMEs’ green, resilient and inclusive recovery. Activities sought complementary and synergies in impact among participating agencies covering key interrelated components of an enabling entrepreneurship ecosystem. Inter-agency collaborations expanded the dissemination of agency-specific products while coordination and communications helped to avoid overlapping activities. Some joint products were also developed by the implementing UN entities, most often involving UNCTAD as one of the partners (see also next paragraph). UNCTAD and ESCWA, for example, cooperated to inform and build awareness on the e-registration platform for MSMEs. ESCWA leveraged US$ 30,000 from the project to organize a capacity development workshop with UNCTAD to familiarize member states in the Arab region with the e-registration system. This generated interest and ESCWA was later able to mobilise US$ 600,000 from different sources for two e-registration projects in Jordan and Syria43. UNCTAD also produced a global report on market access and competition policy44 to which ESCWA and the other Regional Economic Commissions contributed through advice, inputs, data, and review of the document. In Europe, ECE’s COVID-19 impact assessment targeting MSMEs in Georgia45 spurred discussions with the Ministry of Economy and Sustainable Development leading to the integration of Georgia’s non-tariff measures (NTM) information into UNCTAD’s NTM database46,. In Romania, ECE took up the Empretec methodology and provided grants for the Asociatia pentru Antreprenoriat din Romania (APAR) to conduct two in-person workshops of 9 days each which were attended by 60 MSMEs. In coordination with the Regional Economic Commissions, UNCTAD organized a series of regional capacity-building events to raise the awareness of policy makers on the role of competition-related policies for access to markets in the post-COVID-19 resurgence of MSMEs. UNCTAD, ESCAP and the Trade and Competition Commission of Thailand (TCCT) organized, for example, a conference in June 2022 on the Contribution of Competition Policy to the Resurgence of MSMEs post-COVID-1947, to which DESA and ESCWA also contributed. UNECA and UNCTAD co-organised a Regional Policy Dialogue on “The role of Competition Policy in supporting MSMEs economic recovery in the post COVID-19 crisis” and collaborated on a case study on South Africa for the UNCTAD global report on “How Covid-19 affects MSME access to markets and competition: a review of key issues and recommendations for future action”. Evaluation informants also mentioned that the project provided a unique and successful opportunity to learn about the work of the other implementing UN entities in the area of entrepreneurship and MSME development, therefore contributing to the “One UN” agenda. The project website48, developed to gather outputs and increase their visibility and sustainability, was also commended by many informants. However, most of these collaborations had a

42 The role of digitalisation in strengthening capacities of Micro, Small and Medium-Size Enterprises (MSMEs in Southern Africa to take advantage of the AfCFTA): ECA Sub-Regional Office for Southern Africa (SRO-SA) & ECA Digital Centre of Excellence (uneca.org) 43 ESCWA also mobilized an additional US$300.000 for an e-registration project in Sudan but the political context in the country did not allow for implementation. 44 The covid-19 pandemic impact on micro, small and medium sized enterprises: Market access challenges and competition policy (unctad.org) 45 Impact_COVID-19_Georgia.pdf (unece.org) 46 UNCTAD TRAINS 47 Conference on the Contribution of Competition Policy to the Resurgence of MSMEs post-COVID-19 | UNCTAD 48 Home | UNCTAD

27

bilateral character and the evaluation noted the difficulties for Regional Commissions to collaborate with one another.

42. Several factors limited opportunities for amplifying synergies between interventions and for establishing a stronger global coherence. From a total of 100 project outputs formulated across the three phases of the project logframe, only 6 were identified by the evaluation as joint outputs from 2 implementing UN entities (Figure 4, in yellow), out of which just 4 were effectively delivered (i.e., UNCTAD and ECLAC on outcome 5). Joint outputs were formulated for phase 3 of the project, owing to several factors such as the short duration of phases 1 and 2 and the need for a sufficient amount of time to design joint interventions identified after a national agenda, the number of implementing partners progressively increasing between phases 1 and 2, or the limited cross-institutional awareness of each partner’s expertise on and capacities for supporting MSMEs.

Figure 4: Project outputs per implementing UN entity and phase.

Note: Joint outputs are referred to in yellow. Source: PRODOC.

43. Staff from different Regional Commissions attempted to establish synergies with other Regional Commissions for this project but with limited success. ESCWA for example reached out to ECE and ECA to seek their prior experience in supporting MSMEs with a view to uploading their materials on the DEPAR portal to be used in the region and globally. Despite several meetings and some exchange of materials, there was no concerted effort to populate the DEPAR portal. Several evaluation informants in UNCTAD and the Regional Commissions seconded the perspective of an ECE staff member that “everybody was too busy and there was no time to make the effort of cooperation”. According to a staff member, if “capacity development by Regional Commissions is to be jointly done, this is a long process cycle” not suited to a crisis situation. Evaluation informants also mentioned that the geographical focus and mandates of the Regional Commissions created various challenges to collaboration as the Regional Commissions come with different areas of expertise and niches. Furthermore, needs of Member States also varied across and within regions. In addition, although the EPF was found to be a comprehensive and flexible framework, none of the implementing UN entities outside of UNCTAD had the capacity to cover all pillars. On average, UN entities covered about 2 out of 5+1 project outcomes (Figure 5), i.e., they remained geographically and technically focused. Accordingly, several informants shared the perspective that the project was a compilation of activities and outputs based on a unifying and valid conceptual framework (i.e., the EPF) and cluster-based approach but with limited synergies, moderate coherence, and high overall complexity. According to a staff member at ESCAP, “The project did not install joint indicators and reporting. If there was to be a joint event, it would indicate specifically who would deliver what. It was not about the project producing a pool of papers without referring to one or another agency; products were assigned by entity. From then on, it was clear agencies would have different streams of work.” Some evaluation informants questioned the cost and benefit of a global project in a crisis context compared to several inter-regional projects (see section on Efficiency), despite this project enabling strong knowledge exchange between participants.

Figure 5: Number of project outcomes covered by the implementing UN entities.

28

Source: Final report. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID-19 in delivering socio-economic support to Member States?

44. The project triggered some collaboration with other UN entities at national, regional, and global levels. At global level, UNCTAD partnered with UNITAR to provide the e-learning course “Policymaking: Entrepreneurship for Sustainable Development”49, which was followed by 317 policymakers from over 104 countries. ESCAP also published a series of knowledge products on MSME Financing50 that benefitted from a partnership between ESCAP and the United Nations Capital Development Fund (UNCDF). Complementarities with other UN organisations were also at national level. In several cases, the UN RCOs and UNCTs were referred to as project partners. In the Gambia, the Ministry of Trade, Industry, Regional Integration and Employment together with DSDG/DESA and the UN Country Team organized a high-level policy implementation forum in December 2020 on supporting MSMEs in the post-Covid-19 pandemic era, focusing on promoting MSME formalization51. In Europe and Central Asia, UNDP was often involved substantively and/or logistically in the delivery of online or hybrid workshops and dissemination events. In Armenia for example, ECE, UNDP, and other partners organized in March 2022 a conference on “Unveiling Market Opportunities for Boosting Residential Energy Efficiency and Alleviating Energy Poverty”52. UN Women also contributed to the development of a roadmap in Moldova based on the recommendations outlined in a study on women-led MSMEs, conducted by the project. Country office staff from several UN organisations including FAO, UNDP, OHCHR, and UNHCR also attended some of the online trainings or events delivered by the project. Despite such examples, evaluation informants were often of the opinion that the project had not maximised partnerships with other UN entities owing to several constraints or limitations, including the respective crisis response priorities and challenges in UN development agencies, limited staffing capacities from the project to focus on partnership building at global level, limited funds directed towards creating synergies with other agencies, lack of dedicated outputs and indicators referring to joint work with other agencies, and the lack of country presence of the implementing UN entities (see also §46).

45. The project was implemented with a range of partners that complemented the responses of the UN entities. Besides UN entities, project implementation involved a range of other partners at international, regional and national levels that helped to complement the responses from other UN organizations. Expert speakers in ECE events included, for example, important players in greening the MSMEs agenda, including the International Chamber of Commerce (ICC), World Trade Organization (WTO), Organisation of Economic Co- operation and Development (OECD), European Bank for Reconstruction and Development (EBRD), and a number of national authorities and private sector players. In Southern Africa, the project contributed to strengthening the ECA SRO SA partnership with the SADC Business Council and national chambers of commerce and industry in all 11 Southern African countries. Joint events were organized by ECA and the SADC Business Council at the SADC Industrialisation Week in Malawi and the Dubai Expo 2020. ECA also developed a digital platform on technology and innovation for MSMEs to be hosted by the SADC Business Council. According to the evaluation survey, the majority of respondents were of the opinion that the project complemented interventions implemented by other international organizations working in their

49 E-Learning Course on Policymaking: Entrepreneurship for Sustainable Development | UNCTAD 50 ESCAP’s MSME Financing Series | ESCAP (unescap.org) 51 High Level Policy Forum on MSMEs | United Nations Development Programme (undp.org) 52 From buildings to small companies, UNECE supports countries to reap benefits of improving energy efficiency | UNECE

1 1 1

5

1

2

3

2

1 1

6

1 1

2 2

5

1

0

2

4

6

UNCTAD DESA UNECA ESCAP UNECE ESCWA ECLAC

Phase 1 Phase 2 Phase 3

29

country/region (Figure 6). As illustrated by a private sector representative in North Macedonia, “A lot of organizations were conducting projects regarding COVID. However, this one was the only one providing actual information on practical ways to mitigate some challenges arising from the pandemic”.

Figure 6: Perceived complementarity of the project with other interventions according to the survey (n=130 respondents).

Source: Independent evaluation survey, 2023.

46. Partnerships with other UN entities and organisations proved challenging to harness. Despite the above and some other cases, the evaluation found that the project was not able to establish a cohesive eco- system of strategic partnerships. The initial concept note of the project considered that project governance would establish working groups on each outcome area, including “Digital economy solutions for advancing SME resilience (ITC, ITU, UNCDF, UN/CEFACT)”, “Resource Efficient and Cleaner production (led by UNEP and UNIDO)”, “Zoning/urban issues (led by UN-Habitat)”, “Employment (led by ILO)”. The Project Document for Phase 2 of the project also mentioned that “Close cooperation and partnerships should also be sought with World Bank, IFC.” This proved over-ambitious. In practice, the evaluation found limited global collaboration with UN entities also engaged in responding to the crisis and providing support to MSMEs, such as the ILO53, UN Women54, or FAO55. The evaluation also did not find evidence of the project attempting to leverage funding from other UN COVID-19 response mechanisms, such as the UN COVID-19 Response & Recovery Multi- Partner Trust Fund (MPTF) which financed projects at the national or subnational level in some of the countries where the project was implemented. Some informants stressed that Member States were looking for financial resources to support MSME resurgence, which is rather in the realm of the development banks. However, connections with the World Bank and the regional banks were limited. Partnerships appeared, therefore, primarily based on a bottom-up approach, relying on legacy collaborations or immediate opportunities, but not necessarily globally strategized and pursued. It should be noted though that the project did not dedicate specific resources to such activities (see next section).

5.3 Efficiency

How well coordinated was the process for the response among the entities implementing the joint project?

47. Despite unprecedented circumstances with the Covid-19 pandemic, and the lack of clear dedicated resources, the administrative, management and financial processes for the response were well coordinated overall. According to the initial concept note, Phase 1 of the project aimed to implement two components.

53 wcms_854253.pdf (ilo.org) 54 WEA-ActionBrief3-SMEs.pdf (weps.org) 55 COVID-19 Response and Recovery Programme | Resource Mobilization | Food and Agriculture Organization of the United Nations (fao.org)

28%

43%

18%

3% 2% 1%

5%

The project complemented interventions by other international organizations working in my country/region

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

30

Component 1 was to be led by ESCWA and planned to establish a virtual knowledge hub. Component 2 was to be led by UNCTAD and targeted the provision of hands-on assistance to countries on several sub- components grouped as 1+5 thematic clusters56. Although the narrative of the concept note did not identify an agency in charge of the overall coordination of the project, the budget featured the output “Coordination and proposal preparation” for which UNCTAD was to be provided US$ 27,000. This lack of clarity in the initial setup of the coordination of the project was reflected in the evaluation interviews, informants from the various implementing UN entities, including UNCTAD and ESCWA, having different perspectives on the scope of work and responsibilities for the coordination of the process during the first phase of the project (see also section on Effectiveness). The Project Document for Phase 2 clarified that UNCTAD would be the lead agency for the overall coordination of the project. This role was reflected in the budget of the project, with UNCTAD receiving US$ 40,000 for “Output –.1 - Overall coordination”, but not included in the project logframe. A Project Steering Committee was to monitor project delivery and make necessary adjustments to ensure smooth implementation 57. In practice, evaluation informants distinguished three different periods regarding the coordination process. The first period coincided largely with the preparation and implementation of Phase 1 of the project (from April to June 2020) during which the DA-PMT was reported by UN entities’ informants as having a very active coordination role. The second period spanned from August 2020 until April 2021, covering Phase 2 and part of Phase 3 of the project. It had regular (i.e., about bi-monthly) Project Steering Committee meetings coordinated by UNCTAD. Then, from April 2021 until the closure of the project, a third period saw the Project Steering Committee meetings attended by a larger number of participants (up to 40 participants). Project Steering Committee meetings held during the 1st, 2nd and 3rd phases were not technical but provided room to discuss and align activities. According to an informant who attended the 3rd phase meetings, “they were also used by UNCTAD to propose participants to build synergies on topics, but the project was so broad that this seldom happened (sic)”. Most often, the evaluation informants reported that these 3rd phase meetings were useful to learn about the activities of other implementing entities. However, the complexity of the project also made these 3rd phase meetings at times quite long and difficult to follow. Furthermore, the different time zones made it challenging for all staff to participate. In terms of administrative and financial management, project staff in UNCTAD reported an efficient and supportive DA team. This was valued as, according to a staff in UNCTAD, “the UN requirements to obtain the funds are onerous. There is a need for expertise for administrative management and administrative rules to prepare requests and reporting.” Monthly financial statements were provided to project management in UNCTAD. The DA also helped in understanding who among the UN entities was behind schedule and who was on track. According to an informant in UNCTAD, “In the project management meeting, one UN entity could have a narrative saying that all was going great but then, with the DA, we would see the financial delivery was only 12%. Then we knew we had to push delivery. The DA provided the implementation rate. Very helpful.”58 To support its internal coordination, UNCTAD specifically designed in April 2021 a new product, the project monitoring dashboard, consolidating information on UNCTAD’s planned outputs and how each intervention was progressing. Several informants from UNCTAD reported this as a very useful, centralized tool. The dashboard was regularly updated, despite being demanding to maintain. Collecting information from staff was time-consuming. Furthermore, the dashboard was very detailed, recording information at the activity level (some outputs could entail a dozen activities) and across 11 data fields. In preparation for the progress report for Phases 1 & 2 in May 2021, the dashboard was used to compile information from all the implementing UN entities. Despite most informants reporting appropriate and efficient project coordination considering the circumstances, informants in UNCTAD, and across the partner entities, shared the perspective that the project could have benefited from additional staffing capacities dedicated to global coordination as well as to coordination with countries and UNCTs, and to partnership building.

48. The technical coordination of the response proved sometimes difficult to install and sustain. According to the concept note (Phase 1), working groups and task forces were to be formed around thematic clusters to support a coordinated approach and complementary efforts among participating agencies. It was

56 The concept note indicated the creation of the following clusters: A- Access to Resources (Component 1); B- Entrepreneurship and business skills facilitation; C - Business facilitation; D- Access to Finance/financial literacy; E- Access to Technology; F- Access to Markets. 57 As per the Project Document for Phase 2: “Project Steering Committee - Representatives from UNCTAD, DESA, ECA. ECE, ECLAC, ESCAP and ESCWA will have online meetings once in two months to monitor project delivery to ensure smooth and steady implementation. Representatives from all seven entities will meet on an as-need basis for important decisions that affect all seven entities.” (p. 53). 58 As a side note, DA Focal Points reported that meetings of the DA Focal Points network became more dynamic and substantive with the response to the Covid crisis.

31

initially envisaged that members of these task forces would include the implementing UN entities as well as other UN partners. The Project Document (Phase 2) narrowed down the prospective setup for the clusters’ technical coordination by referring to Task Forces comprised of experts from the implementing entities. According to an informant, “The discontinuation of the original concept note was positive because it was unrealistic. The problem is that staff members were on other jobs, no one exclusively working on the project. It was time-consuming to coordinate and staff had different agendas and there were different priorities between partners.” In practice, this cluster-based thematic coordination was loosely implemented but fluid, sometimes involving sharing of information between several implementing agencies, though more often one- on-one bilateral and direct exchanges. As noted earlier, interviewees reported information sharing with UNCTAD and sometimes directly between the Regional Commissions “but then entities were doing their own part”. One of the cluster leads for example proposed that the Regional Commissions undertake a global analysis on market access and COVID-19 to develop a joint policy paper, but this did not work: “Colleagues wanted it but there were so many things to do. Project managers did not approve it and it was not in the logframe.” In that sense, the technical coordination of the project and its contribution to the efficiency of the response was eventually more closely aligned with the aim stated in the initial concept note as “(t)he effort will help to enhance coordination in the area of the MSME promotion in the UN system, and avoid confusion, duplication, and waste of resources”. Informants in UNCTAD and the Regional Commissions shared the perspective that directing adequate staffing capacities specifically to technical coordination may have facilitated the establishment of stronger synergies between the implementing entities. Furthermore, despite most UN informants being supportive of the global architecture of the project, including as a modality to enlarge partnerships and seize the benefits of a “One UN” system approach, a few staff from the Regional Commissions suggested to commit more time in the future to analyse the added value of global/inter-regional versus regional projects. It was argued that coordination costs increase with the complexity of the project and with the number of actors involved. As indicated by a staff in ECE, options to balance would include designing a global project or inter-regional or regional projects “and then coming together for dissemination at global level". Therefore, “the added value of a global project should be thoroughly assessed before project design. But during Covid it was a global challenge. […] Regional projects might have been an option. If going through inter-regional projects, their added value should also be well assessed. But in crisis, with the short timing and need to bring everybody under the project, one needs to cut some corners.”

How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

49. The phased approach was found to be a valid mechanism to cope with the uncertainty of the crisis. The project was developed and implemented under a three-phase budgeting and programming approach: Phase 1 spanned over 2 months (May-June 2020), Phase 2 was implemented in 6 months (July-December 2020), and Phase 3 was provided a total of 18 months (initially January-December 2021, extended twice, first to March 2022 and then to June 2022). On the programming side, during Phase 1 various entities were able to leverage existing products to deliver some immediate responses to countries that had requested assistance. In general, during Phase 2, the project contributed to develop tools, deliver technical support, and customizing materials required by countries. Phase 3 enabled implementation at country level of the tools developed during phase 2 and, building on country experience, to collect good practices and lessons learned that further informed policy makers. Technical staff interviewed by the evaluation were largely positive about the 3- phased approach. Interviewees recalled that this was a very difficult period and that it was unclear when the pandemic would end. Although a clear project plan for the entire period would have been beneficial, the context was too unstable to warrant it. According to a staff member in one of the Regional Commissions, the 3-phased approach “was the optimal and rational thing to do”. It was well aligned with the uncertain circumstances and supportive of a learning process. According to staff at UNCTAD, “Phases helped adaptive management” as “each phase was an innovative stage that gave a chance to introduce new things not foreseen at the beginning”. Incidentally, as put forward by a staff member in one of the Regional Commissions, it shows that the UN had an agile way of thinking to cope with the challenge. On a few occasions, a more nuanced assessment was conveyed by interviewees. Several informants reported that the phased approach added to the complexity and uncertainty of the project and made it more difficult to understand its design, and that a Theory of Change was missing.

50. The project would have benefited from providing clearer visibility on the funding available. From a financial standpoint, the five Development Account COVID-19 projects, including this project, were funded from various DA funding sources. This entailed relying on interest savings, residual balances from the 10th and other closed tranches, and savings from the 11th tranche. Accordingly, the overall available funding was

32

confirmed by phase. It was determined in late 2021 that the combined proposed phase 3 budgets submitted by the five projects exceeded the funding that became available by 1 million USD. As a result, the projects were requested in November 2021 to reduce their proposed phase 3 budgets, and for this project, the budget was reduced by $310,000, from a total of $3,165,500 to $2,855,500. UNCTAD received the final instalment for phase 3 in December 2021. Due to the requirement to reduce the budget and revise the planned activities for phase 3, all five projects were extended by three months to March 2022. However, according to informants at UNCTAD, this initial 3-month extension did not fully account for the late final instalment in the context of a global crisis. A UN staff member mentioned, for example, that if implementing UN entities “want to recruit a consultant, they need funds, then to identify candidates, then 2-3 weeks to issue the contract, then the consultant needs to do the work. That can take 3 months in normal circumstances”. However, the context of the pandemic imposed additional delays. Some activities that were planned to be delivered in person had to be delivered online instead in 2022, which required time for reprogramming based on the new budget. Furthermore, counterparts were also challenged as there were uneven capacities in beneficiary countries to put in place some of the interventions. Accordingly, activities for which disbursements were made shortly before the closure of the project in June 2022 were still underway several months later. Informants indicated that this contributed to the delayed completion of the final report of the project, together with limited resources for project monitoring and reporting in the Regional Commissions, particularly when the DA Focal Points were not associated with project implementation (see next section). Some staff also felt that the phased approach created a lack of visibility on the funds that would be mobilized by the DA. This was sometimes found to be a challenge when designing the project, as it prevented setting clear expectations and a clear design. Better visibility on the resources in the pipeline could have helped to construct a different logframe, which would have been designed at phase 2 and used at phase 3 with little revision. According to a UN staff member, a clearer view on the funding available for the entire duration of the project could have helped the teams to consider additional scenarios, such as focusing some of the activities on a more limited number of countries with greater depth.

5.4 Effectiveness

To what extent did the programme (Development Account) and project governance and management structures and processes enable, or hinder, the effective implementation of the joint project and the achievement of its results?

51. The Development Account programme, as a funding modality, was an effective tool in supporting the design of a global response to support the resurgence of MSMEs post-COVID-19, with remaining questions about it being the most adequate tool in comparison with other UN agencies and mechanisms with a mandate for humanitarian response. DA is a regular budget capacity development programme. It funds, among others, operational workshops, advisory services, and travels to countries to deliver capacity-building activities. Projects are approved by the General Assembly as part of the programme budget for the first year in the biennium. The project implementation period (from receipt of funds by implementing entities to the financial closure of the project) usually is 4 years, following a design or planning period that takes about 1.5 years. 59 Given project specificities and circumstances, and, possibly, limited knowledge of the DA rules and procedures, evaluation participants reported different views on the duration of DA-funded project cycles, the requirements for partnerships and indicators, and the DA governance structure. The converging perception is that a regular DA-funded project cycle is long and onerous to plan for. Nevertheless, to provide a swift and global answer to the COVID-19 crisis, an adaptive management approach was adopted. These procedures were made flexible, the design phase was significantly shortened, and a phased planning was introduced. For the five COVID-19 projects, the Surge project included, the GA approval of project concepts was skipped, while the endorsement by the DA Steering Committee remained and the use of residual balances from closed and closing projects to fund the five projects was retrospectively reported in the fascicle for 2022, which was submitted to the GA in 2021.60 Hence, the concept note of the Surge project and 1.5 years of implementation

59 Under regular tranches of DA, the design or planning phase of projects starts with the launch of a DA new tranche by the USG for Economic and Social Affairs and the development of concept notes by 10 DA entities, and ends with the approval of the project documents by the DA Programme Management Team and the endorsement of DA budget and funded projects by the General Assembly. 60 A report commentator referred that “this has happened in the past when developing projects using residual balances from the tranche budget, but has never happened, nor is allowed to happen, for regular tranche projects.”

33

period was endorsed to initially respond as quickly as possible to the urgent requests from Member States to address the socio-economic fallouts of the pandemic. It was widely reported by informants that this was the best approach to the circumstances, as no one knew how long the crisis would last, with expectations that it could be just a few months. An advantage of this alternative path to access residual balances from closed and closing projects, reported to this evaluation by a DA representative, is the almost immediate availability of resources. This promptness is praised by stakeholders in many implementing entities as a ‘unique’ achievement.

52. Nevertheless, when the DA was compared with other UN agencies and mechanisms with mandate and resources to provide humanitarian assistance in a crisis context, informants referred to the difficulties to be as prepared as these other agencies. The strategy, then, was to focus on post-crisis recovery and development, as a suitable DA approach to actively support countries and, specifically to the Surge project, MSMEs. Yet, informants from the regions referred that adaptations to the project document guidelines kept the highly demanding requirements of the fund and this was assessed by them as an issue to be reviewed if the fund is to be used in future crises. A better crisis-fit approach would have involved an even more agile track, and a focus on deliverables of immediate effect to mitigate the crisis’ impact on MSMEs and government policies (e.g., enhanced access to emergency finance and digital tools for trade). Informants shared expectations about the DA-PMT’s review of the five COVID-19 evaluations toward helping to inform the DA’s role in dealing with future shocks.61 Among members of the project coordination team, for instance, there are ideas about the usefulness of the Entrepreneurship Policy Framework as an overarching tool with relevant potential outputs for crisis response, as discussed later. Considering that the EPF is the key framework of UNCTAD to fulfil its role as the UN focal point for entrepreneurship and MSME policy for sustainable development (RES/71/221 and A/RES/73/225), EPF-related ideas can be considered as a tool to support crisis response funded by the UNDA.

53. Project governance and management structures and processes, overseen by the DA-PMT, supported a quick and overall effective response in light of the UNDA standards, identifying available resources and potential partnerships that ensured the delivery of an impressive number of outputs and achievements at the outcome level, even if joint delivery between implementing entities took time to come to fruition and some DA-FPs were ambivalent about the extent to which they were involved. Addressing a crisis of global scope such as the COVID-19 pandemic was new to the DA’s management structure. Seeking to find quick and effective ways to deal with it, the Head of the Capacity Development Programme Management Office of DESA, in his role of assisting the DA Programme Manager, sent out a memo to the DA focal points in the 10 implementing entities on 18 March 2020, inviting them to a Skype call to discuss the DA's COVID-19 support to Member States, in particular the possible reorientation of the ongoing 10th, 11th and 12th tranche projects, on 20 March 2020. The meeting minutes indicate that the participants were informed of two proposals for potential response initiatives from ESCWA (on SMEs) and from DESA (on statistics). A follow-up meeting was convened on 26 March, where five other project proposals were discussed, and other implementing entities were requested to provide comments on those proposals, as well as submit any additional proposals. After these extensive consultations with DA-implementing entities, DA-PMT selected five topics considered as the most relevant to a socio-economic response to COVID-19 and built on comparative advantages of DA- implementing entities. This included strengthening the capacity and resilience of MSMEs to mitigate the economic and social impact of the pandemic – ESCWA’s proposal. The preliminary idea of this project, as reflected in its 1-page concept note, focused on digital tools to support SMEs. UNCTAD further enriched it with additional ideas, such as the e-registration platform and entrepreneurship for SMEs. The project concept note finally took shape with the inputs of more entities, including DESA and other four regional commissions. As more implementing entities joined the project, UNCTAD supported the clustering of activities in 5+1 outcomes, with implementing entities taking outcome leadership according to their expertise and capacities. This structuring of the project governance, with DA-PMT’s oversight, and the management structure with UNCTAD in the overall coordinating role and each implementing entity leading the operational implementation at regional level ensured the global and regional scope of the project delivery. At the level of the ongoing processes, implementing entities relied more on partnerships with local and regional institutions rather than coordination within the larger project management structure. The online meetings of the steering committee

61 An updated information about this review received in feedback to this evaluation report refers that “this programme- level evaluation had to be cancelled due to the delay with the completion of four of the five project evaluations (only one has been completed to date, and two are expected to spill into 2024). A meeting of the DA focal points to discuss the findings of the five project evaluations will be organized by DA-PMT as soon as they are completed; the DA's role in dealing with future shocks would certainly be discussed at this meeting.”

34

and the smaller-scale coordinating role of the task forces to weave efforts worked to a limited extent, as these meetings were not as regular as planned in the project document. The result was a high delivery rate with limited inter-agency coordination in the process. Yet, this independent evaluation identified that the higher- level governance and management structures and processes were effective in bringing the project together with identifiable results at the outcome level.

54. Given the uncertainties about how the pandemic would evolve, and considering the pressing immediate needs, a first phase of the project was launched with a limited number of outputs (n=7), in May 2020. It resulted from meetings of the DA Network in March and April 2020. From April 2020, these became bi-weekly meetings of the DA focal points. It is noteworthy that this process was not recollected as such by the informants of this evaluation. There is a perception that not all DA-FPs were involved from the start, that this inclusion was gradual and followed advances in the project. Informants acknowledged that these meetings also included staff from the implementing entities to discuss the next steps to be taken, the different country and regional needs, and the installed regional expertise to propose responsive actions62. This trickled down into several smaller-scale bilateral meetings at the regional and local levels. It became clear that the DA’s mandate on development had to be taken into consideration when planning the project outputs and adapting activities associated with crisis response (e.g., providing direct health support was out of the scope of possible answers). The DA’s development mandate, instead, could support the strengthening of institutions that needed preparedness and resilience to design supportive measures in a post-pandemic recovery perspective. Hence, out of five projects, the Surge was one with the objective of developing and implementing capacity-building tools for governments and MSMEs to facilitate the resurgence and strengthen the resilience of MSMEs in developing countries and economies in transition. There was a need to bring the relevant expertise of different entities quickly on board. Because of its global mandate and expertise on entrepreneurship and MSME policy, UNCTAD became the lead agency of the Surge project, and coordinated the elaboration of a project document to deliver a global response with more entities. For the first time, the DA-PMT was intensively involved in the oversight and monitoring of project implementation, with more direct contacts with the technical cooperation staff – rather than going through organizational-level DA focal points. “The DA PMT was in contact with the DA-FPs but had more contacts with the project managers than it would normally have.” (DA-PMT representative). This was made possible by the small number of projects responding to the COVID-19 crisis. This centralized approach allowed for a swift response at a global level, with a record time endorsement of the concept note and the Phase 1 budget by the DA Steering Committee. DA-PMT’s oversight and direct contact with technical staff resulted in the institutionalization of periodic meetings with DA-FPs and other stakeholders. UNCTAD DA-FP, represented by the Technical Cooperation Section (TCS), served as a bridge between the DA-PMT and UNCTAD project teams, especially at the design stage. UNCTAD TCS also provided substantive support to the UNCTAD project coordinator in the draft of the concept note of the MSME Surge project. During the implementation stage, UNCTAD DA-FP participated in several project Steering Committee meetings and addressed questions from project teams. UNCTAD, as the leading entity, and under the guidance of the DA-PMT, communicated directly with the project focal points to monitor progress. This monitoring was supported by UNCTAD Budget and Project Finance Section through the sharing of financial information on implementation rate by the implementing entity. This, and the evolution of the pandemic, brought more clarity about the needs and the implementing capacity of UN entities and member states in responding to the crisis. It supported the design and implementation of phases 2 and 3. There is consensus between informants that the introduction of the phased planning was consistent with the uncertainties of the pandemic crisis, and they would not recommend another way to respond to it, despite reporting some bottlenecks.

55. There are different perspectives on how well the DA procedures were adapted for crisis response. Following the requirements for joint proposals as much as possible, according to informants, felt like squeezing a 4-year long implementation period into 1.5 years. It was suggested by some informants that, instead of a concept note and a PRODOC, a crisis response could have required only one of them. Outcomes were over- ambitious for a 2-year project, but project planners felt they had to comply with the DA requirements, even if this would imply challenges to deliver at the outcome level. Indeed, some outputs were delivered after the end of the project (see next section). There was and internal quality control of the PRODOC with multiple inputs, oversaw by the DA-PMT to ensure a ‘good enough quality’ level, however, a higher-level involvement of all relevant DA-FPs in dialogues and feedback loops from the start and the gender review and other quality assurance steps (i.e., internal clearance processes of the implementing entities of interim documents) could

62 Other informants referred to these meetings as more about sharing information than coordination between implementing entities.

35

have improved the quality of the PRODOC further. For instance, it could have ensured that project indicators cover the SMART63 criteria and be better aligned with a crisis response context – some were the same Empretec-based indicators in use for the past 10 years. In one example suggested by an informant, instead of having an indicator on increased sales of MSMEs, the project could have one about ensuring that a certain percentage of MSMEs that received project support remain in business. Informants confirmed that designing the logframe of the project was challenging because of the scale of the COVID-19 crisis, the number of implementing entities involved, and the uncertainty about the level of funding associated with the phased approach. The high number of planned activities and countries would imply a thin distribution of financial resources. What some project teams did to manage the uncertainty of resource availability and ensure delivery in the Surge project was saving resources from regular projects. Informants acknowledged that these managerial and processual arrangements were the best that could have been done during the crisis and the uncertainties associated with it, but also acknowledged that, even with some trimming in the proposed activities, the end result was less focused than would usually be the case. A lesson learned is the need to assess what crises the DA programme is equipped to respond to (e.g., economic, financial, market, health, climate, national, regional, global, etc.) and what tools can be immediately made available to countries in need. For instance, the e-registration platform was mentioned as an existing technology for e-government that was rapidly scaled up to respond to the needs of MSMEs in the COVID-19 crisis.

56. The project governance and management structures and processes were effective in supporting bilateral collaborations between UNCTAD and individual Regional Commissions in delivering project outputs, but there were limited project-level management mechanisms. Nevertheless, the operating mechanisms supported communication and seeded exchanges of information between regions with increased joint deliveries toward the end of the project. From the perspective of the DA-PMT, according to interview data, one of the purposes of bringing all implementing entities together in bi-weekly meetings was to foster dialogue and help collaboration for a quick response to the crisis. It seems that this did not represent an inter-entity coordination function of these meetings, but a sharing of knowledge to be taken up by other coordination structures. A project coordination team, with two representatives of UNCTAD and one of each of the other implementing entities, was set up. The actual coordination was mostly centralized in UNCTAD, as the project lead. The perception of some members of the PCT is diverse in relation to how the PCT worked, with some of them referring to themselves as focal points rather than members of the PCT, and others not providing substantive evidence on how the PCT, as a whole-of-project coordination structure, worked. Evidence from deliverables indicates that collaboration was bilateral, mostly between UNCTAD and individual Regional Commissions, and built on previous successful collaborations. Some informants referred to parallel streams of work without coordination with the work of other Regional Commissions. Others saw the Surge project as an opportunity to learn about what other Regional Commissions were doing and to set the stage for future collaboration. For instance, ESCAP, ESCWA and ECA exchanged information about the DEPAR portal, exchanged materials, and participated in meetings; ECE and ESCAP acknowledged that mutual cooperation can be facilitated by them having member States being in both commissions. Interviewees from all implementing entities referred to attending events organized by other entities. An advanced level of cooperation was achieved by ECLAC and ECA (through its Africa Trade Policy Center (ATPC) and IDEP), on the online course material on competition policies delivered in Portuguese. They engaged in initial conversations and, because the pace of implementing this specific output was different in each entity, this was an asynchronous cooperation, mostly involving the sharing of course materials. Yet, this is reported by ECLAC as a successful South-South cooperation during a turbulent crisis context.

57. UNCTAD, as the lead agency, focused on bilateral cooperation according to regional expertise and planned outputs. UNCTAD cooperated with ECLAC, ESCAP, ESCWA, ECA, and ECE in the delivery of outputs such as “The COVID-19 pandemic impact on micro, small and medium-sized enterprises – Market access challenges and competition policy” (phase 1, output F7). With ECLAC, UNCTAD collaborated in delivering a training course on competition law and policy, an online course on competition and SMEs and cross-border trade, outputs on competition policy, and technical assistance for projects on Asia exports in the LAC region. With ESCAP, UNCTAD collaborated with the creation of the ASEAN SME policymakers’ network, and a discussion paper and meetings proceedings of this network. With ESCWA, UNCTAD collaborated in actions on topics such as e-registration, and the organization of an event on digitalization. This partnership grew stronger and new partnerships emerged in the area of competition. With ECE, UNCTAD collaborated with a report on SMEs, and counted on ECE speakers contributing to its workshops. This is consistent with UNCTAD’s traditional approach of engaging in bilateral cooperation, rather than interregional. A factor associated with

63 Specific, Measurable, Achievable, Relevant, and Time-Bound.

36

this approach is the source of demands for assistance, which normally are country- or regionally based. In addition to collaboration between implementing entities, the Surge project also promoted intra-agency cooperation. This was reported within UNCTAD, involving Empretec, the competition and consumer protection policies branch and the investment branch, within ESCAP, on the Bangladesh-related outputs, including the UNCTAD Voluntary Peer Review of Competition law and policy of Bangladesh, with a focus on SME, and within ECLAC, between the economic affairs and international trade divisions. This independent evaluation identified that, toward the end of the project, in phase 3, there were more joint deliverables involving different Regional Commissions (e.g., the online events on competition policy and MSMEs held in phase 3), which suggests that cooperation increased as the project advanced. A lesson learned from these collaborations is that cooperation depends on a combination of countries’ needs, sources of resources (e.g., donors’ requirements, modalities of funding and what they allow to be funded), continuous presence of staff and time for coordination. This makes planning for cooperation a challenge, especially under a global crisis of the COVID-19 scope. Nevertheless, higher collaboration could have been fostered had the project coordination team worked more effectively and planned joint outputs. Hence, despite Surge’s opportunities for cooperation, actual results of these initial efforts are at an early stage.

To what extent has the project contributed to the expected outcomes as enunciated in the project document?

58. Eighty-six outputs were delivered across the world’s regions, built on the expertise of UNCTAD and the Regional Commissions, and some degree of inter-entity collaboration. These deliverables composed a mosaic of actions that, on a global scale, contributed to achieving the outcomes enunciated in the project document. Implementing entities adapted resources and expertise that they already had and expanded (mostly bilateral) inter-entity partnerships to quickly attend to the most urgent needs of entrepreneurs and policymakers to support MSMEs in the first 2-3 months of the project. These actions comprised making information digitally available64, converting face-to-face training into online, developing digital platforms to support MSMEs and governments in responding to the crisis, and producing and disseminating assessment reports on the impact of COVID-19 on MSMEs. UNCTAD, ESCWA, and ECE delivered the eight outputs of phase 1, with some collaboration with the other entities to collect data for the development of these outputs. For the delivery of the Competition and Consumer Protection assessment report (output F7 of phase 1), under UNCTAD lead, for instance, all regional commissions were involved. Outputs in phase 1 were delivered not knowing how long the crisis would last but considering it could extend to 2021. As successive COVID-19 infection waves hit the world, the end of sanitary measures was not foreseeable. Phases 2 (planned to last until the end of 2020) and 3 (planned to last until the end of 2021) expanded the project to support more countries and MSMEs, and increasingly relied on inter-entity collaboration to ensure a high rate and quality of delivery. The PRODOC of these phases, then, involved seven implementing entities and shifted from general responses (phase 1) to the delivery of a wide range of country- and region-based interventions65. The project’s responses to these needs depended on the locally installed capacities to develop more resilient institutions and supportive policies to MSMEs. Some of these deliverables, mostly those related to online awareness- raising, policy dialogue and workshops, relied on inter-entity partnerships to be successfully delivered. Examples from phases 2 and 3 are the UNCTAD-ESCAP Regional Dialogue held in cooperation with the ASEAN Secretariat, the series of Online regional capacity-building events on the role of competition policies for access to markets in post COVID-19 resurgence of MSMEs (UNCTAD-ECE, UNCTAD-ECLAC, UNCTAD-ESCAP), the UNCTAD-ESCWA-ECA Regional Policy Dialogue: The role of Competition Policy in supporting MSMEs economic recovery in the post COVID-19 crisis, the Sub-regional workshop: Global initiative towards post- Covid-19 resurgence of the MSME sector – Women in informal and small-scale cross-border trade (UNCTAD- ECA), the online course Oportunidades y retos para Mipymes de América Latina en el e-commerce transfronterizo66 (ECLAC-UNCTAD), and the Global Policy Dialogue: Post COVID-19 Resurgence of MSMEs and Competition Policy (all seven implementing entities), The Surge project, therefore, effectively delivered a wide range of interventions (i.e., advisory services, trainings, workshops, seminars, digital portal, events, and

64 Smart search engine: Manara Portal (unescwa.org); MSME knowledge portal: https://msme-resurgence.unctad.org/ 65 From a project management perspective, phase 2 was to develop intervention tools and phase 3 was to implement them at country level. The analysis of the deliverables, however, shows a mixed picture, determined by the local capacities to move from the development of a tool to its implementation. Many outputs in both phases 2 and 3 combine, for instance, course development and delivery (e.g., OP 1.5 and 1.6, OP 3.4 and 3.5, and OP 5.10 in phase 2, and OP 1.14, and 5.3 in phase 3). 66 Translation: Opportunities and challenges for MSMEs in Latin America in the cross-border trade.

37

direct technical assistance through policy advice, assessment reports, research studies, and tailored guidelines), directly assisting 96 countries in specific ways. A good practice that comes from this approach is that a global response can be made of multiple local and regional actions that are the most relevant and impactful for its recipients during a crisis. Figure 6 shows the distribution of project-planned activities per phase and implementing entity. The number of outputs planned increased from 7 in phase 1, to 38 in phase 2 and 60 in phase 3 (+57.9% from phase 2), totalling 105 outputs, even under a budget cut of USD 310,000. UNCTAD was responsible for 50% of the outputs in phase 2 and 43% in phase 3.

Figure 7: Surge Project outputs delivered per implementing entity and implementation phase67.

Source: Project’s final report, 2022.

59. It is noteworthy that the number of outputs slightly changed between the PRODOC and the project final report. Additionally, this independent evaluation received evidence of deliverables completed after the conclusion of the final report, in 2022 (e.g., five outputs delivered by UNECA in phase III), and others that do not clearly fit in any of the project planned outputs (e.g., ESCAP’s papers on MSME Financing Series, delivered in 2022). Table 7 summarizes these changes and updates and presents the percentage of outputs delivered.

Table 7: Planned and delivered outputs per project phase. Phases PRODOC

outputs Final report

outputs Delivered outputs*

% delivered Not delivered**

1 7 8 8 100 2 38 39 36 92.3 4.3, 5.9, 5.13

3 60 54 42 77.8 1.25, 2.2, 2.3, 3.2, 3.3, 3.4, 4.1, 4.3, 4.5, 4.7, 5.5, 5.6

Total 105 101 86 85.2 * Based on evidence collected in this independent evaluation. **These outputs fall into the following situations: not delivered due to budget cuts, in progress (i.e., reports in the inception phase by the time of this evaluation), or missing information.

60. Overall, 85% of all project outputs were delivered. Phase 3 has the highest number of outputs and the lowest percentage of effectiveness, around 78%. It was impacted by the budget cuts, the longer time needed to conclude some outputs and the lack of information for this independent evaluation. In cases of budget cuts, this evaluation counted as delivered those outputs with at least 50% of the activities concluded. It is noteworthy that the delivery of some of these outputs was concluded after the end of the project68, given factors such as the pandemic barriers to implementation, coordination difficulties, personnel turnover, and

67 Figure 7 presents the number of outputs fully delivered as stated in the project’s final report. In addition, 6 outputs were still in progress when the final report was submitted (ECLAC 1 output, UNCTAD 1 output, UNECA 4 outputs in progress). 68 Interviewees used the project’s final report (2022) as reference for the end of the project, since some of their deliverables, pending then, were concluded afterwards.

Phase 1

Phase 2

Phase 3

0

5

10

15

20

UNCTAD DESA UNECA ESCAP UNECE ESCWA ECLAC

3

0 0 0 2 2

0

19

2 4 4 4

1 2

19

1 3 2

9 7

4

Number of project outputs delivered per phase

38

limited capacities of member states. Considering all these barriers, this independent evaluation considers the level of effectiveness high, since these rates of delivery contributed to the achievement of the main project objective through significant achievements in each project outcome, as described next.

61. National capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship and MSME promotion in post-COVID-19 resurgence (outcome 1.A) improved through e-learning courses, awareness-raising activities, technical assistance in designing and implementing national entrepreneurship policies, the creation of regional networks of policymakers, and knowledge production and hubs on MSME’s policies. Deliverables to support national capacities started in phase 1, with the launch of the DEPAR portal by ESCWA. This portal provides a wide range of tools and services to MSMEs, such as the MSME toolbox69, with business guides, inspiration stories, start-up tools, external tools and news from partners on innovation, technology, policy-making, etc. It allows visitors to become part of a community of practice, in which participants can share content, build their own community, create events, develop courses, and announce opportunities. One of the ideas for the portal was to provide a one-stop shop for materials from other Regional Commissions too, but, later, it became clear that this would lengthen the path70 for MSMEs and policymakers to access materials from their respective regions and this was not implemented. Figure 8 indicates a growing number of users to the portal, with about 2000 users four months after launch, and about 3000 users one year after launch. Most of these users are from Jordan (59%, according to data from 20 June 2023). There was no qualitative information to explain the quantitative trend.

Figure 8: DEPAR number of users.

Source: ESCWA.

62. Other deliverables under outcome 1.A included e-learning courses based on the EPF; a series of webinars to policymakers on entrepreneurship policies for MSME resurgence; technical assistance in designing and implementing national entrepreneurship policies; the creation of the Asia-Pacific MSME policymakers’ network and follow-up workshops; the creation of a knowledge hub on policy measures, tools and practices on MSMEs’ resurgence; regional workshops to promote dialogue on women’s role in small-scale and informal cross-border trade and tailored recovery; development of the ‘Compendium of Renewable Energy Technologies’ with an online toolbox and training materials on green recovery of MSMEs; policy roundtables with communities and SMEs; and design and implementation of entrepreneurship policies and strategies based on regional assessment. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 74% across project dimensions (Figure 9). It is noteworthy that, given the small response rate, survey results need to be interpreted with caution, keeping in mind that these findings reflect the perceptions of those who responded to the survey questions, and are not representative of all participants (See the methodology section for details).

Figure 9: Effectiveness results of outcome 1.A (n=51 respondents).

69 https://depar.unescwa.org/msmetoolbox. 70 This would increase the need to liaise with ESCWA to upload information according to regional needs (i.e., more time to have information available online), and could impose a barrier to MSMEs and policymakers from other regions to access their regional information (i.e., by adding ESCWA as an intermediary institution between local stakeholders and their respective RECs).

39

Source: Independent evaluation survey, 2023.

63. In the second half of 2021, the 5-week UNCTAD/UNITAR EPF e-course launched with the objective of enhancing the capacity of policy makers to provide policy response and support measures on MSMEs for post-COVID-19 resurgence and develop longer-term strategies for a resilient and competitive MSME sector. One of the course contents explicitly addressed how the environment for and role of entrepreneurship evolved during and in response to the COVID-19 pandemic. It was developed in English, French and Spanish, and covers all five pillars of the EPF. It was preceded by the launching of webinars with the participation of UNCTAD officers, UNITAR representatives, policymakers, and thematic experts. The UNCTAD / UNITAR 2021 Summary Report of this EPF online course presents data for each of its three versions. Results indicate a gender balance between the number of male (n=221) and female (n=223) registrants to the course in the English version, a higher number of female registrants (female=65, male=48) in the Spanish version, and a higher number of male registrants (male=78, female=52) in the French version. Between 5-7% of registrants indicated another gender in the three course versions. The majority of survey respondents, in the three versions, indicated that the course presented new information (>75%) and was relevant to their jobs/occupations (100%). The likelihood of survey respondents applying the information acquired from the course is above 96%, and all respondents would recommend the course to a friend or colleague. This e-course supported UNCTAD’s assistance to Uganda, Seychelles and South Africa in designing and implementing national entrepreneurship policies (NES) based on the EPF. The NES of South Africa focuses on Youth Entrepreneurship, the NES of Uganda on migrant and refugee entrepreneurs, and the NES of Seychelles on the marine biotechnology sector with reference to youth entrepreneurs. Although there was an impact assessment of the NES implemented before the COVID-19 pandemic in The Gambia, this was not delivered due to the budget cuts in phase 3.

64. A set of deliverables by ESCAP included the launch of the terms of reference of the Asia-Pacific MSME Policymakers’ Network (February 2021), followed by a concept note. Three online regional dialogues of the Network were organized by ESCAP in partnership with UNCTAD and ASEAN secretariats. They covered the digital economy (including how to support women entrepreneurs through ICT, business skills training and innovative financing), the role of competition policy in strengthening the business environment, both in March 2021, and the launch of the Policy Guidebook for MSME Development in Asia and the Pacific71, in May 2022. Documents produced to support the Network include a paper on the digital economy72, UNCTAD’s broad survey on the impacts of the COVID-19 on the business environment, particularly in relation to competition issues (2020)73, and the regional report ‘The Role of Competition Policy in Strengthening the Business

71 This book describes the MSME contribution to achieving the SDGs, with comprehensive and practical information to policymakers on MSME development. 72 Callo-Müller, M. V. (2020). Micro, small and medium enterprises (MSMEs) and the digital economy. 73 UNCTAD (2020). Covid-19: Firmer Action Needed to Better Protect Consumers. [online] https://unctad.org/news/covid-19-firmer-action-needed-better-protect-consumers.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

provided access to information and best practices on government support measures for post COVID-19

recovery

improved my knowledge on entrepreneurship/MSMEs policies design and implementation in the context of

post COVID-19 recovery

contributed to identifying new/improved policy measures for MSME promotion in post COVID-19

resurgence

The project...

Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

40

Environment for MSMEs in the ASEAN Region’ (2021)74. Following up on the second meeting, there was a report with recommendations to governments, competition authorities, development agencies, and multiple stakeholders on competition policy measures in a pandemic recovery context. Assessment data is available for the two first meetings and shows that more than 75% of the respondents indicated enhanced knowledge and skills, allowing them to effectively receive from and share information with others in the region regarding MSME policies (>67%), and providing opportunities to enhance regional cooperation in competition policy and law (>69%). Comments from participants referred to the advantages of sharing experiences with other countries, learning how they try to overcome MSMEs’ problems during COVID-19, and providing ideas on how to promote digitalization to MSMEs via education. Interview data indicates that the Network became inactive afterwards, since ESCAP dropped the MSME agenda after the project to focus on other demands.

65. In September 2021, the ‘UN-ESCWA Entrepreneurship Community Workshops and Policy Roundtables – How to improve existing government and non-government programs’ took place, with a set of interactive discussions on existing government programs, existing non-government programs, needed government programs, and needed non-government programs, with follow-up recommendations on these areas. Specifically on the health economy, a set of online conferences on ‘Entrepreneurship in the new health economy’ were delivered in October 2021. One targeted the Asia-Pacific region75, with 45 attendees, another targeted Latin America76, with 25 attendees, and another, Africa77, with 33 attendees. They were organized in partnership with private sector organizations in the health sector for policymakers. It is noteworthy that, except in the African event, these conferences did not include policymakers among their panellists. These conferences generated inputs to the publication ‘Entrepreneurship & innovation in the new health economy’78, launched by UNCTAD in 2022. This document highlights the role of SMEs and start-ups in the new health economy in developing countries, with main findings on the role of regulation in the e-health sector, and the role of public institutions in building bridges between different stakeholders.

66. The resilience and competitiveness of MSMEs in post-COVID-19 resurgence (outcome 1.B) improved through a large set of Empretec-related activities (including trainings tailored to low literate entrepreneurs), cross-border trade workshops for female traders, adaption of the Farming as a Business program, and country-tailored guidelines and best practices on circular economy and sustainable resource management. Outcome 1.B is the one with the highest number of deliverables (n=28), each encompassing sets of activities and supporting MSMEs in multiple countries. They can be grouped in four packages: Empretec (responsible for most of these deliverables and demonstrating the effectiveness of UNCTAD’s pre-existing capacity to deliver trainings, including onsite, during the COVID-19 pandemic), informal and small-scale cross-border trade for women traders, UNCTAD’s Farming as Business program, and Guidelines and best practices on circular economy and sustainable resource management79. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 68% across project dimensions (Figure 10).

Figure 10: Effectiveness results of outcome 1.B (n=39 respondents).

74 ESCAP (2021). The Role of Competition Policy in Strengthening the Business Environment for MSMEs in the ASEAN Region. 11 February 2021. NOTE: there is no authorship in the document. This reference may need revising. 75 Scaling innovation in the new health economy – lessons learned from the Asia-Pacific region. 76 The rise of medtech in Latin America. 77 Unlocking digital health innovation in Africa. 78 UNCTAD (2022). Entrepreneurship & innovation in the new health economy. Geneva: UNCTAD. 79 Given the higher emphasis on government advice, more details on these guidelines are in the highlight box on the contribution of the Surge project to the country’s government responses to the COVID-19 crisis regarding the resurgence of MSMEs.

41

Source: Independent evaluation survey, 2023.

67. Effectiveness evidence of the Empretec package of deliverables indicates that development of the updated online training course (E6DM) was a strategy to facilitate post-COVID-19 entrepreneurship recovery. Ten Empretec webinars, jointly organized and delivered by UNCTAD and Empretec Centres between April and December 2020, were attended by Jordan, Malaysia, Ghana, Nigeria, Russia, South Africa, The Gambia, Argentina, Benin and Zimbabwe. In tandem, 5-day E6DM TOTs were delivered in seven batches with participants from 15 countries80, between June and October 2020. In total, 65 men and 32 women participated in these trainings. Training assessments show positive results, with more than 85% of the respondents rating the course content as good or very good, and as a useful opportunity to progress as an Empretec trainer or trainee trainer. The 21-day TOT, delivered in English, French and Spanish in November 2020, had 35 participants from African countries81 and Romania, and 35 participants from Latin America82. The content was assessed as good or very good by more than 83% of the respondents (16 in Africa and Romania, and 25 in Latin America), and useful for their progression as Empretec facilitators (>91%). Training survey comments highlighted, for instance, the online approach, the follow-up and participant-centered approach, and the possibility of interacting and sharing opinions with colleagues as the most interesting aspects of the training. In sequence, ten 6-day Entrepreneurship Training Workshops (E6DM) were delivered in seven countries83 between September 2021 and July 2022. The total number of participants across countries was around 30084, with about 50% of women (estimated on the data available) and some countries reporting the participation of youth entrepreneurs up to 54% (i.e., Ghana). When post-training assessment is available, results are positive with more than 88% rating the overall workshop as effective/useful/good or very good/excellent85. More than 84% rated the workshop content as good-excellent/relevant. Good-very good improvements in entrepreneurial skills ranged between 43% and 100%, good-very good relevance of the training to run the business was above 80%, and estimated good-very good impact on the future of their businesses was between 33% and 100% in a set of 10 possible impacts. Subsequently, UNCTAD delivered an additional online 1-day E6DM TOTs in September and October 2022, as a pioneer effort to homogenize the application of new versions of the Empretec method to different Empretec Centres and trainers. Participants totalled 109 (64 male, 45 female) from 19 countries86.

80 Argentina, Benin, Cameroon, Ethiopia, Ghana, Jordan, Kenya, Peru, Romania, Russia, Tanzania, The Gambia, South Africa, Venezuela, and Zambia. 81 Benin, Botswana, Ghana, Mauritius, Romania, South Africa and Tanzania. 82 Argentina, Chile, Colombia, Ecuador, Panama, Peru and Venezuela. 83 Benin, Nigeria, Cameroon, Zimbabwe, The Gambia, Ghana, and Malaysia. 84 Note: Some country reports do not inform the total number of participants. 85 Measurement scales could be different between countries. 86 Angola, Argentina, Botswana, Colombia, Ghana, Ecuador, El Salvador, Jordan, Kenya, Malaysia, Mauritius, Mozambique, Namíbia, Panama, Romania, Saudi Arabia, South Africa, Tanzania, and Uganda.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

provided information useful for starting a new business or expanding a business venture

improved my knowledge on entrepreneurship and business development

contributed to identifying new/improved measures to increase business sales

The project...

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

42

68. Other innovations to the Empretec programme implemented in 2021 were the development of a version for low-literate people and a full online training. Between April and May 2021, UNCTAD delivered three sessions of the TOTs for low-literate people, in English, French and Spanish, for 62 male and 58 female participants. Trained trainers, then, delivered three 8-day workshops for low-literate people, between March and July 2022, in Benin, Malaysia and Uruguay. The total number of participants was 72, 57% female. The assessment was positive87 in relation to the workshop in general, its content, and its relevance for the participants’ businesses. Regarding the development of the online Empretec training course using gamification and other IT-based tools88, there is evidence of three pilot workshops with entrepreneurs, delivered in hybrid format and in partnership with SEBRAE (Empretec host institution in Brazil) between December 2021 and August 2022. Each pilot tested the effectiveness of the digital tool and led to the identification of, and solutions to, technical issues. Interviews for this independent evaluation conducted with Empretec centers and UNCTAD staff indicated that the performance of Empretecos in the online course was below face-to-face training. An informant said that “workshops are better when people share in the classroom, with safe environment, with facilitator. Online does not facilitate that.” However, even if the online format is not suitable for a workshop that requires six full days of commitment, some procedures, e.g., tracking participants’ progress, and resources, such as web platform for Empretec centers, can be online. Another challenge of delivering the Empretec training online was conciliating the objective of reaching out to vulnerable beneficiaries with no access to the necessary technology. The solution, then, was to keep Empretec workshops offline. Only the TOTs remained online, to ensure there would be trainers when the presential workshops could be resumed.

69. The Surge project delivered a series of webinars on the role of Empretec on enhancing entrepreneurship promotion in post-COVID-19 resurgence. At the global level, it supported the virtual 7th Empretec Global Summit, ‘Boosting Entrepreneurship: The Contribution of Empretec into the Post-Covid-19 Resurgence of the MSME Sector’, in April 2021. The Summit was attended by over 570 participants from 64 countries and was interpreted in seven languages89. “It was also followed by 783 viewers on UNCTAD Facebook Live and almost 2,000 views were recorded on UN WebTV from 94 countries.” Another global event was the Global Entrepreneurship Congress, in Riyadh, in March 2022. UNCTAD contributed to the Ministerial Roundtable on Entrepreneurship. Regional webinars were held face-to-face in Colombia, in November 2021, with Empretec center’s representatives from Colombia, Argentina and Ecuador and entrepreneurs; and with the francophone Empretec centres, with 68 participants from Benin, Cameroon and Mauritius, in May 2022. Country webinars included an online event with NAWEN in Malaysia, in December 2021.

70. To expand knowledge on relevant topics of the modernized version of Empretec, UNCTAD delivered a series of ad-hoc online courses between August and October 2021. They covered business model generation (64 attendants from nine countries90), design thinking (83 participants from 11 countries91), entrepreneurship and marketing (Seychelles, no attendance data available), capacity-building for women in cross border trade (59 women participants in Kenya and Tanzania), and a 4-day TOT on sustainable businesses and SDGs (62 participants from 10 countries92). Post-training survey data (22 respondents) is available for the last course, indicating 90% of participants were satisfied with the training content and 86% agreed that sustainability as per the 2030 Agenda should be included in their Empretec training activities.

71. An impact survey conducted by UNCTAD in March 2023 collected data from a representative sample of Empretecos in Benin, Cameroon, Ghana, Nigeria, The Gambia and Zimbabwe. The Empretec programme was positively assessed by respondents (8.75 points in 10) and was considered better than the average of other business training tools by 54%. Most respondents reported sales growth after the ETW (67%), and 94% believed sales will increase in 2023-24. Among non-entrepreneurs, 60% reported an increase in their income levels after the workshop, 88% felt more motivated to venture after the ETW, 67% foresaw opening a business in the coming three months, and 91% of the Empretecos believed their employability level had increased after the ETW, a trend even stronger among women. Interviews with Empretec centers for this independent

87 100% of good-excellent ratings in Benin, and an average range between 4.5 and 4.7 in a scale of 5 in Uruguay. The training report for Malaysia did not present specific assessment data. 88 LMBInterativa (2022). Empretec Online. Delivery report. 89 Arabic, Chinese, English, French, Portuguese, Russian and Spanish. 90 Ecuador, Ethiopia, Ghana, Jordan, Kenya, South Africa, Tanzania, Zambia and Zimbabwe. 91 Ecuador, Ethiopia, Jordan, Kenya, Malaysia, Romenia, Saudi Arabia, South Africa, Tanzania, Zambia and Zimbabwe. 92 Angola, Argentina, Brazil, Colombia, Dominican Republic, Ecuador, Guatemala, Peru, Uruguay, and Venezuela.

43

evaluation indicated that, in Zimbabwe, for instance, women and youth facing business difficulties were the majority of those trained. Two of these participants are the founders of the Women in Agriculture Union. They learned how to keep records of funds received from members, and the advantages of doing so. Young entrepreneurs in agribusiness improved product processing knowledge, leading to expansion and new markets, in Zimbabwe. In Argentina, course participants prefer the online course format, even when their access to the internet is hard, because of the flexibility of time and schedule and the possibility to connect with people across the country. The number of participants is much higher in the online version in Argentina. In Jordan, the online TOT targeted vulnerabilities, such as internet connection and digital literacy. The online delivery was a challenge also from the Empretec centres’ perspective. More than three hours of online training was not possible; and the programme is based on learning by doing, i.e., it requires observation of behaviours. The new online tools helped to mitigate this challenge, but they did not always work because of poor IT literacy, people’s commitment, and internet connection.

72. The outputs related to the UNCTAD component on women and cross-border trade, were delivered in phases 2 and 3 of the Surge project. Evidence indicates 6-day presential workshops93 delivered in Zambia (36 participants), Malawi (32 participants), Kenya (64 participants) and Tanzania (64 participants), between February and November 2021. They tackled two main critical issues: lack of knowledge about trade rules and customs procedures, and lack of entrepreneurial skills. Workshop beneficiaries were mainly small- scale/informal cross-border traders, especially women, representatives of cross-border trade associations, representatives from the Revenue Authorities, and other authorities based at the borders. Workshop assessments were positive, with more than 45% of the participants rating that their knowledge about trade rules and customs procedures had improved very much or extremely. Most of them acquired confidence in crossing the border through normal channels (>61%) and will be able to formalize within the next 12 months (>54%). Assessment data for Kenya and Tanzania indicated that more than 80% of the participants rated the programme as excellent, the training content as very relevant for running their businesses, and more than 90% indicated the training will help them improve professional behaviour, management skills, business planning skills, establish a network with other business owners, learn about business opportunities and help learn about funding possibilities. The workshop’s methodology, content and results are compiled in the activity report94 commissioned by UNCTAD. Country-specific guides on ‘Women in informal cross-border trade: A small- scale trader’s guide to trade rules and procedures’ developed by UNCTAD provided tailored information for female traders in Kenya, Botswana, Tanzania, Malawi and Zambia. These guides, except for Malawi and Zambia, contain traders’ obligations regarding COVID-19 measures related to crossing the border and a checklist that includes COVID-19 certificates.

73. A 2022 impact assessment on project activities carried out by UNCTAD on women and cross-border trade95 indicated that 100% of the participants became very or extremely familiar with their rights as cross- border traders, 95% became very or extremely familiar with the obligations of cross-border traders, and 43% acquired considerable knowledge on the use of COMESA and EAC STRs and the SADC Trade Protocols. The enhanced knowledge about the use of official border points was very or extremely helpful for 92% of the respondents. “Some traders reported positive results in terms of increased profits, a larger clientele, and the ability to offer a more diversified range of products” associated with the enhanced use of official border points. Almost all participants (96%) reported that they had adapted their business model. Also, 80% of the respondents considered registering their businesses. Responses were positive on acquiring extremely good or very good understanding of the COVID-related rules (65%). Qualitative data indicated that the training helped trainees to be proactive in protective measures, such as in encouraging customers to wear masks and wash their hands. Regarding new strategies, “some traders turned to digital tools to continue their businesses, including Facebook, Instagram, WhatsApp groups, and online platforms.” Interviews with trainers conducted by this independent evaluation indicated that the results of the training for low literacy beneficiaries in cross- border trade were ‘amazing’, based on a follow-up four months after the training. Proper understanding of the trade rules and procedures, rights and obligations of the traders coupled with upgrading of their entrepreneurship skills helped traders to overcome the adverse trade environment caused by the Covid-19

93 They built on training activities completed by UNCTAD in 2019 in the framework of the project ‘Informal cross-border trade for the empowerment of women, economic development and regional integration in Eastern and Southern Africa’. 94 Tigere, F. (December 2021). Phase III Final Report – Global initiative towards post-COVID-19 micro, small and medium enterprises (MSME) sector: Kenya and the Republic of Tanzania. 95 UNCTAD (2022). Cross-border Trade in the Pre- and Post-Pandemic Environment Evidence from Malawi, the United Republic of Tanzania and Zambia. Geneva.

44

pandemic. Some were able to bounce back and build back their businesses. More information on female traders in cross-border trade is in the specific section on gender and LNOB.

74. The delivery of the Farming as Business (FaaB) programme took place in phases 2 and 3 of the Surge project. Adaptation of this programme referred to the online delivery of the TOT, between February and June 2021. Training material does not refer to the pandemic crisis96. The English version of the TOT trained 25 male and 28 female small farmers from 13 countries97. The French version trained 25 male and 28 female participants from Angola, Benin and Cameroon. The Spanish version trained 9 male and 7 female participants from eight countries98. Post-training assessments, available for the English (15 respondents) and French (4 respondents) versions, were positive. On average, 79%, rated the course content as good or very good, 84% found it a useful learning opportunity, and 95% considered the course useful for them to progress as Empretec trainer or trainee trainer. Following the TOT, a grant agreement between Empretec Ghana Foundation and UNCTAD allowed for training of 65 vegetable farmers in Ghana, in November 2021. A report on this training99 indicated that 90% of the participants were male and 10% female. There is no reference to resilience in post-COVID-19 recovery.

75. The Surge project implemented tools that facilitated MSMEs’ registration and formalization (outcome 2) through an e-registration platform, policy toolkits for governments on formalization, and virtual workshops for knowledge dissemination. Under this outcome, UNCTAD delivered the e-registration platform, implemented it initially in El Salvador (cuentamype100), and expanded implementation to Benin, Cameroon and Mali. DESA developed policy toolkits for the governments of Kenya and The Gambia in streamlining MSME formalization and inclusive delivery of formalization services. The dissemination of these toolkits was with virtual workshops in the two countries. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, but, because the number of respondents was between 3 and 4, no further conclusions can be drawn from our survey.

76. Interview data indicated that the pilot implementation of Cuentamype in El Salvador involved private and public institutions, and the signing of an inter-institutional agreement, considered one of the biggest project achievements in the country. It brought together, for instance, social security institutions, the Ministry of Finance, and municipalities, and attracted the interest of financial institutions in relation to providing credit to MSMEs. The preparation of the platform included the organization of sectoral working tables with these institutions, and a pilot, in 2020, with a group of 20-25 entrepreneurs for feedback, for around 3-5 months. After adjustments, the platform launched in a forum where entrepreneurs could learn about the tool. The project trained government technicians as trainers for the end-users of the platform – 3-4 training days of about 15 nationwide professionals. During technical visits to MSMEs (e.g., women’s owned beauty salons), specific needs of beneficiaries were identified, such as basic knowledge in accounting and individual monitoring, mostly for women. The answer to these needs included the development of a printed simplified booklet on accounting (i.e., a paper version of cuentamype), follow-up visits for six months, and the creation of a WhatsApp group for direct contact with the registry office (CONAMYPE) to get direct support in interpreting financial analysis outputs and suggest improvements to the platform. Out of about 400 trained entrepreneurs101, CONAMYPE estimates that at least half of them use the tool regularly (about 60% of them are women and covering entrepreneurs from 18 to 70+ years old). The e-registration platform was, then, expanded to Benin, Cameroon (guichet electronique) and Mali (guichet unique) in 2021. The online platform increased MSMEs’ registration 91% in Benin (84% women, 181% youth), and 40% in Mali (49% women, 110% youth). Replicating the case of El Salvador, these e-registration platforms are incorporating additional services, such as modifications in business registry and grant applications. Real-time data generated by these platforms have been used to design better entrepreneurship policies.

96 The trainer’s guides (i.e., coffee and dairy farming), in English and French, were produced before the pandemic and their content was not updated to the new situation. The guide on dairy farming in Spanish was issued in 2021 and reproduces the content of previous guides, also not referring to the COVID-19 pandemic situation. 97 Angola, Benin, Botswana, Ghana, Kenya, Malaysia, Myanmar, Panama, Saudi Arabia, South Africa, Tanzania, Zambia and Zimbabwe. 98 Argentina, Chile, Colombia, Ecuador, El Salvador, Peru, Uruguay and Venezuela. 99 Empretec Ghana (Feb.2021). Report on training of farmers on the UNCTAD Farming as a Business (FaaB) programme - Organized for vegetable growers in the Ayawaso West municipaliy – Greater Accra region. Ghana. 100 https://cuentamype.org/ 101 This included entrepreneurs who could not read or write and were not tech savvy. In these cases, entrepreneurs indicated collaborators to take part in the trainings.

45

77. Policy toolkits issued by DESA considered regional and country-based approaches. A regional toolkit was ‘Best Practices: Formalization of Micro-, Small and Medium-Sized Enterprises (MSMEs) in Africa’102, a document that covers a wide range of best practices associated with formalization (e.g., institutional frameworks, regulatory environment, supply and value chains, access to finance, human capital development, and social dialogue). It assembles data from African countries’ best practices, such as Kenya, Nigeria, Rwanda, South Africa, Tanzania, and Zimbabwe, and benchmarks on examples from countries in other regions, such as India, Sri Lanka, China, Argentina and Colombia. Country-based policy toolkits focused on Kenya and The Gambia and included policy documents and virtual or hybrid workshops for knowledge dissemination. In Kenya, the focus was on leveraging policies governing cooperatives of MSMEs, with the issuing of a report103, a toolkit with policy recommendations for the formalization and growth of MSMEs through a cooperative model104, policy guidelines for the formalization of MSMEs in Kenya, modelled on the guidelines for the formalization of cooperatives105, and the toolkit promotion in a virtual workshop in March 2021. A workshop assessment (responded by 60 participants, 41 male, 19 female from African, Asian and Europe countries106) indicated that 98% of the participants were satisfied or very satisfied with the event, and 100% considered the event relevant to their work. Later, in 2022, DESA supported an assessment of the impact of COVID-19 on MSMEs led by women and youth in Kenya 107 and stakeholder consultations, based on these assessments, in a hybrid workshop. For instance, 100% of the 22 respondents (14 male, 8 female) of the assessment of the workshop on the Assessment of the Covid-19 Impact on Women and Youth-led MSMEs in Nairobi, were satisfied or very satisfied with the workshop, and considered it relevant to their work.

78. MSMEs’ access to finance (outcome 3) improved through online training to policymakers and MSMEs (e.g., financial literacy, accounting, SDG reporting), publications, case studies, training of trainers, and national and regional workshops for knowledge dissemination. Outputs under outcome 3 were carried out by UNCTAD and ESCAP. They produced online training material for policymakers on MSMEs’ access to finance, developed training courses for MSMEs for financial literacy and accounting and reporting, including on the SDGs – supported by a guidance –, TOT’s courses on accounting and reporting for MSMEs, published a book on MSME finance, developed case studies on accounting and reporting for MSMEs, and delivered national and regional workshops based on the online materials produced. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 70% across project dimensions, keeping in mind that the number of respondents is low for robust conclusions and this result needs to be interpreted in qualitative terms (Figure 11).

Figure 11: Effectiveness results of outcome 3 (n=10 respondents).

102 DESA (2021). Best Practices: Formalization of Micro-, Small and Medium-Sized Entreprises (MSMEs) in Africa. 103 DESA (2021). Leveraging policies governing cooperatives to encourage the formalization of micro-, small and mediu- sized enterprises (MSMEs) in Kenya. 104 DESA (2021). Promoting MSME Formalization through the Cooperative Enterprise Model. 105 DESA (2021?). Policy guidelines for the formalization of micro-, small and medium enterprises in Kenya. 106 Kenya, South Africa, Burundi, Tanzania, The Gambia, Lesotho, Morocco, Philippines, Indonesia, Germany. 107 MSEA (2022). Assessment of the impact of COVID-19 among micro, small and medium-sized enterprises in Kenya and their resilience mechanism. Nairobi: Micro and Small Enterprises Authority (MSEA).

46

Source: Independent evaluation survey, 2023.

79. ESCAP led the delivery of a book that set out a post-crisis policy agenda for MSME finance, in 2021108. It addresses MSMEs’ finance gaps and the financial landscape for MSMEs before, during and after the pandemic crisis. The book draws policy recommendations and practical suggestions in MSME finance in relation to, for instance, business services ecosystem, regulatory sandboxes, supply chain and trade finance, financing women-led MSMEs, multi-stakeholder collaboration for mutual benefits, and regulatory framework for fintech. Two thematic studies followed-up, in 2022. One was on the role of banks in Asia in lending to MSMEs109, and the other on the role of digital payments110, including a topic on women-led MSMEs and SDG 5.

80. UNCTAD developed the guidance on core indicators (GCI) ‘Implementation of core SDG indicators for sustainability reporting by companies’ in English, Spanish, French, Arabic, Chinese and Russian, and corresponding training material in partnership with UNITAR. This guidance supported GCI workshops on accounting and reporting for MSMEs in Kenya (23 participants), China (150 participants), Latin America (55 participants, 14 countries), Francophone countries (122 participants, 11 countries) and the MENA (79 participants, 4 countries) and Eurasian (200 participants, 10 countries) regions, between February and November 2021. Across workshop sessions, assessments responded by 246 participants (78 male, 47 female111) indicated that more than 90% of them rated the overall course content as good-excellent/satisfied-very satisfied.

81. MSME’s access to technology and innovation (outcome 4) increased through a combination of survey-based reports assessing the impacts of Covid-19 and the Ukraine crisis on MSMEs at country and regional levels, regional seminars for knowledge dissemination and sensitization of policymakers on needs of MSMEs, online courses on the role of technology and innovation for entrepreneurs and policymakers, including on harnessing green technologies, and knowledge products on a range of areas including building linkages between MSMEs and multinational companies. Activities under outcome 4 were led by ECA, through its Subregional office for Southern Africa, and included surveys circulated to more than 3,000 MSMEs across its 11 member states in Southern Africa, to assess the impacts of the Covid-19 and Ukraine crises on MSMEs112, the development of an online training course for entrepreneurs and policymakers on the role of technology and innovation in addressing this impact (December 2021), an online training course on harnessing green

108 ESCAP (2021). Rethinking MSME Finance in Asia and the Pacific: A Post-Crisis Policy Agenda. Bangkok. 109 ESCAP (2022). Small and Medium-Sized Enterprises Lending Approaches: The Role of Banks in Asia. MSME Financing Series No. 6. Bangkok. 110 ESCAP (2022). MSME Access to Finance: The Role of Digital Payments, MSME Financing Series No.7. Bangkok: United Nations. 111 No gender disaggregated information for the Eurasian workshop. 112 The result was 11 country level reports which, at the time of the evaluation, were being finalized and not yet published.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

provided useful information on financial inclusion and MSMEs access to finance

improved my knowledge on accounting and reporting, including on the SDG reporting, and

improved my capacity to manage financial resources

contributed to identifying new/improved measures for financial inclusion and MSMEs access to finance

The project...

Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

47

technologies to enhance MSME competitiveness in Southern Africa (August 2022), two side events at the margins on the SADC Industrialisation Week on strengthening linkages between African MSMEs and Southern led MNCs and on the role of technology incubators in MSME development in collaboration with the SADC Business Council Southern Africa113. An additional unplanned output, at the request of the Government of Mauritius, was a regional seminar on “The impact of Covid-19 on MSMEs in Southern Africa: Country Experiences and a Focus on Building Back Better in Mauritius” was held at the margins of Dubai Expo 2020, in collaboration with the SADC Business Council and the Ministry of Industrial Development, SMEs and Cooperatives of Mauritius. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 83% across project dimensions, keeping in mind that the number of respondents is low for robust conclusions and this result needs be interpreted in qualitative terms (Figure 12).

Figure 12: Effectiveness results of outcome 4 (n=12 respondents).

Source: Independent evaluation survey, 2023.

82. Evidence from the technical report114 of the 4-week online training on the role of technology and innovation for African businesses in addressing the impact of COVID-19 indicated that 41 participants attended the course. Out of those who responded to the post-training assessment, 95% noted that the course modules were relevant to their professional needs and 90% felt satisfied with the relevance of the topic and the modules. Regarding the course on green technologies for African SMEs, the assessment report indicates that 30% of the 155 registrants completed the course successfully by passing the quizzes. Out of these, 45 responded to the post-course assessment. Results indicated that 93% of respondents rated the quality of the course as good or very good, 93% indicated that the content of the course is relevant to their work, and 100% indicated that they will use the knowledge acquired in this course.

83. MSME’s access to markets (outcome 5) increased through activities that combined the production of national and regional assessments, studies and reports with online events for knowledge dissemination of findings, and online trainings for policymakers and MSMEs at the regional level on topics such as competition policies and access to markets, agricultural quality and food loss, and the promotion of export capacity. Outcome 5 is the second in number of outputs planned (n=25) and the first in collaborative deliveries between UNCTAD and the Regional Commissions. Deliverables cover a wide range of activities on competition policies to regulate access to markets during the pandemic crisis and promote the resurgence of MSMEs in post- pandemic recovery. Survey evidence from this independent evaluation corroborates the positive assessments observed for individual deliverables, with approval rates above 66% across project dimensions, keeping in

113 Updated information, received after the finalization of this independent evaluation, indicated that, as of October 2023, the two publications on strengthening linkages between African SMEs and Southern led MNCs and the role of technology incubators for MSMEs development in Southern Africa are completed and being professionally edited. Two other publications, finalized in 2023, are “Opportunities for MSMEs within the Blue and Green Economy: the case of SADC” and “The role of digitalization in MSMEs development in the context of the AfCFTA”. 114 IDEP, ECA (2022). The role of technology and innovation for African businesses in addressing the impact of COVID-19 – Technical report.

0% 20% 40% 60% 80% 100% 120%

provided useful information on MSMEs access to innovation and technology

improved my knowledge on how to increase MSMEs access to innovation and technology

contributed to identifying new/improved measures to increase MSMEs access to technology

The project...

Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

48

mind that the number of respondents is low for robust conclusions and this result needs to be interpreted in qualitative terms (Figure 13).

Figure 13: Effectiveness results of outcome 5 (n=12-13 respondents).

Source: Independent evaluation survey, 2023.

84. A good illustration of coordinated outputs to enhance access to markets to MSMEs is the joint work produced by UNCTAD’s competition and consumer protection policies branch with ECLAC, ESCAP, ESCWA, ECA and ECE. During 2020, multiple regional preparatory meetings between UNCTAD, the Regional Commissions and regional stakeholders discussed how to mobilize participation and build collaboration in designing and implementing these dialogues to raise awareness of the role of competition policies for post- pandemic access to markets by MSMEs. The resulting five online regional dialogues took place between March and April 2021 and had approximately 650 attendees across regions115. This inter-institutional partnership supported the delivery of a global report on the articulation between competition policy and MSMEs (produced in phase I, between June and September 2021)116. This global report provided a snapshot of the challenges MSMEs faced around the world to deal with the impact of the COVID-19 pandemic and drew key recommendations centered around levelling the playing field for MSMEs. It was disseminated with a global policy dialogue, organized by UNCTAD in partnership with ECLAC, ESCAP and ESCWA, in December 2021 (phase 3). This global dialogue explored how competition policy could support MSMEs post-COVID-19, with the presence of representatives and officials of government MSME bodies and competition agency representatives from across the world. It was divided into three sections, one with a focus on digitalization, another on access to finance and government support, and the third on inter-agency coordination between MSME agencies and competition authorities. It was delivered in English, French, Spanish, Chinese, Russian, and Arabic.

115 The UNCTAD-ESCAP-ASEAN dialogue was delivered in English, Thai, Burmese, Bahasa Indonesian and Bahasa Malay and had 149 participants. The UNCTAD-ECA dialogue was delivered in English and French and had 97 registrants. The UNCTAD-UNECE dialogue was delivered in English and Russian and had 140 attendees, 30% from outside the UNECE region. The UNCTAD-ECLAC dialogue was delivered in English, Spanish and Portuguese and had 115 registrants. The UNCTAD-ESCWA dialogue was delivered in English and Arabic and had 150 attendees. 116 UNCTAD (2021). How COVID-19 affects MSME access to markets and competition: A review of key issues and recommendations for future action. NOTE: the previous title is in a press release by UNCTAD and the next refers to the report shared with the evaluation as output F7. Alternate title and reference: UNCTAD (2022). The COVID-19 pandemic impact on micro, small and medium-sized enterprises. Market access challenges and competition policy. Geneva.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

activities and publications provided useful information on MSMEs access to markets

improved my knowledge on how to increase access to markets, including export and integration into value

chains

activities contributed to identifying new/improved measures on consumer protection and competition

activities contributed to identifying new/improved measures on agricultural quality and food loss reduction

The project...

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

49

85. Other cooperation-based and replicated outputs were the three country case studies on the impact of COVID-19 on specific SME sectors in Brazil117, Thailand118 and South Africa119, delivered between March and December 2021. They were implemented with the support of national institutions to ensure a grounded approach in understanding what competition policy is, how it is framed in the country, and how it can impact MSMEs in the context of the COVID-19 crisis. Each case study provided recommendations to competition authorities and representatives and support entities for MSMEs. A global event in June 2021, ‘DA COVID-19 technical cooperation project on Competition Policy & MSMEs’ resurgence’, disseminated the findings of the case studies and promoted policy advocacy on this subject to 238 attendees. Also, five regional webinars with competition authorities and MSMEs’ institutions delivered on how the recommendations of the global and the three national reports could support the resurgence of MSMEs. A follow-up hybrid global workshop, held in Bangkok in 27-28 June 2022, to discuss ‘The Contribution of Competition Policy to the Resurgence of MSMEs post-COVID-19’ had panellists from UNCTAD, ESCAP, DESA, ECLAC, ESCWA and African countries. This event covered recommended practices for digital platforms for MSMEs120 and presented the guidance document for Thai competition and SME institutions121 for 534 virtual participants. It was assessed as relevant (>95% of participants) and effective in enhancing skills, regional cooperation and methods for integrating MSMEs into regional value chains (>95%). This workshop presented the newly designed online course on SMEs and competition policy, primarily for government officials. The country case studies are cited as having inspired the course showing what capacities SMEs needed to engage in better trade practices. This 5-module course was delivered between November and December 2022 to 141 participants (73 male, 68 female) from 41 countries122. The post-course survey indicated that the experience was very/extremely valuable to 98% of the 72 respondents, 81% felt that the course provided them with learning opportunities not available otherwise, 82% felt able to serve the community with what they have learnt, and 68% considered the course relevant to advance in their career.

86. At the level of the indicators of achievement, despite somewhat lacking in specificity, and the high level of ambition and long-term-orientation for a crisis response, considerable progress is observed. The elaboration of the project indicators was reported by informants as one of the challenges in the Surge’s design phase, given the uncertain pandemic context, the lack of clarity on the resources that would be available for each phase of the project, an understanding that the DA requirements for indicators had to reflect development goals (adequate for longer projects123), and time pressure to deliver as soon as possible. As a result, the management decision was to focus on what would be right for MSMEs’ support from each entity’s perspective (i.e., separate streams of work), beneficial for countries, and already available as standard indicators for the entities. Reviews of the initial indicators followed advances in the project and the evolution of the pandemic situation. From phase 2 to phase 3, some indicators were revised to become more ambitious, since there was more time and resources available. According to members of the project coordination team, there was awareness of the broad scope of these indicators, that some of them were not crisis-driven, and it would be difficult to achieve them within a year. Nevertheless, there is also the shared perspective that they are needed to track longer-term effects of the project. A mitigation monitoring strategy by project managers used proxies or partial reporting from specific delivered outputs when short-term impact could not be measured. This is reflected in the Surge project’s final report, concluded in November 2022. Adding to this

117 UNCTAD (2021). Competition and Market Access Policies in the Resurgence of MSMEs in Post-Covid-19 in Brazil. 118 Charoenrat, T., Harvie, C. (2021). Analysis of the impact of COVID-19 on micro, small and medium-sized enterprises (MSMEs) in Thailand from competition policy and market access perspectives. 119 Nsomba, G., Tshabalala, N., Vilakazi, T. (2021). Analysis of the impact of COVID-19 on micro, small and medium-sized enterprises in South Africa. 120 UNCTAD (2022). Harnessing the interaction between digital platforms and MSMEs: Recommended best practices for digital platforms (RDP). 121 UNCTAD (2022). Guidance for action: Supporting MSMEs’ recovery from the COVID-19 pandemic. Prepared for the trade competition commission of Thailand. 122 Kenya, St. Kitts, Botswana, Philippines, Indonesia, El Salvador, Azerbaijan, Trinidad and Tobago, Kuwait, Belize, Brazil, Barbados, Suriname, Eswatini, Burkina Faso, Peru, Malawi. Côte d'Ivoire, Bahrain, Colombia, Russia, Malaysia, Zambia, Ghana, Jamaica, India, Tanzania, Spain, Grenada, Switzerland, Ethiopia, San Marino, South Africa, Nigeria, Senegal, Montserrat, Egypt, Saint Lucia, Zimbabwe, Madagascar, Liberia. 123 It is noteworthy that the DA guidance on the expected outcomes for PRODOCs of joint COVID-19 response projects states that “The outcomes (OCs) describe the changes that are expected to occur as a result of the completion of outputs. The OCs should be achievable within the project’s timeframe and budget and should be specific enough to be measured by the associated indicators of achievement.” Nevertheless, implementers seem to have been driven by an understanding that the PRODOC approval was conditional to a longer-term perspective on outcomes – an understanding that reflected on their indicators too.

50

evidence, this independent evaluation conducted additional desk review with the support of interviewees who shared the results of the latest outputs delivered, and a survey with relevant stakeholders. This evaluation corroborated comments on the project’s final report in relation to finding mismatches between indicators, project phases and some deliverables, as implementing agencies seem to have shifted activities around, repeated the same activities in different outputs or created new ones since the PRODOC. Other activities in the evidence pool were associated with more than one output and are reported here in the output where their content makes more sense. Also, this independent evaluation pooled together evidence from different implementing entities to make an expert judgement of the progress achieved per indicator, not being restricted to the implementing entity associated with specific indicators. This promoted a better global perspective of the results achieved. Importantly, since most quantitative indicators lack a baseline, the baseline used was the number of countries/participants engaged in the activities under that indicator. At a later stage, more evidence on sustainability and impact indicators will be reported in the UNCTAD Division on Investment and Enterprise’s Annual Research and Impact Report. Table 8 summarizes the findings per indicator at the time of this independent evaluation.124 Out of the 26 indicators of achievement, this independent evaluation located evidence of good progress for 11 of them. Some evidence of progress, from some of the implementing entities involved, but not all, or from a limited number of the beneficiary countries/participants engaged in activities under a specific indicator, was found for other 11 indicators. One indicator showed limited progress and activities related to it are continuing in another project. This independent evaluation could not locate evidence for other three indicators. Overall, this evaluation identified levels of progress for 84.6% of the performance indicators. Indicators under outcome 1A are the ones with more progress reported. Outcome 5 is the one with the most uneven performance in terms of evidence of progress identified by this independent evaluation.

Table 8: Level of achievement of the Surge project indicators. Expected Outcomes Indicators

(Important: each indicator was planned to reflect the work of one implementing entity and

was not designed to cover all participating countries)*

SMART* * criteria covered

Level of progress identified

Source of evidence

OC 1A: To improve national capacities on formulating and implementing enabling polices on green, resilient and inclusive entrepreneurship MSME promotion in post COVID- 19 resurgence

IA 1.1. At least 75% of the countries that participated in the project, have adopted and/or implemented a revised/improved policies aimed at building a resilient and competitive MSME sector in post COVID-19 resurgence

MR Significant progress.

DA 2023W Final report, data from UNCTAD, ESCAP and ESCWA

IA 1.2. At least 75% of policymakers that participated in the interventions of this component indicated their enhanced capacity on entrepreneurship/MSMEs policies design and implementation in the context of post COVID-19 recovery

MAR Significant progress.

DA 2023W Final report, data from UNCTAD, ESCAP, ESCWA, survey

IA 1.3 At least 75% of policymakers and other stakeholders of the ESCWA network indicated their improved access to information on best practices on government support measures for post COVID- 19 recovery

SMAR Significant progress.

DA 2023W Final report, data from ESCWA, survey

IA 1.4. At least 75% of policymakers, MSMEs and other key stakeholders indicated their improved access to the ESCWA knowledge hub on issues on the MSME post-COVID-19 resurgence

SMAR Significant progress.

DA 2023W Final report, data from ESCWA

IA 1.5. At least 75% of users of training materials developed under this component indicated their usefulness for their improved capacities in policy- making for enabling MSME environment

MAR Significant progress.

DA 2023W Final report, data from UNCTAD and ESCWA

IA 1.6. Positive feedback of users of publications under this component (assessment reports and other publications)

AR Some progress.

DA 2023W Final report, data from UNCTAD and ESCWA, survey

124 Importantly, the baseline for each indicator was the activity or activities associated with it, rather than the project beneficiaries as a whole.

51

OC 1B: To improve resilience and competitiveness of MSMEs in post-COVID-19 resurgence

IA 1.7. % of Empretec125 trainees expanding or starting a business venture, disaggregated by country and gender, including by people with low literacy

MAR Some progress.

DA 2023W Final report, data from UNCTAD and ECE

IA 1.8. % of Empretec-supported MSMEs able to increase sales in one year, disaggregated by country and gender

MART Some progress. Gender disaggregated data not available

DA 2023W Final report, data from UNCTAD and ECE

IA 1.9. Number of new jobs created by MSMEs which participated in the Empretec programme, disaggregated by country and gender

MAR Some progress. Gender disaggregated data not available

DA 2023W Final report, data from UNCTAD and ECE, interviews

IA.1.10. At least 75% of participants to the entrepreneurship training courses under this component indicated usefulness of the activities for increasing their entrepreneurship and business skills in the related areas by country and gender

MAR Significant progress. Lacking data by gender for some of the training courses.

DA 2023W Final report, data from UNCTAD and ECE, survey

OC2: To facilitate MSME registration and formalization

IA 2.1. Increased number of new businesses registering, disaggregated by country and gender of business owner

R Some progress.

DA 2023W Final report, data from UNCTAD

IA 2.2. Decreased administrative costs to start a business, disaggregated by country

R Some progress.

DA 2023W Final report, data from UNCTAD and interviews

IA 2.3. At least 75% of participants of the workshops on policy guidelines for MSME formalization stated their increased capacity to facilitate MSMEs formalization

MAR Significant progress.

DA 2023W Final report, data from DESA, survey

OC 3: To improve MSMEs access to finance

IA 3.1. Number of participating MSMEs with improved financial literacy and ability to prepare financial statements, disaggregated by country, and gender

MAR Some progress.

DA 2023W Final report, data from UNCTAD and ESCAP, survey

IA 3.2. At least 75% of MSME participants of online training courses state their improved financial literacy in accounting and reporting, including on the SDG reporting, and improved capacity to manage financial resources, by country and gender

MAR Some progress. Gender disaggregated data not available

DA 2023W Final report, data from UNCTAD and ESCAP, survey

IA 3.3. At least 75% of policymakers that participated in the workshop state their increased capacity to facilitate financial inclusion and MSMES access to finance

MAR Significant progress.

DA 2023W Final report, data from UNCTAD and ESCAP

IA 3.4. Positive feedback of users of the publication prepared under this component

AR Significant progress.

DA 2023W Final report, data from UNCTAD, survey

IA 4.1. Number and type of new/improved adopted measures to increase MSMEs access to technology

MR Data not available

-

125 Empretec is a flagship capacity-building programme of UNCTAD for the promotion of entrepreneurship and micro, small and medium sized enterprises (MSMEs) to facilitate sustainable development and inclusive growth.

52

OC4: To increase MSMEs access to Innovation and technology

IA 4.2 Digital community of practice on use of technology and innovation to build resilience to shocks is created with at least 100 members

MAR Limited progress.126

Data from ECA, interviews

IA 4.3. Positive feedback from users of online training materials and assessment report on their usefulness for increased access to innovation and technology

AR Significant progress.

DA 2023W Final report, data from ECA, survey

IA 4.4. At least 75% of participants of training activities under this component indicated usefulness for their increased access to innovation and technology, by country and gender

MAR Significant progress. Lacking data by gender on usefulness for training activities.

DA 2023W Final report, data from UNCTAD, survey

OC5: To enhance MSMEs access to markets

IA 5.1 Number and type of new/improved measures endorsed by policymakers on consumer protection and competition disaggregated by country

MR Data not available127

DA 2023W Final report, data from ECE

IA 5.2 Number and type of adopted new/improved measures on agricultural quality and food loss disaggregated by country

MR Data not available

-

IA 5.3. Increased number of MSMEs in regions targeted by interventions with increased access to markets, including export and integration into value chains, disaggregated by country and gender

R Limited progress. Gender disaggregated data not available

Anecdotal data from UNCTAD

IA 5.4. At least 75% of policymakers that participated in the workshops indicated their increased capacity to improve MSMEs access to markets, including by integration into the value chains

MAR Significant progress.

DA 2023W Final report, data from ECE and ESCAP, survey

IA 5.5. Positive feedback on usefulness of publications under this component (studies, guidelines, videos, recommendation) by users of these publications

AR Some progress.

DA 2023W Final report, data from UNCTAD and ECE, survey

Source: Evaluation team analysis of available evidence. *Given that each indicator refers to the work planned by a specific implementing entity, this independent evaluation was bound by the coverage of activities of the corresponding entity. Nevertheless, whenever possible, evidence from other implementing entities on the same content of the indicator was considered as evidence for that indicator. **Specific, Measurable, Achievable, Relevant, and Time-Bound.

87. The EPF and the cluster-based approach formed an effective framework to organise outputs delivered by different implementing entities under specific outcomes in the Surge Project, and, also, to support the inter-relatedness between outputs whose content cut across different outcomes. Nevertheless, questions remain on the extent to which this approach is suitable for a crisis response (see para. 89). The EPF and the cluster-based approach were introduced in phase 2 of the Surge project as a strategy to provide a global narrative and weave opportunities for collaboration, mostly evidenced by bilateral partnerships between UNCTAD and respective Regional Commissions, and by collaborations between different UNCTAD divisions (e.g., registration and trade). According to members of the project coordination team, UNCTAD already had the EPF as an agreed menu of best practices to support enterprise development, supported by latest resolutions of the UNGA128. Therefore, as the project moved from phase 1 to phases 2 and 3, the EPF components and the cluster-based approach became the design template for the logframe and structured activities around the five outcomes described previously. Activities delivered by different Regional

126 Update received from ECA in feedback to this evaluation report indicates that this work is continuing under the

UNDA13th Tranche project “Innovative approaches for MSME competitiveness to promote trade and inclusive

industrialization in Southern Africa in the Post-Covid context”. 127 There is evidence of national case studies and regional studies collecting measures taken by governments, but these measures do not result from interventions implemented by the project. 128 UNGA resolutions /RES/71/221 and A/RES/73/225.

53

Commissions and UNCTAD’s divisions, then, became conceptually connected, providing the UN with a clear picture of how country-level deliverables fit the global framework. As stated by a member of the PCT, “The original idea was to put together the areas of expertise of Regional Commissions and divisions within UNCTAD under the pillars of the EPF, but each pillar is a different type of product.” It is acknowledged that only UNCTAD has the capacity to deliver a full EPF project at a global level, as this is its flagship intervention model acknowledged in resolutions of the UNGA. Regional Commissions do not have this global mandate on entrepreneurship policy, so the project benefited from their mandates and expertise on MSME development129. Nevertheless, the implementation of the project by the Regional Commissions was constrained by their work programme for the year – previously agreed with governments –, limited staffing resources and online delivery. Figure 14 reflects how the EPF structured the delivery of the project’s outputs by different implementing entities.

Figure 14: Outputs per project outcome and implementing entity.

Source: Final report.

88. An example of delivery that cuts across the EPF pillars is ECE’s series of guidelines and best practices (located under outcome 1.B) that follow a template that covers topics across the spectrum of the EPF: policy, legal and regulations (outcome 1.A), business facilitation and business registration (outcome 2), access to data, information and knowledge, entrepreneurship skills facilitation, access to finance (outcome 3), access to technology (outcome 4), and market access and logistics and supply chains (outcome 5). Other examples indicate the potential for replicability provided by this framework, such as the national entrepreneurship strategies (outcome 1.A), the e-registration platform (outcome 2), and the case studies on competition and access to markets (outcome 5).

89. Interview data with members of the PCT indicates that the Surge’s implementation was consistent with the way UNCTAD implements the EPF, by assessing what exists at country level, making practical considerations on what can be done, and responding to countries’ demands. A risk of this EPF-centred approach in a global crisis response situation is not building enough synergy with partners and facing implementation difficulties imposed by competing local or regional priorities and uneven competencies on MSMEs. Interview findings corroborate this, with informants referring to challenges in balancing regional priorities and competencies with the delivery of outputs and some focal points in the Regional Commissions reporting a lack of knowledge on the role of the EPF in structuring the project. In these cases, project focal points reported the delivery of what was agreed with UNCTAD on specific outputs and outcomes without a coherent conceptual perspective. Some of them felt that decisions on the project were top-down to create a global response, and delivering under the given circumstances was challenging. Informants from different groups of stakeholders shared the perspective that, for a crisis response, most EPF outputs would be low to medium impact, since they follow a long-term development perspective (e.g., the development of national

129 The Subregional office for Southern Africa, for instance, has a mandate to work on MSME development as contained in the outcome statements of its Intergovernmental Committee of Senior Officials and experts Meetings.

0

10

20

30

40

Outcome 1A

Outcome 1B

Outcome 2 Outcome 3

Outcome 4 Outcome 5

Number of outputs per project outcome

UNCTAD DESA UNECA ESCAP UNECE ESCWA ECLAC

54

entrepreneurship strategies). So, in future crises, it would be an advantage to know from the start what EPF and MSME development-related outputs are fit for immediate results and how to implement them accordingly.

How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

90. The Surge Project contributed to the participating country governments’ responses to the COVID-19 pandemic crisis by improving policymakers’ capacity to design and implement enabling policies to MSME resiliency and post-COVID-19 resurgence through studies, technical assistance, training, digital tools, and interventions at the regional level. Survey results from this independent evaluation indicated that 71% of respondents agree or strongly agree that the project contributed to formulate and implement enabling polices on green, resilient, and inclusive entrepreneurship for MSME promotion in post COVID-19 resurgence (Figure 15). Given the small response rate, these findings cannot be generalisable to all project participants and represent the perceptions of those who responded to the question.

Figure 15: Survey responses on the project contributions to formulate and implement enabling policies to MSMEs (n=41 respondents).

Source: Independent evaluation survey, 2023.

91. This was achieved through higher level outputs, such as the Entrepreneurship National Strategies resulting from technical assistance of UNCTAD to the governments of Seychelles, South Africa and Uganda (outcome 1.A) and country-specific case studies with tailored recommendations and follow-up workshops for knowledge dissemination in the Asia and Pacific region (outcome 1.B, see Box 1 below) and Latin America, Southeast Asia and Africa (on the role of competition and consumer policies in Brazil, Thailand and South Africa, outcome 5). The production of assessments on the impacts of COVID-19 on MSMEs at the regional level (Southern Africa in outcome 4, multiple studies in Latin America in outcome 5), followed by policy advocacy supporting inter-institutional partnerships between different areas of government to align policies in efficient and effective ways and learning from exchanges with other countries represent a great contribution to institutional strengthening (e.g., Europe, Latin America). Informants referred to previously ‘unthinkable’ dialogues, collaborations, and policy alignment between different ministries, with policies for the MSMEs becoming part of the bigger inter-institutional government agenda. In Latin America, interview data referred to how SME policy capacity was initially poorly coordinated with other productive development policies. During the pandemic, however, the same institutions adopted a different working methodology by coordinating with other ministries, such as health, economy, and other productive ministries. The intention was to make the most of the money and avoid dispersed actions. The modalities of policy implementation were a) localized, with targets per sector, or certain geographical locations, b) made flexible and adaptable on a case-by-case basis, according to set targets and sectors most affected, and c) with support from local governments. This was confirmed by government officials in El Salvador and Peru. In Kenya, the survey results supported government action in facilitating access to finance to MSMEs through, for instance, the creation of a targeted fund to increase the access of MSMEs to digital platforms. Now, the new government is focused on MSMEs and DESA is supporting the design of a strategic plan aligned with that. In Zambia, the Ministry of Small and Medium Enterprises improved the participation of women in business through various cooperatives.

The project contributed to formulate and implement enabling polices on green, resilient and inclusive

entrepreneurship for MSME promotion in post COVID-19 resurgence

0% 10% 20% 30% 40% 50% 60% 70% 80%

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

55

Learning from other countries in webinars and regional dialogues is reported as an additional contribution to building and strengthening government capacities to respond to the crisis.

92. In relation to providing innovative tools to government, the development and implementation of specific tools, such as the e-registration platforms (outcome 2) in El Salvador, Benin, Cameroon, and Mali improved the delivery of public services to MSMEs through this digital technology. Real-time data on the number of businesses, categorization by type of business, locations, main business activity, gender, age, etc., from the e-government platforms, is reported to be used in the design of better entrepreneurship policies (outcome 4). The e-registration platform increased the number of MSMEs formalized by about 90% in El Salvador130 and Benin131, and 40% in Mali132, supporting mostly women and youth. Benin became the fastest country in the world to register a business simultaneously with 5 different public agencies (in less than 2 hours). These positive results led to requests from these countries to expand the functionalities of the platform, e.g., by providing analytical financial statements, modifications in the business registry, grant applications to access financial resources (outcome 3), and access to social security benefits for employers and employees. The Prime Minister in Cameroon is signing a new Decree on entrepreneurship making online business registration mandatory.

93. Another type of contribution, regarding capacity-building, refers to the development of the online training course on accounting and reporting on the SDGs (outcome 3), with a specific guidance in French, Russian, Arabic, and Chinese and training workshops for MSMEs in English, Spanish, French, Russian, Arabic and Chinese. This approach expanded the outreach of these courses to over 600 participants in 39 countries, indirectly contributing to countries’ advances on the sustainable development agenda. Having these relevant materials available in more than the three official UN working languages represents an additional contribution to governments, since it may expand opportunities for knowledge uptake through other initiatives in the country.

94. It is noteworthy that contributions to governments often take more time than the project duration, since deeper changes require rounds of consultation and chains of approvals. Hence, challenges to make transformational contributions to governments are often beyond the project’s managerial control, such as government’s capacities, institutional changes in governments due to shifts in power or political priorities (e.g., Georgia, Moldova, The Gambia, Brazil).

Box 1: ECE’s guidelines and best practices – from a general approach to supporting country-specific responses. In phase 1, ECE developed two general guidelines and best practices for MSMEs. One of them was on circular economy and sustainable resource management and critical raw material supply chain solutions133 and the other on delivering energy-efficient products and in providing renewable energy equipment134. They refer to the UNFC and UNRMS as a global framework that can help MSMEs to build innovative business models and gain from the new opportunities created by the pandemic crisis. Under outcome 1.B, this expanded to cover a wide set of studies and interventions aimed at supporting governments to develop an enabling environment for MSMEs (outcome 1.A) in the areas of circular economy and critical raw materials, and energy efficiency and renewable energy. Overall, these country-specific guidelines replicate the same structure of the original documents, allowing for country-specific analysis and recommendations, while providing information comparable between countries and sub-regions within the ECE region. They cover opportunities for entrepreneurs, business development managers, technical experts in existing and potential MSMEs, and financiers on the following topics: business facilitation and business registration (outcome 2), policy, legal and regulations (outcome 1.A), access to data, information and knowledge, entrepreneurship skills facilitation (outcome 1.B), access to finance (outcome 3), access to technology (including digitalization, outcome 4), market access, and logistics and supply chains (outcome 5). Recommendations to governments include providing immediate financial support to MSMEs in the event of a crisis (e.g., temporary tax waivers, temporary tax breaks, national financial programs to support MSMEs, offer payment delays, wage subsidies, line of credit and guarantee free loans to MSME), supporting MSMEs in networking and information gathering, and adopting policies to avoid workforce layoffs (e.g., offer employee development

130 In El Salvador, 12,000 new businesses formalized (40% women-led). 131 In Benin, 52,000 businesses formalized between 2019 and 2021, mostly by young entrepreneurs. 132 In Mali, business creations increased from 6,087 in 2019 to 14,796 MSMEs in 2021. 133 UNECE (2020). Guidelines and Best Practices for MSMEs to assure resiliency and progress towards a circular economy in sustainable resource management and critical raw material supply chain solutions. August 2020. 134 UNECE (2020). Guidelines and best practices for micro-, small and medium enterprises in delivering energy-efficient products and in providing renewable energy equipment.

56

programs, such as in digitalization, wage subsidies to cover temporal downsizing and financial support to laid-off workforce). Following the OECD strategic directions for SME development, these recommendations include the development and promotion of 1) an institutional framework and operational environment (e.g., coordination mechanisms for SME policy, collect relevant SME statistics and disseminate the results, improve business closure and insolvency procedures, use targeted public procurement for MSMEs, and a specialized clean energy financial institution to enable MSMEs as a driving force to clean and green economic recovery); 2) access to finance (e.g., public grants, especially for MSMEs working on low-carbon technologies, supply-side financial skills of banks, demand-side financial education for entrepreneurs, credit guarantee scheme, other non-bank financing for SMEs, with a clear governmental guidance); 3) skills and entrepreneurial culture (e.g., training needs assessments, implementation of vocation training policies, access to non-formal training, women’s entrepreneurship); 4) internationalization (e.g., export promotion activities and export-related skills, SMEs to adapt to DCFTA requirements, financial support to exporting SMEs, and SME integration into global value chains); and 5) innovation and R&D (e.g., legal framework for innovation, collaboration between industry and academia, and financial support to SMEs for innovation). Country-specific guidelines on delivering energy-efficient products and in providing renewable energy equipment were produced for Georgia and North Macedonia in phase 2, and expanded to Armenia Albania, Kyrgyzstan, and Republic of Moldova in phase 3. Likewise, customized guidelines on circular economy in sustainable resource management and critical raw material supply chain solutions were produced for Tajikistan and Ukraine in phase 2 and expanded to Bosnia and Herzegovina, Kyrgyzstan, Kazakhstan, and Serbia, in phase 3. Once the guidelines for a country were done, there were regular contacts with the government’s agency responsible for that piece, for dissemination and organization of the online training. These were about half-a-day training sessions, sometimes with presentations from the WB, or UNDP. There were a couple of events by the end of the project delivered face- to-face. Also in phase 3, an updated version of the guidelines and best practices for MSMEs in delivering energy-efficient products and renewable energy was delivered. This update responded to changes in the work environment of MSMEs in this sector in the ECE region, where countries laid out economic recovery through ‘green’ approaches. A follow-up on the original guidelines’ recommendations indicated that “Most of the countries in the ECE region have adopted various economic and operational tools for recovery from the COVID-19 crisis, helping MSMEs in the energy efficiency and renewable energy equipment sector to boost their growth and to bring the clean energy transition across the region. Intervention measures like low-interest financing, innovation grants and funds, operational relief funds, provision of subsidized/common manufacturing units and workplaces, etc. are being issued for supporting MSMEs during the COVID-19 crisis.” The assisted countries can now request technical assistance from the regular programme on technical cooperation to implement the recommendations in these guidelines.

95. The absence of strong policymakers’ networks on policies for MSMEs and of mainstreaming gender- and minority-inclusive approaches hindered the effectiveness of the Surge project in supporting governments’ responses to assist MSMEs recover from the COVID-19 crisis. The two initiatives on building policymakers’ networks, one by ESCAP and another by ECA, showed high relevance and appreciation by country stakeholders interviewed. In the case of ESCAP, after three meetings and a set of associated outputs, such as input studies and follow-up meeting reports, interest on the topic of MSMEs faded away and this network seems to be inactive now. In relation to ECA, the plan was to deliver a ‘Digital community of practice’ (Techni Africa portal) on technology and innovation for SMEs in Southern Africa. Progress was made in developing and validating the digital platform (TechniAfrica) that will host the community of practice, but it had not been operationalized by the closing of the Surge project.135 Another tool with potential to support strategic networks of policymakers is the DEPAR portal. Although it has the digital structure and input materials in place (e.g., the ecosystem maps), there seems to be a lack of institutional effort to build and maintain an active network, as a community of practice. Networks of policymakers could sustain mutual and regional learning on policies to support MSMEs in different sectors, including in cross-border trade.

96. The project could have improved its support to government capacities in cross-border trade by involving government officials, i.e., immigration officers, customs and police with crucial roles on CBT, in the training courses, as trainees, or having a dedicated training session for them. Interviewees referred to their lack of knowledge on CBT and business skills and negative attitudes toward cross-border traders as remaining barriers to facilitating cross-border trade, especially for women. This independent evaluation also found that

135 Update on the operationalization and launch of this platform was provided by ECA in feedback to this evaluation report. The actual operationalization and launch of the digital platform, titled “TechniAfrica”, to be hosted by the SADC Business Council, will be implemented under the UNDA13th tranche project “Innovative approaches for MSME competitiveness to promote trade and inclusive industrialization in Southern Africa in the Post-Covid context”.

57

some sectoral interventions on access to markets were gender-blind by design and they benefited women because a large proportion of workers in those sectors are female (e.g., cocoa and coffee in Peru). Informants referred to a lack of government interest in looking at gender issues amid a pandemic crisis, so implementing agencies, in those cases, focused on their priorities.

What innovative approaches or tools, if any, did the response use, and what were the outcomes and lessons learned from their application?

97. One of the key institutional innovations in the Surge project was the use of adaptive management through an increased consultative process, and a phased approach. The COVID-19 crisis was an opportunity for the DA programme to innovate on how it promotes sustainable development. Its orientation remains geared to the long run rather than to crisis responses, but the efforts to make the Surge project and the other COVID-19 response projects work created momentum to stimulate more joint projects, according to interview informants. They refer to the advantages of learning what other regions are doing and the possibilities to connect through digital technologies, reducing the costs of inviting someone from Asia to an event in Latin America, for instance. Informants also acknowledge the constraints to implementing such collaborations, such as distinct programmes of work in different entities, translation costs, time zone challenges, etc., but feel that the project promoted the interest and motivation for more synergies. The emerging benefits of, and new expertise, on joint projects through frequent consultation with DA focal points and relevant stakeholders (e.g., entities’ staff working on MSMEs) made this modality a good practice to be considered when planning regular DA projects that could benefit from joint implementation. The streamlining of guidelines136 to support the phased approach associated with the Surge project is also an innovative practice. The phased approach was qualified by several UN staff in the implementing entities as ‘unique’ in DA history and seen as one of the main innovative approaches at the institutional level.

98. The Surge Project developed innovative approaches and tools to respond to country needs, such as online delivery of trainings and knowledge-sharing events, the introduction of MSMEs into policy discussions in response to the pandemic crisis, and e-government services. Some of these innovative approaches and tools are being incorporated into the daily work of stakeholders. The most highlighted innovation by relevant stakeholders was the conversion of activities into the online format. Some advantages of this format were the massive extended access to training, inclusive during lockdowns (e.g., e-learning EPF TOT courses delivered, outcome 1.A, Empretec TOT courses, outcome 1.B, courses on role of technology and innovation and green technologies for MSMEs, outcome 4, course on competition policy for MSME resurgence, outcome 5), the opening of opportunities for exchanges between implementing entities (e.g., joint events and participation in each other’s events, all outcomes) and between policymakers (e.g., regional webinars, global conferences, all outcomes), and to bring together multiple stakeholders for wider discussions (e.g., webinars on the health economy in Africa, Asia and Latin America, outcome 1.A). At the content level, an innovative approach praised by different stakeholders was the focus on MSMEs in the context of competition policies (Latin America) and the inclusion of MSMEs in the existing circular economy and resources management tools (Europe) to answer to a global pandemic crisis. This approach had not been thought about and it was transformative to see it associated with the rapid digitalization of services, such as delivery services, and in policy guidelines on circular economy (outcomes 1.B and 5). This brought MSMEs to the mainstream debate on digitalization and competition policies and into ECE’s work on energy efficiency. Innovative online tools that generated rapid results were the e-government and e-registration tools (outcome 2), since they kept public services working during lockdowns and increased the interaction between MSMEs and public services in a digital one-stop shop that can be accessed by computer or mobile phone and saves on paper for accounting books. An additional gain is improving the digital literacy of users.

99. Respondents to the survey conducted by this independent evaluation listed other innovative or unique approaches and tools of the Surge project that they thought the UN should consider replicating or upscaling, with 57 contributions from 48 respondents. Here is a summary of these responses, categorized according to main themes:

136 The timeline for concept note and PRODOC development was shortened, and the concept note guidelines were

shorter than those for regular tranche projects (as the concept note was only subject to approval by the DA Steering

Committee, and not the General Assembly). The PRODOC template still included all the elements of the regular

tranche prodoc, and there was an additional Phase 3 budget proposal.

58

● Leaving no one behind (15 comments): reaching out to resource-poor areas in developing countries and seeking cooperation to do so and following up with them; business that clearly accommodates those with disabilities; supporting the local women and youth-led initiatives to increase accessibility of services by women and girls in hard-to-reach communities (e.g., rural areas); promoting and upscaling global entrepreneurship with e-commerce, especially for marginalized and vulnerable groups; considering the triple impact (economic, social and environmental) of women-led businesses; profiling women-led MSMEs was insightful and helped to develop legal amendments supportive to women employability.

“I live in a rural and poorest province in my country. I was impressed by the reach of the project because while my country (the capital city) is hosting many UN regional offices, UN programmes are unheard of in these parts, though I suspect they are visible elsewhere in the country and region.”

● Capacity-building for MSMEs, entrepreneurs and communities (12 comments): capacitating communities to engage in entrepreneurship; financial literacy training; entrepreneurial training (including on appropriate technology application and innovation); diversification of business opportunity innovations and mentoring of entrepreneurs.

● Scope of the project and mode of delivery (9 comments): online activities (i.e., courses, webinars and information dissemination); large-scale multifaceted project allowing for rapid intervention; the combination of independent study, YouTube videos, exercises, moderated online participation and tests facilitated greater engagement with the material, learning from the facilitator, and peer learning; room for project managers to try out new ideas, to innovate; undertaking a needs analysis and developing programs based on the needs of the MSMEs.

● Access to finance (5 comments): a simplified micro-lending business method, such as funding local small businesses with a day-to-day transaction of lending to collect daily loan offers with affordable interest rates; innovative digital finance solutions and regulatory frameworks to support both financial inclusion and growth of SMEs, especially those owned and led by women (new perspective for the private sector); increasing non-refundable investment support for SMEs in developing and underdeveloped countries; supporting MSEs to access grants; development of investment attraction mechanisms for the energy system.

● Institutional capacity-building and national capacities for policymaking (5 comments): opportunity or skills-sharing from neighbouring countries and building business relationships and Empretec relationships; face-to-face courses for policymakers to support SMEs; improving cooperation between countries; hybrid policy dialogues including the private sector; improving private sector governance to make it an engine of growth.

● Access to markets (4 comments): digitalization in a globalized world; reducing border barriers; the role of competition policy in the economic recovery of MSMEs (e.g., MSMEs continue to face challenges related to their interaction with bigger counterparts in the agricultural sector and current competition laws do not present effective solutions); the work dedicated to the analysis of structural barriers to trade helped to identify measures necessary for facilitation of exports in Africa.

● Guidance for crisis response (4 comments): recommendations to respond economically to a crisis; emphasis on sustainable and inclusive development; studies might contain innovative and/or unique conclusions and recommendations (should upscale and replicate); the whole approach was innovative, enabling one-stop shop for mechanisms and best practices towards post-COVID-19 resurgence of the MSME sector.

● Access to technology and innovation (2 comments): unique in tackling the novelty of COVID-19 for future resilience (upon lessons learned); the project's efforts to leverage technology and digital solutions for MSMEs are forward-thinking.

● Better policies and government strategies (1 comment): improving EE/RES (energy efficient/renewable energy) secondary legislation.

100. Challenges to the institutionalization of some of these innovations include the digital gap and the lack of technological infrastructure and literacy. Two examples illustrate these challenges. First, the pilot tests of the online version of the Empretec training indicated that, even if online training had worked well in training of

59

trainers’ courses, this is not the case for entrepreneurs. Empretec is a lengthy training programme that includes the observation of behaviour and the performance of team tasks, and the online format does not favour these characteristics of the program. Moreover, the course’s online games and activities would not be supported by the bandwidth and the computers the entrepreneurs have access to. Second, the implementation of the e- registration platform in El Salvador required many intermediary – presential and paper-based steps – to include female entrepreneurs of low literacy and technological skills. Implementers developed a paper version of the platform, where entrepreneurs could write down their financial information until they developed the financial and technological skills needed to use the platform. They needed a 6-month follow-up with visits by the implementers and, during training, they would require their help and the help of younger entrepreneurs to, for instance, create an e-mail account and a password and access it afterwards. Another challenge, at the implementing entities’ level, is access to digital platforms for large conferences and meetings. ECE, for instance, depended on UNDP to set up Zoom meetings with interpretation or had to hire interpreters, since the tools available to them would not offer this service.

5.4 Sustainability

What measures were adopted to ensure that outcomes of the response would continue after the project ended?

101. From the perspective of the implementing entities, the Surge project sowed the seeds for joint projects that are being planned or may emerge in the future. Lessons learned from the responses to the COVID-19 crisis seem to be part of a process of internal reflection by the DA-PMT to support better the design and evaluation of joint projects. The lessons learned from the 5 COVID-19 response projects will be reviewed and discussed at a meeting of the DA Network, as soon as all of these project evaluations are completed. Meanwhile, the management response to the final evaluation of the DA T10 Programme on Statistics and Data137, which report138 also took into consideration the impacts of the COVID-19 pandemic crisis, addresses actions to be taken in the course of 2023 and 2024 to improve and develop guidelines on planning, monitoring, and evaluation of joint projects, especially those with budgets above USD 1 million. In June 2023, DA-PMT expressed its intent through interviews to do more joint projects with a long-term view and responding to the evolving world. Since funds are non-earmarked, they can respond to the mandate of UN entities, including by operationalizing the HQ’s analytical work by cutting silos. Joint projects currently in the design phase (16th tranche) refer to the triple crisis – food, water, energy – and the food crisis response for Ukraine. The former has a global scope, covering urban resilience, energy security, financing etc., with a regional perspective (e.g., the climate crisis plays out differently in Europe and in Africa). DA-FPs interviewed acknowledge the uptake of learning from the COVID-19 response in the DA call for crisis-response projects. They refer to faster procedures, with an improved system of calls for proposals supporting the development of concept notes, project documents, and improved templates that are simpler and more concise. Reported improvements include clearer and more specific feedback of the DA-PMT to proposals, e.g., details on indicators, countries, logframe, including suggestions for aggregated activities. At the regional level, informants referred to how partnerships between Regional Commissions in implementing the Surge project have promoted learning about what others are doing and raised interest in seeking further partnerships. Some of the key factors that might limit or facilitate such joint initiatives, under regular project planning conditions, are the availability of resources, the Regional Commissions’ mandates, and their priorities in a moment in time. Future evaluations of joint projects that are planned under such regular conditions (i.e., not called for to a swift response to a major unpredictable global crisis) might benefit from considering evidence on these and other factors to demonstrate the added value of joint projects under more predictable global and regional contexts.

102. There is evidence of the sustainability of e-government tools, training activities and regional and national interventions, with measures related to expanding the functionalities of digital tools, training courses and the formalization of agreements between partner institutions. Across regions, e-registration platforms are being increasingly used by entrepreneurs, mostly youth, with their potential being acknowledged by MSMEs, governments and other relevant stakeholders, such as banks. The result is new requests from different countries to expand the functionalities of these digital platforms and the higher probability that they will remain active in the future. In Cameroon, for instance, the government is signing a decree to make e-registration

137 https://www.un.org/development/desa/da/wp- content/uploads/sites/52/2023/05/MR_to_Final_Evaluation_Report_1617A.pdf 138 https://www.un.org/development/desa/da/wp-content/uploads/sites/52/2023/01/DAT10-Programme-on- Statistics-and-Data-Final-Eval-Report.pdf

60

mandatory. Surge’s managers referred to the e-government platforms as a tool that can be available in crisis response to keep government offices operational. However, there is awareness that these platforms may not respond to all types of crisis.

103. In Southern Africa, the Surge project strengthened relationships between the ECA’s Sub-regional Office for Southern Africa and the SADC Business Council, with joint meetings on MSMEs being held to this day139. Recommendations from the side events held at the SADC Industrialisation Week 2021 and funded under the Surge project were incorporated into the key priority areas for stakeholders, including member states, in the Lilongwe Declaration, issued at the 5th Annual SADC Industrialisation Week (November 2021), under the section on SME and local development, gender, and youth.

104. The project raised the opportunity to pilot an UNCTAD/Empretec Farming as a Business training with small-holder vegetable farmers in Greater Accra, Ghana. However, despite the pilot’s success in empowering farmers who embraced the improved practices with measured increased yield and income, the sustainability of this pilot is not ensured, since its small scale may prove challenging in sustaining access to markets and pro-poor initiatives. In Nigeria, the Entrepreneurship Training Workshop has been recommended by former participants from the government, bringing in more government agencies and state governments to the program, thus enhancing its sustainability.

105. In Latin America, sustainability is rooted in strengthened links between government officials and their stakeholders, such as MSMEs, business associations, and training centers. Based on the courses of the Surge project, ECLAC has been supporting subregional and national initiatives for training MSMEs, using the tools generated to help and promote greater incorporation and formalization of MSMEs in the region, with special emphasis on those led by vulnerable groups such as women and youth. This follow-up work involves different divisions at ECLAC, such as international trade, social development and gender. Also, as a follow-up to the online generic courses on cross-border trade, ECLAC received sub-regional and country requests for tailor- made courses. This generated a customized course for Central America, organized in collaboration with the Regional Center for the Promotion of MSMEs (Cenpromype), on “the "Development of strategic capacities on public policies for electronic commerce" (2022140). This was followed by the planning of a Portuguese version of the course for Brazil.

106. In Thailand, after the end of the Surge project, the SME authority (OSMEP) and the Competition authority (OTCC) signed an agreement in August 2023 that incorporates SMEs into competition policies. This is evidence of the implementation of recommendations from the national case study discussed in the ‘Formal consultation between UNCTAD and Thai institutions: OTCC, NESDC, OSMEP, ISMED and SME Development Bank under the UN COVID19 Project: SME resurgence’ in which representatives of these two institutions were present.

107. The beneficiaries’ perspective on the potential impact and sustainability of results at the MSME level of the Surge project indicates potential long-lasting effects on resilience and competitiveness in post-COVID- 19 resurgence, through high-level achievements in each project outcome. MSMEs and government officials report on implementing activities learned during capacity-building, advantages of the new registration facilities, improved management practices, access to knowledge, empowerment of women and youth entrepreneurs, preparedness to respond to crises through innovation, access to finance and markets, and improved capacities of policymakers. Across the board, 73% of 106 survey141 respondents agree or strongly agree that the project contributed to improving the resilience and competitiveness of MSMEs in post- COVID- 19 resurgence (Figure 16). Given the small response rate, these findings cannot be generalisable to all project participants and represent the perceptions of those who responded to the question.

Figure 16: Contribution of the project to improve the resilience and competitiveness of MSMEs in post-. COVID-19 resurgence in a sustainable way (n=106 respondents). Source: Independent evaluation survey, 2023.

139 The latest one was a 3-day regional meeting, held in June 2023. ECA partners with SADC Business Council on technology and innovation for MSMEs in Southern Africa | United Nations Economic Commission for Africa (uneca.org) 140 136 participants mainly from Belize, Costa Rica, Dominican Republic, El Salvador, Guatemala, Honduras, and Panamá. 141 Survey conducted by this independent evaluation in July-August 2023, responded by responded by government, MSMEs, and other private sector-related stakeholders (e.g., business associations, commercial registry offices).

61

108. Most of these respondents also agree or strongly agree that the project impacted on better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups (64%) (Figure 17). Given the small response rate, these findings cannot be generalisable to all project participants and represent the perceptions of those who responded to the question.

Figure 17: Contribution of the project to better outcomes for women and youth entrepreneurs and other vulnerable groups in a sustainable way (n=105 respondents).

Source: Independent evaluation survey, 2023.

109. Survey results confirm the predominance of positive assessments of the potential impact of the project in each outcome. Because of the low number of responses in outcome 2 (between 2 and 3 respondents), outcome 3 (9 respondents), outcome 4 (11 respondents) and outcome 5 (between 8 and 9 respondents) to the questions on the potential impact of the Surge project, these results are considered as qualitative information that reflects the assessments of those who responded and do not represent a robust finding in quantitative terms.

110. Figure 18 presents the survey results for the impact of outcome 1.A, where 71% of respondents agree or strongly agree that the project contributed to the formulation of better policies for MSMEs.

Figure 18: Contribution of the project to formulate and implement enabling policies for MSME post-COVID- 19 resurgence in a sustainable way (n=41 respondents).

0% 10% 20% 30% 40% 50% 60% 70% 80%

The project contributed to improve the resilience and competitiveness of MSMEs in post COVID-19 resurgence

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

0% 10% 20% 30% 40% 50% 60% 70%

The project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and

marginalized groups

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

62

Source: Independent evaluation survey, 2023.

111. Figure 19 shows moderate impact of the project on creation of new jobs, increases in sales, and creation of new or expansion of existing business ventures, with between 42% and 57% of the survey respondents agreeing or strongly agreeing with the survey statements (outcome 1.B).

Figure 19: Contribution of the project to improve the resilience and competitiveness of MSMEs in post-COVID- 19 resurgence in a sustainable way (n=33 respondents).

Source: Independent evaluation survey, 2023.

112. Qualitative survey responses regarding the potential impact and long-lasting effects of the project for beneficiaries were mostly about gains to MSMEs’ resilience. Out of 47 comments, 25 referred to impacts for MSMEs. They cover multiple ways in which the project supported MSME’s survival and resilience. The main service of the Surge project was capacity-building for entrepreneurs, such as in relation to registration, internal management practices, good guidelines towards circular economy, knowledge sharing about critical issues (e.g., market dynamics and export opportunities and barriers), empowerment of women (incl. women with disabilities) and youth to do business, access to support mechanisms, and contribution to community development (incl. on green technology), and preparedness to respond to crises through innovation and reaching out to consumers. Regarding MSMEs’ operation, comments refer to increased access to finance and support programs, economic improvement through income generation activities, methods to improve access to markets and supply chains through innovation, facilitation of business operation through e-commerce and digitalization, and contribution to increase the number of MSMEs. The 16 comments on the results of the project that were beneficial to governments covered policy advice on national MSME policies (e.g., e- commerce, law enforcement, sustainable development, gender-responsive support, MSME formalization,

The project contributed to formulate and implement enabling polices on green, resilient and inclusive

entrepreneurship for MSME promotion in post COVID-19 resurgence

0% 10% 20% 30% 40% 50% 60% 70% 80%

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

0% 10% 20% 30% 40% 50% 60%

contributed to expanding my business venture or to start a new business

supported my business increasing sales

contributed to the creation of new jobs

The project...

Do not know Disagree/Strongly disagree Somewhat disagree Somewhat agree Agree/Strongly agree

63

new support schemes for MSMEs), e-registration, stimulus to public-private partnerships, awareness of the leveraging power of ICTs for MSMEs, awareness of the role of rural areas for development, and grounded work through project’s partnerships with local institutions.

113. Another source of evidence for sustained results was the survey responses on the ways in which beneficiaries use the knowledge or skills acquired through the project. Three main categories of knowledge/skills use emerged out of 71 comments:

● Use of knowledge or skills to improve MSME policy (22 comments): project beneficiaries reported that they had been raising MSMEs’ issues in new projects to respond to the COVID-19 crisis (e.g., introduction of entrepreneurial training, business counselling and access to credit), improving policies for youth employment and community development, incorporating MSMEs’ challenges in accessing markets and finance into work in competition law, mainstreaming sustainability principles in MSMEs’ development initiatives, supervising partnerships between MSMEs and larger companies, onboarding MSMEs to the e-commerce platform, easily assessing women’s problems in farms, establishing priority policies for women-owned, female-intensive businesses and social enterprises, using analytical tools (E-view, SPSS, GIS) to identify policy trends and priorities for MSMEs, sharing best practices within the organization and with other partners (e.g., the critical role of cooperatives in formalization). An unexpected impact was on increased exchanges between policymakers and the UN through presentations of the work they have done, and another was on changing cultural relationships with the environment:

The ocean economy is not very well explored in my area. The ethnic groups have always avoided water out of cultural and spiritual reverence. My development programmes never included the ocean economy until after this project. I have started an awareness initiative that presents the ocean as a possible and lucrative source of livelihood. I believe the initiative is making a breakthrough because locals (though still very few) are beginning to lodge applications for fishing licenses to the authorities.

● Use of knowledge or skills to strengthen MSME resilience (24 comments): project beneficiaries

reported that they have been applying their new knowledge in supporting and mentoring MSMEs in topics such as financial literacy, sustainability and ecological concerns, product exportation, and diversification of business opportunities. These consulting services have been assisting family enterprises too. MSMEs reported that they have been applying the new knowledge in their business growth plans and accounting systems, in designing and upscaling projects within the circular economy and the energy sector, creating an association to buy larger volumes from suppliers, applying entrepreneurship competencies to avoid fraud, diversifying production with facial masks for COVID-19, improving production by reducing pollution, implementing biosecurity protocols, and using digital technologies to reach out to prospective clients, do business marketing, expand collaboration with partners to participate in tenders and expand the business activities. Two unexpected impacts were a beneficiary who started an NGO to focus on climate change, and another who installed wi-fi internet in his village using solar panels to ensure that they could continue working remotely.

● Use of knowledge to take training forward (25 comments): beneficiaries reported that they are taking training forward either by taking further training (e.g., on green technologies, computer training) or by training others (e.g., teaching at vocational courses, designing new courses for MSMEs about internationalization and digitalization, and sustainable development, developing new training material for national counterparts, business development services’ offers). They had also been using what they learned to explain to work colleagues how to operate more effectively and efficiently, to do female entrepreneurship advocacy to other women entrepreneurs, and to protect their families from COVID- 19.

114. Challenges for beneficiaries to use their new knowledge or skills refer, mostly, to lack of financial resources, follow-up mentoring, and a favourable business environment. Fifty-three survey respondents indicated their difficulties in making use of the knowledge or skills acquired through the Surge project. The main difficulties were around unfavourable business environment for MSMEs in the country or weak business ecosystems, lack of local resources (policies, technologies, access to remote regions), financial constraints

64

(including barriers to access financial resources by entrepreneurs and governments), turnover of government officials, low political interest and commitment from senior management, limited technical skills and resources, lack of country data for effective policy analysis, lack of education for sustainable development, lack of entrepreneurial mindset in the community and among small business owners, lack of skilled workers, lack of post-training follow-up and mentoring, how to transfer knowledge to complex country situations, challenges to coordination and synergies between relevant agencies, and internet costs.

115. Beneficiaries’ suggestions for ways in which the UN could support MSMEs in their countries covered continuing to invest in capacity-building activities, facilitating access to finance, supporting inter-institutional coordination inside the country’s government bodies, and increasing interventions at the local level. Survey respondents suggested numerous ways in which the UN could support MSMEs in their countries (72 comments). They can be grouped into the following main categories: continue developing capacities of all actors (i.e., governments, MSMEs, other national partners, such as chambers of commerce and business development services) through training, mentoring and webinars, strengthen or contribute to facilitating access to finance for the MSMEs (including seed money, grants, access to credit, etc.), support improving the coordination across ministries of MSMEs-related policies, and increase interventions at local level (i.e., supporting NGOs and MSMEs outside of the main cities, supporting the organization of cooperatives, launching communication campaigns for information dissemination), conducting more regional and country-specific studies (e.g., policy papers, sectoral approaches), and provide tools to support internationalization of MSMES (e.g., network program). There is great emphasis, both in terms of number of comments as well as in relevance for inclusive and sustainable development, on the inclusion of women, youth, persons with disabilities and rural entrepreneurs across these suggestions.

5.5 Gender, Human Rights, and Leave No One Behind

116. This section explores the extent to which gender equality, human rights and leaving no one behind perspectives were considered in the design and implementation of the project.

To what extent were perspectives on gender equality, protection of human rights and reaching underserved groups integrated into design and implementation of the project? What results can be identified from these actions?

117. The Surge Project´s design was guided by two key UN documents that provided clear direction for the integration of gender, human rights, and LNOB: The UN Secretary-General’s Shared Responsibility, Global Solidarity report and the UN framework for the immediate socio-economic response to COVID-19. The Global Solidarity report 142 provided overall guidance by recognizing that the most vulnerable were the hardest hit by the pandemic, stating the UN’s commitment to help people and societies, especially the most vulnerable, and emphasizing the need for full respect of human rights, ensuring no one is left behind. Aiding in implementation of the Global Solidarity’s guidance, the Framework143 outlined detailed responses to the pandemic with specific deliverables and activities related to gender dimensions, human rights and Leave No One Behind (LNOB). Particularly, the relevant responses including the assessments were expected to be gender- responsive and involve a human rights-based approach to data collection with disaggregated data such as age, sex, migratory status, health status, socio-economic status, place of residence and other factors. The analysis of the human rights and gender impacts would then inform the design of policies that address these risks considering gender aspects and disability-inclusive responses. The Framework also provided a set of indicators to monitor and assess the human rights implications of the COVID-19 crisis, including socio- economic impacts and LNOB aspects. In addition, the Guidelines for the Preparation of Project Document for the Development Account144 required the applicants to outline how the project would contribute to gender equality and the enhancement of human rights, with particular emphasis on “leaving no one behind”, taking into consideration how the specific needs of groups such as youth, persons with disabilities, older people, refugees, migrants, the poor, and others would be addressed.

142 Shared Responsibility, Global Solidarity: Responding to the socio-economic impacts of COVID-19. March 2020. 143 A UN Framework for the immediate socio-economic response to COVID-19. April 2020. 144 https://www.un.org/development/desa/da/wp-content/uploads/sites/52/2022/07/T15-Project-Document- Guidelines-vf.docx

65

118. In line with the guiding UN documents, the Surge Project aimed at supporting the MSMEs that were considered the most vulnerable to COVID-19 within the private sector, highlighted the gender, HRBA and LNOB perspectives and fairly covered the gender aspects. Youth and other vulnerable groups were also covered, though to a lesser extent.145 Given the larger number of vulnerable groups working in the informal MSMEs- particularly women and youth - the project was developed to contribute to inclusive growth during the economic resurgence after the pandemic146. As highlighted in the project document, most informal MSMEs are led by the working poor, women, youth and other marginalized and vulnerable groups who need to be integrated into an inclusive economic resurgence after COVID-19 and who are at most risk of falling into poverty and being left further behind. The initial project document included a brief section on the assessment of vulnerable groups such as the women traders involved in cross-border trade, and initiatives benefiting the vulnerable groups, especially women and youth. The second phase proposed a coherent approach towards MSMEs, with the purpose of reaching out to the most affected target groups, including women and informal workers. A specific section on mainstreaming gender equality and human rights aspects including social protection was developed to outline the project approach, and intended to ensure a gender perspective, LNOB aspects and HRBA would be integrated into the activities of different clusters across preparedness, response, and recovery stages. The project document147 further provided a brief assessment of the gender issues and made reference to youth and vulnerable groups under each cluster and stated that the project interventions would address issues in an inclusive manner leaving no one behind. The 3rd phase project document also referred to the project approach of reaching out to the most affected target groups, including women and informal workers148. Specific activities integrating gender and youth were also covered including capacity- building workshops for national government officials and informal MSME entrepreneurs, especially youth and women, an entrepreneurship training workshop to assist entrepreneurs from vulnerable backgrounds and to deliver support to MSMEs in the region, placing its efforts on supporting the poorest, marginalized and women-led micro and SMEs. The result frameworks included specific outputs mainly benefiting women entrepreneurs such as training initiatives for cross-border traders (almost all female traders) and outcome- level indicators covering gender-disaggregated data as well as COVID-19 impact assessments. However, the project document covering the initial phase only mentions persons with disabilities in relation to the project’s links to the SDG targets 8.5 and does not specifically outline disability-inclusive responses. Furthermore, some of the outcome indicators include gender-disaggregated data but do not propose effective data collection methods to capture other vulnerable groups and vulnerabilities or gender dynamics beyond the limited quantitative data.

119. In addition to the key guidelines provided by the UN Secretariat, it was unclear to what extent internal procedures and programming guidance for each UN partner on gender mainstreaming, gender equality or LNOB played a role in the integration of all aspects in the design and implementation of the Surge project. Although the overall project was designed with a strong alignment to the UN guiding documents on HRBA and gender, the integration of these cross-cutting components in the implementation within each UN partner’s activities was less consistent. As an informant from one of the implementing UN entities stated, “gender mainstreaming and reducing inequality are part of their [the staff’s] DNA in all they do.” On the other hand, another claimed that due to many partners involved and the urgency of the situation, internal clearance processes regarding gender aspects for DA projects were not necessarily followed for the Surge project. Also, it was implied by yet another informant that with additional resources (i.e. human, financial, and time) then gender responsiveness, HRBA, and LNOB could have been better integrated into the implementation of the project components.

120. There was limited evidence available of systematic sharing of experiences, capacity, or lessons learned related to the integration of the cross-cutting themes, which might have enhanced leveraging synergies among the UN partner organizations or with other relevant agencies, such as UN Women. One example of a good practice was in the Republic of Moldova where UNECE worked closely with the UN Country team and UN Women contributed to the development of a gender-sensitive roadmap for the government based on a study on MSMEs conducted by the Surge project. One of the survey respondents further

145 UN. COVID-19 Response. July 2020. Global Initiative towards post-Covid-19 MSME sector: MSME Surge Project. Phase II Project Proposal. 146 UNCTAD. 2020. Concept note: Global Initiative towards post-Covid-19 MSME sector Phase I. 147 UN. COVID-19 Response. July 2020. Global Initiative towards post-Covid-19 MSME sector: MSME Surge Project. Phase II Project Proposal. 148 UN. COVID-19 Response. February 2021. Global Initiative towards post-Covid-19 MSME sector: MSME Surge Project. Phase III Project Proposal.

66

emphasized that the project was largely complementary to the work of a few agencies - such as UNDP, UNCTAD and UN Women - and helpful in the process of the CCA analysis and gender assessment. UN Women was also involved in supporting a regional conference on women in the Latin American region. Collaboration with Empretec enabled the development of a training programme for people with low literacy and the Farming-as-a-Business training, facilitating participation and empowerment of the vulnerable groups in the sector. Reflecting on the available documentation and evidence, the collaboration and sharing of experiences among project partners or other UN organizations regarding the integration of gender, HRBA and LNOB perspectives were limited to some cases as outlined above and not systematic across the project clusters.

121. The level of integration of these cross-cutting themes by the main counterparts depended on the counterpart’s understanding, willingness, and priorities. The UN partner organizations worked with the governments, including corresponding ministries and officials, as their main counterparts. One informant stated that as a DA initiative, their main counterparts were governments and they looked at how governments shifted their policies to support women in SMEs. Under cluster 5, one of the governments was not interested in women's studies, mentioning that they wanted everyone to benefit without discriminating against men or women-led enterprises. It was also challenging to receive inputs in places where there was no institutional structure dealing with gender issues or women’s empowerment such as in Cluster 5 or to ensure women’s participation, as it was reported that government counterparts were responsible for selecting participants for the Latin American and the Asia and the Pacific regions. The Bangladesh Start-up Ecosystem Assessment Report integrated gender aspects but the women participation at the follow-up panel was low. In Brazil, as reported by an informant, there was a presence of women in the webinars but no gender parity149. Even if there was a supportive government in Moldova where a roadmap for MSMEs covering gender aspects was developed, new priorities, the crisis in the region and limited funding were cited as reasons why there was very limited follow-up to the government roadmap or recommended actions in the project study supported by the Surge project.

122. In Gambia, however, the government considered youth and women as the main target beneficiaries, given that they made up the largest portion of the informal MSMEs sector in the country and looked at how to revitalize this sector and address their needs. As another example, in Malaysia, as a local counterpart, the National Association of Women Entrepreneurs of Malaysia (NAWEM) helped support women entrepreneurships and ensure women participation through engaging in the webinars. Furthermore, the Cross-border Trade Associations, together with the governments, were involved in selecting the participants, who were almost all women as the activities were targeting women as main stakeholders, for the training programs organized as part of the UNCTAD component on Women and Cross-border Trade, including in Malawi and Zambia. One informant suggested that a key lesson learned for the success of the overall cross- border trade and gender initiative was to collaborate closely with the government authorities and ensure their buy-in and support for the activities.

123. Several needs and impact assessments integrated gender and human rights impact of the pandemic among MSMEs to varying degrees. Under different clusters, several assessments of the impact of COVID-19 on MSMEs particularly focusing on gender aspects were conducted to inform design of project interventions including policy recommendations aiming at leaving no one behind. UN DESA conducted a study on the impact of COVID-19 among MSMEs in Kenya and their resilience mechanisms. The assessment included youth and women organizations, focused on the impact of the pandemic on women and youth-led entrepreneurs and provided specific recommendations for vulnerable groups like the elderly, widowed and orphans in Kenya. UN ESCAP Bangladesh Startup Ecosystem Assessment Report150 included a detailed gender analysis and highlighted the importance of integration of gender and inclusivity angles in the funding opportunities to support the ecosystem by bringing in women, people with disabilities, and people from underrepresented and marginalized groups. UN ECE rapid impact assessments on MSMEs were undertaken in the region with a view of informing gender-responsive trade policies. The assessment reports covered gender gaps, gender- disaggregated data, and strategic priorities to achieve women equality including gender-responsive trade policies. 151 UN ECA supported one region and 11 countries to assess the impacts of the recent crises on the MSMEs of its member states in Southern Africa. Almost all the reports cover gender and/or youth aspects

149 The participants list of the region does not have gender disaggregated data to confirm the informant´s statement. 150 Bangladesh startup ecosystem assessment report | ESCAP (unescap.org) 151 UNECE. The impact of COVID-19 on the trade and business development prospects of female-owned enterprises in the Republic in Armenia. 2021; and The impact of COVID-19 on the trade and business development prospects of female-owned enterprises in the Republic in Armenia. Available at: https://unece.org/trade/studies-regulatory-and- procedural-barriers-trade

67

while the study conducted in Eswatini only indicated that the gender details in terms of ownership were not provided by the state. Furthermore, ECLAC reviewed policies to mitigate the impact of the COVID-19 pandemic on SMEs in nine Latin American countries152 and identified government measures with special emphasis on employment opportunities for women, people with disabilities and youth. The global report of the UNCTAD on COVID-19 Pandemic Impact on Micro, Small and Medium Sized Enterprises: Market Access Challenges and Competition Policy identified gender inequalities, provided disaggregated data for gender, age, minorities, persons with disabilities, and recommendations for entrepreneurs and MSMEs from vulnerable backgrounds particularly affected by the digital divide. The national studies on analysis of the impact of COVID-19 on MSMEs in South Africa and Thailand covered gender issues and gender aspects and proposed recommendations for women, youth and vulnerable groups. The national study on competition and market access policies in the resurgence of MSMEs in post-pandemic in Brazil, however, has no reference to gender, human rights or LNOB aspects.

124. The majority of assessments were gender responsive and primarily focused on gender aspects and on the impact of the pandemic on women-led MSMEs. While several assessments included youth aspects, only a few studies and data analyses covered people with disabilities. When asked, two informants expressed an awareness that other vulnerable and marginalized groups were more rarely covered, though they did not explain why. Despite this limitation, the studies were well received and helped identify the needs of those vulnerable groups. When the survey respondents were asked if the project activities and the content reflected the perspectives of women, youth, people with disabilities or other vulnerable groups, the majority (68%) responded positively. The work dedicated to profiling women-led MSMEs was found insightful to develop legal amendments supportive of women employability, according to one survey respondent. Another respondent mentioned that the research on women’s entrepreneurship was considered helpful for policymakers and ministerial staff in Member States to improve their understanding of how support given to women in business pays off.

125. The degree of integration of gender, human rights and LNOB aspects and implementation of assessment recommendations varied across clusters. In a number of cases, documentation was lacking about how those assessments or policy documents were followed up and to what extent the issues pertaining to vulnerable groups were addressed. Even if the assessments were conducted to identify needs and actions were proposed to address those needs of the vulnerable, there were cases where the evidence showed implementation of the actions was lacking or limited. The Project readiness report on the development of a digital platform on technology and innovation by SMEs in Southern Africa required that users and partners consider women, youth and environmental issues. However, the output report of the digital platform did not cover engagement with women, persons with disabilities or other vulnerable groups or aspects. While the Bangladesh Startup Ecosystem Assessment Report153 included a detailed gender analysis, one of the informants stated that “there was a section on gender, that´s all.” There was no systematic follow-up on how the project-funded studies were utilized. Another informant stated that gender mainstreaming is often on paper and not in reality.

126. On the other hand, in Moldova, the recommendations based on the gender-responsive assessment were translated into a roadmap, in collaboration with the government and UNCT. The government stakeholders considered the study as a big input. However, as one informant described, only some of the recommendations were able to be followed up, such as cross-border transit physical capacity and e- commerce funded by USAID, due to the crisis in the region and limited funding opportunities, among other challenges. In Kenya, the rapid impact assessment findings on women and youth-led entrepreneurs informed the planning of another initiative integrating gender and youth aspects with funding from a project. The project team worked closely with the government and linked their funding streams to increase the overall impact. As informed, none of the proposed activities were stand-alone, but rather aligned with the results chain – cross- cutting vertical and horizontal. Entrepreneurship policy review informed the new MSMEs strategy in Uganda. For the first time, with the support from the Surge project and building on the existing partnership with UNCTAD, Uganda included migrants and refugees in the high-level strategic plan, in addition to women and youth as vulnerable groups, according to an informant. The strategy serves as a framework to mobilize resources to realize different pieces. Another informant described that recently the government launched a strategy to help refugee communities.

152 Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, Mexico, Panama, and Uruguay 153 Bangladesh startup ecosystem assessment report | ESCAP (unescap.org)

68

127. Building on the “Informal Cross-border Trade for the Empowerment of Women, Economic Development, and Regional Integration in Southern and Eastern Africa” project of UNCTAD’s Trade, Gender and Development Programme, the capacity-building activities were designed based on the existing gender responsive programme which was then tailored to address emerging needs of women traders during the pandemic in Botswana, Kenya, Malawi, the United Republic of Tanzania, and Zambia. Given the urgency and time limitation, an informant said this component was adapted and implemented very quickly by including COVID-19-related considerations. As highlighted by another informant, it was relatively easy because of the existing relationship with the stakeholders. The training content was revised to include new trade regulations and address needs of different vulnerable groups to enhance knowledge of their rights and obligations, regional trade rules, and on the formalization and registration of business. In partnership with Empretec, training contents were tailored both for participants with low literacy and high literacy levels, enabling engagement with the most vulnerable communities. Gender considerations specific to women traders including harassment were mainstreamed into the activities. Beyond the training activities, the initiative brought together the border officials and authorities dealing with immigration and customs to raise the issue of harassment against women traders. In addition to raising the gender issues, as highlighted above (para 96), the project could further enhance its support to government capacities in cross-border trade by involving government officials, including border officials, in the training courses, as trainees, or having a dedicated training session for them. The majority of the participants who responded to the workshop surveys found the workshops useful and relevant for their businesses. The policymakers and other stakeholders who attended the regional policy dialogue on women and cross-border trade considered the workshops successful at providing a forum to discuss the challenges faced by women informal and small-scale cross-border traders, and formulate practical solutions to improve their situation and enhance the contribution of cross-border trade to economic development and regional cooperation (Box 2).

Box 2: UNCTAD Cross-border Trade and Gender Initiative.

128. Another project initiative primarily focusing on gender aspects was the ESCWA´s Women Empowerment for Technology and Entrepreneurship (AWETE)154 programme which was launched through the DEPAR platform to address the national challenges hindering women empowerment in technology and

154 https://www.unescwa.org/news/boosting-women%E2%80%99s-empowerment-technology-and-entrepreneurship https://depar.unescwa.org/regional/awete

This initiative is a good example of integrating gender responsiveness in the Surge project interventions in extremely vulnerable communities facing additional challenges due to the impacts of the pandemic. Building on the pre-existing programming, the Trade and Gender initiative was adapted quickly to respond to urgent and emerging needs of the cross-border traders, primarily women.

The cross-border trade and gender component of the Surge project replicated the training activities for informal and small-scale cross-border traders of the previous UNCTAD initiative “Informal Cross-border Trade for the Empowerment of Women, Economic Development, and Regional Integration in Southern and Eastern Africa¨ which was implemented from 2016 to 2019. The training workshops were based on the tailored Empretec programme by adapting an existing methodology to meet emerging needs of women traders during the pandemic. The expected results of this initiative included increased awareness of trade rules and procedures, improved knowledge of their rights and obligations, enhanced use of official border posts and trade through formal routes, ability to adapt the businesses and developed strategies, and awareness of COVID-19 measures. Under this initiative:

✓ Almost 300 cross-border traders participated in the workshops, almost all were women.

✓ 9 in-person workshops (6 days each) with two components organized: training on cross-border trade rules and development of entrepreneurial skills.

✓ Botswana, Kenya, Malawi, the United Republic of Tanzania, and Zambia covered.

✓ Project partners included relevant ministries, trade authorities and Cross-border Trade Associations of the respective countries.

✓ Over 90% of the participants found the workshops useful and relevant.

✓ 95% of the participants of the regional policy dialogue considered the workshops successful.

✓ Five traders´ guides ¨Women in informal cross-border trade: A small-scale trader's guide to trade rules and procedures¨ tailored for Botswana, Kenya, Malawi, Zambia and Tanzania.

69

entrepreneurship across the Arab region. AWETE provides a community space for female entrepreneurs, innovators, and professionals to share knowledge and build new partnerships. Under this initiative, the project developed women empowerment ecosystem maps for 22 Arab counties155 and organized a series of five roundtable discussions in the region, bringing key experts and stakeholders advocating for women´s rights.

129. The majority of the survey respondents had the perception that the project activities were accessible for women, youth, people with disabilities, or other vulnerable groups, although the document review identified limited data in project reporting and other relevant sources referring both to cases with varying degree of gender parity and others with no data on gender or other aspects. The ability to assess comprehensively the participation of vulnerable groups is limited to available documentation that mostly covers gender and no other vulnerabilities. In some cases, the data indicated that the events successfully facilitated the engagement of women, ensuring gender parity. The online Empretec TOTs organized in French, English and Spanish for low-literate people reached out to around 100 people, 49% women. Whereas the workshops/courses/webinars on circular economy reached out to 600 participants, only one-third of them women. In other cases, the information on gender or other aspects is not available to make a comprehensive assessment covering all project interventions. However, the majority of survey respondents (70%) believed that the project activities were accessible for women, youth, people with disabilities, or vulnerable groups (Figure 20). The respondents also mentioned that the project included or supported women, youth, people with disabilities or other vulnerable groups through empowering vulnerable groups especially women and supporting their access to funds.

Figure 20: Perceived accessibility of project activities for women, youth, people with disabilities or other vulnerable groups (n=126 respondents).

Source: Independent evaluation survey, 2023.

130. Adapting to COVID-19 restrictions enabled the engagement of the vulnerable groups to some degree, particularly women entrepreneurs in the project activities while, in some cases, their participation was hindered due to the lack of access to technology, selection process of beneficiaries, location of events, or the pandemic impacts and restrictions that made it difficult to bring people together. Going virtual equalized those who were able to participate in project activities and built bridges across countries and regions. However, participation was limited to only those with physical and financial access to the necessary technology, and ensuring participation was challenging as there were too many online initiatives from a range of development partners. Given the pandemic restrictions, most of the interventions particularly in the early stages of the pandemic including workshops and training programs were organized online, and online portals enabled free access to everyone with access. As reported by one of the informants, NGOs could follow some of Empretec´s work online, particularly vulnerable groups and women on the ground. Empretec´s TOT workshops were

155 https://depar.unescwa.org/regional/awete/ecosystemmaps

33%

37%

21%

3% 2% 0%

4%

Project activities were accessible for women, youth, people with disabilities, or other vulnerable groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

70

revised, and short videos were developed to better serve the vulnerable groups including rural entrepreneurs, farmers and low-literacy entrepreneurs156. One informant believed that having everything online helped people from diverse populations to link across groups and locations in the country. The final report of the project also highlighted that the skills development was facilitated thanks to the online workshops increasing delivery efficiency and accessibility with specific attention to vulnerable groups and people with low literacy levels157.  In addition, the online regional policy dialogues on the role of competition policy in supporting MSME’s economic recovery in the post-COVID-19 crisis were new, in the sense that they were online and open to all stakeholders from governments to MSMEs covering several countries at once. Although the online platforms provided unique opportunities and, in some cases, the only way to reach out to the vulnerable communities, they hindered the engagement of those marginalized groups with no financial and technological access.

131. Additionally, cases were also found of implementation modalities limiting participation of some vulnerable groups. For example, selections of cross-border trade workshop participants, almost all women traders, was the responsibility of the governments in collaboration with the Cross-border Trade Associations. In the Latin American region, it was reported that there were challenges to ensuring the participation of women or other vulnerable groups since it was the governments that conducted the selection of participants. In addition to the selection processes, it was reported that criteria in a few training programmes limited the participation of the most vulnerable. . When translation was provided in another aspect of the Surge project, it was indicated that the translation was difficult to understand, which limited the effectiveness of reaching out to the most vulnerable. In most cases, there was no reporting on participants’ self-identifying with a range of vulnerable groups, such as people with disabilities, making it difficult to assess the extent of engagement in the Surge project by various vulnerable groups. It was indicated by an informant that they had no way to measure how many and with which vulnerable group the participants identified. The locations of some of the events, such as holding them only in the capital city, were also considered as inhibiting participation from diverse groups and regions.

132. A few impact assessments point towards positive outcomes particularly for women. Overall, 64% of the survey respondents felt that the project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups (Figure 21). The survey respondents revealed that the relevant areas on which the project has contributed to the resurgence of MSMEs, policy or economic improvements, or advancing sustainable development included improved inclusion of vulnerable groups with a special reference to women entrepreneurs, funding opportunities for women, addressing the needs and supporting women-led MSMEs. Capitalizing on the knowledge or skills acquired through the project, some survey respondents reported that they facilitated priority policies for women-owned and women-responsive businesses, development of female entrepreneurships, and access to credit for women entrepreneurs and women capacity-building.

Figure 21: Perceived contribution of the project to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups (n=105 respondents).

156 Under the Surge project, Empretec offered in-person, hybrid as well as online training activities to respond to the different needs and COVID-19 restrictions of various contexts. 157 UNCTAD. September 2022. Final Report: Global Initiative towards post-Covid-19 resurgence of the MSME sector.

71

Source: Independent evaluation survey, 2023.

133. The UNCTAD component on Women and Cross-border Trade delivered solid results in terms of enhancing cross-border traders’ capacity, mostly women. The impact assessment on the cross-border trade in the pre- and post-pandemic environment covering 2019-2021158 indicated that the training workshops that were attended by almost all women helped women entrepreneurs become more effective cross-border traders. The impact assessment further outlines how the initiative helped enhance the knowledge and abilities of the cross-border traders to benefit from greater business resilience (Please see the effectiveness section of this report for assessment findings). The initiative also contributed to the formalization of the businesses of the cross-border traders. As reported by an informant, female traders were informal but now most of the training participants have registered their businesses.

134. The impact assessment on the Empretec workshops (ETWs) which assessed the economic impact of the ETWs on the participant entrepreneurs in selected project countries indicated positive results for women entrepreneurs, only for those areas where gender disaggregated data was provided159. Based on the findings, the number of women in business after the ETW increased by 15% in Benin, 7% in Ghana and 9% in Nigeria. 160 The F/M TEA ratio (Female and male, Total Early Stage) also demonstrated positive outcomes in Benin (15% higher161), Cameroon (13% higher), Gambia (57% higher) and Zimbabwe (20% higher).162

The project was also effective in facilitating women and youth-led MSME registration and formalization. UNCTAD helped facilitate e-registration and formalization of MSMEs in El Salvador, Benin, Cameroon and Mali through an e-registration portal. In Benin, the portal increased business registration by 91%, with 84% increase in women-led MSMEs and 181% increase in youth-led MSMEs. A similar trend was observed in Mali where business registration increased by 40% including 49% increase in women-led MSMEs, and 110% in youth-led MSMEs.

135. In contrast to survey respondents’ perceptions, the available data on inclusion of vulnerable groups and vulnerabilities is limited and hinders the evaluation’s ability to assess the degree to which gender, HR and LNOB aspects were integrated into the Surge project. Of the data that is available, it is mostly for the gender-

158 UNCTAD (2022). Cross-border Trade in the Pre- and Post-Pandemic Environment Evidence from Malawi, the United Republic of Tanzania and Zambia. Geneva. 159 The assessment provided limited gender disaggregated data and did not cover any other cross-cutting dimensions. 160 UNCTAD. MSME Surge Project. Empretec: Impact Assessment Survey Report. 2023. 161 In Benin, for instance, the ratio 0,8/1 referring to 8 MSME businesses opened by women to every 10 MSME businesses started by men – at Empretec the ratio is 0,92:1 (15% larger). 162 The F/M TEA (Total Early Stage) (Female / Male TEA ratio, applying only to early-stage businesses) is an indicator of how Entrepreneurship may be used as a tool for equity and women empowerment, especially in traditional male dominated societies as usually verifiable in Africa.

32%

32%

28%

2%

3% 0%

3%

The Surge project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

72

disaggregated information for the project beneficiaries with no or limited information on age, persons with disabilities or other characteristics. As reported by one informant, the initiative focused on female youth but there was no record on how many female youth entrepreneurs were reached. In some cases, the gender- disaggregated data was only available for those who responded to the event´s survey and not for all the participants, making it difficult to assess the gender parity163. The data collection tools were mostly not designed to collect information necessary to assess the cross-cutting dimensions. For example, one of the workshop evaluation forms used under cluster 1 did not have any questions to identify issues related to gender, age, disabilities, and other vulnerabilities of specific groups. This is also observed across clusters covering various project initiatives.

136. Furthermore, the project results framework mainly captured quantitative gender-disaggregated data, but did not capture the degree of the project´s contributions to different vulnerable groups and vulnerabilities. There was also limited follow-up to assess the impact of the training or other initiatives from gender and human rights perspectives. As one informant observed, follow-up monitoring should have been included to assess the impact of the training programs and nobody was doing it.

137. One of the exceptions was the Impact Assessment on the Empretec Workshops (ETWs) covering 6 African countries between 2020 and 2022, namely Benin, Cameroon, Ghana, Nigeria, The Gambia and Zimbabwe. The assessment on ETWs did not capture all the cross-cutting aspects comprehensively but provided gender-disaggregated data for the number of women in business after ETWs. Another assessment was conducted for the capacity-building activities organized by the UNCTAD Trade and Women initiative in Malawi, the United Republic of Tanzania and Zambia aimed at small-scale and informal cross-border traders, mainly women. The impact assessment covered the capacity-building activities carried out from 2019-2021 at selected border posts, including previous analytical work carried out by UNCTAD´s Trade, Gender and Development Programme since 2016 under a 10th tranche DA project164. The compilation of success stories of women entrepreneurs who received Empretec training - such as the story of how four women in Malaysia built success during COVID-19165- also helped provide anecdotal evidence on the project’s contributions to women empowerment and inclusive growth.

163 The number of participants of the UNCTAD Webinar Series on SDG Reporting by companies, including MSME—The adaption of “Guidance on core indicators for entity reporting on contribution towards implementation of the Sustainable Development Goals (GCI)” in China was 150 while the gender-disaggregated data is only available for 26 survey respondents. 164 The Development Account project “Informal Cross-border Trade for the Empowerment of Women, Economic Development, and Regional Integration in Southern and Eastern Africa”, implemented during 2016-2019. 165 https://unctad.org/news/how-four-women-malaysia-built-success-during-covid-19

73

6. Conclusions

138. Based on an analysis of the findings, the evaluation formulates the following conclusions.

Conclusions

Relevance

The project responded to the needs of Governments and MSMEs. Interventions contributed to and were informed by needs and impact assessments. Recipients of project interventions were often involved in or contributed to the design and delivery of project activities and outputs. On various occasions, project outputs were tailored to the needs of target users and beneficiaries, being countries or vulnerable groups. The relevance of the interventions was somewhat mitigated by the mandates and capacities of the implementing UN entities. The latter are equipped for supporting medium- and long- term policy changes rather than immediate crisis responses. Other challenges included the restrictive Covid-19 measures and the short time frame for implementation of the project. Nonetheless, on the whole, the implementing UN entities exercised adaptive management to ensure high relevance of their interventions. The phased approach, in particular, provided the opportunity to introduce new activities not foreseen at an earlier stage.

Coherence

The project was anchored in the mandates and comparative advantages of the implementing UN entities. Selected interventions followed previous work and specific requests from member States and were often coordinated with or complementary to existing activities. At conception, the target of the project was to address specific constraints faced by MSMEs. It was designed around the Entrepreneurship Policy Framework to show the complementarities across interventions that each implementing UN entity would deliver in response to regional priorities and capacities. Some collaborations between implementing UN entities were pursued and complementarities established, but on a limited level. A few synergies in the form of joint outputs were integrated in the 3rd phase of the project. By design, technical task forces created through the project were expected to bridge the implementing UN entities. However, this proved difficult to launch and sustain, owing to several constraints, including a highly ambitious objective given the global crisis and competing priorities within each agency. Nevertheless, the coordination and complementarities established by the project among the implementing entities contributed to expanded dissemination and outreach of outputs, avoided overlaps between them, and allowed significant knowledge exchanges aligned with and supportive of a “One UN” system. On the other hand, collaboration of the implementing UN entities with other UN organisations, including UNCTs, was limited. Despite this shortcoming, at national level, target recipients of project outputs found the interventions complementary to those of other UN agencies.

Efficiency

Project coordination was complex and demanding, involving seven UN entities and the target delivery of around 100 outputs, of which some were composed of many activities. Nevertheless, the coordination of the project was found to be efficient overall given the circumstances and resources available. The coordination of the design phase of the response was actively supported by the DA-PMT. Coordination of project implementation was led by UNCTAD in collaboration with a Steering Committee that met on a regular basis, most often bi-monthly. UNCTAD’s Budget and Project Finance Section (BPFS) produced the project’s financial monitoring table every month, which included the expenditure data of UNCTAD as well as the other implementing entities. A monitoring dashboard was created by UNCTAD to facilitate the tracking of UNCTAD’s project delivery. Data from some of the other implementing UN entities was added, when provided, at two reporting points during the lifetime of the project.

The project would have benefited from the allocation of more resources towards the coordination and monitoring of overall implementation. Across the implementing UN entities, the project was considered complex and hard to follow outside of the interventions under their direct control. Some staff perceived that there were too many participants in the coordination meetings, that the project lacked a Theory of Change, and that it was primarily a compilation of activities rather than a cohesive and mutually reinforcing bundle of interventions supporting in depth any given country. Some staff in the UN Regional Commissions questioned the efficiency cost of a global or inter-regional response versus regional responses. The project was implemented in three phases, which were supportive of adaptive management and perceived by staff as the most rational approach considering the circumstances. However, it was also pointed out that clearer visibility

74

from the onset on the funding available for future phases would have facilitated planning and the search for synergies.

Effectiveness

The Development Account programme, as a funding modality, was quick to respond and adapt to the COVID- 19 crisis. The prompt availability of resources and rapid orientation toward a global response contributed to the rapid launch of a wide range of interventions. However, questions remain as to its adequacy for addressing the immediate needs of a crisis in comparison with other UN agencies and mechanisms with a mandate for humanitarian response. The governance of this project, with DA-PMT’s oversight, the management structure with UNCTAD in the lead coordinating role, and each implementing entity leading the operational implementation at the regional level, ensured the global and regional scope of project delivery. At the operational level, implementing entities relied more on partnerships with local and regional institutions rather than coordination within the larger project management structure. The project governance and management mechanisms evolved in tandem with the COVID-19 crisis and the project’s emerging needs. The result was a high delivery rate but with limited inter-agency collaboration in the process. Yet, despite limited collaboration, this independent evaluation identified that the project was effective in producing identifiable results at the outcome level.

Overall, the project delivered 85% of the planned outputs. Phase 3, particularly, delivered the highest number of outputs but had the lowest outcome effectiveness rate, mostly because of budget cuts and some outputs still in progress at the time of this evaluation. There is evidence of the effective improvement of national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship and MSME promotion (outcome 1.A) [such as the adoption of a revised entrepreneurship strategy by South Africa, Uganda and Seychelles]; improved resilience and competitiveness of MSMEs (outcome 1.B) [for example with 64.5% of participants opening a new business after attending Empretec-based training]; facilitated MSME registration and formalization through e-platforms (outcome 2) in El Salvador, Benin, Cameroon and Mali; improved access to finance, including upskilling in financial literacy (outcome 3) [for example with 97% of the MSMEs in Latin America that attended trainings reporting improved financial literacy in accounting and reporting and also improved capacity to manage financial resources]; increased MSME access to innovation and technology (outcome 4) [including through capacity development on green technologies for SMEs in Southern Africa]; and increased access to local, regional and international markets through digitalization and non-tariff measures (outcome 5) [for instance with case studies and on competition in Thailand, South Africa and Brazil, followed by webinars and the creation of an online course on SMEs and competition policy, primarily for government officials].

In relation to the project indicators, the assessment of their effectiveness was based on limited information, proxies, and expert judgement since they often lacked SMART criteria. It was possible to infer that 85% of them showed some level of progress, either through objective measures (e.g., sales increase) or proxies (e.g., post-training satisfaction assessments and survey responses). The high effectiveness results can be partly attributed to the conceptual framework provided by the EPF, as well as the cluster-based approach with each cluster addressing specific regional constraints and responding to demand from member States. In addition to clustering about 100 outputs from seven implementing entities into 5+1 outcomes, the EPF provided a coherent thread to relate outputs that cut across outcomes. Further analysis would be needed, however, to clearly identify which of those outputs are the most suitable or adaptable for a crisis response.

The project improved the capacity of policymakers in designing and implementing policies supportive of MSMEs, especially in terms of contributing to a country’s government responses, country-specific studies, technical assistance, training courses, and the development of digital tools and regional interventions. This improved capacity is reflected in beneficiaries’ reports of how they have been considering the needs of MSMEs, including those led by women and youth, in their daily work. These results could become more transformative and sustainable with more institutional support to networks of policymakers and communities of practice for mutual and regional learning on supportive policies for MSMEs, as well as by mainstreaming gender and Leaving No One Behind (LNOB) approaches in government officials' capacity-building.

Sustainability

The project developed approaches, tools and capacities that are being transferred at multiple levels. At the policy level, the addition of more functionalities to e-government tools, expanding training courses to more countries, and the establishment of agreements to implement the project recommendations are all strong

75

evidence of the sustainability of the project's achievements. At the behavioural level, beneficiaries are applying new knowledge and skills to their daily work, be it by including MSMEs' issues in policy making or by improving MSMEs' management, performance, outputs and, by extension, resilience. Challenges to this transfer of knowledge relate to the lack of financial resources, need for follow-up mentoring, and lack of a favourable business environment.

Gender, human rights, and disability

The overall project design was well aligned with the two key UN documents that provide clear direction for the integration of a human rights-based approach, which include addressing the aspect of gender equality and Leaving No One Behind. Although these aspects were well-articulated in the project documents, strong follow-through and documentation of their implementation was limited to a few project components. For those components that did include these aspects, gender was more commonly integrated and youth to a lesser degree. There was very little evidence of the outreach, participation, or impact of project activities for people with disabilities or other vulnerable groups. The UNCTAD component on Women and Cross-Border Trade targeting five countries in Africa was a good practice in how to design and implement this type of intervention with a strong reflection of the needs of some of the most vulnerable women across border areas.

76

7. Recommendations 132. Based on the evaluation findings and conclusions, the following recommendations could be considered by the implementing UN entities and the DA-PMT.

139. Recommendation 1: (i) UNCTAD should further leverage the experience gained through the project to map out how components of the Entrepreneurship Policy Framework relate to and can be best positioned to support MSMEs in their recovery from different types of crises and (ii) DESA and the Regional Commissions should add their analysis of how their work can contribute towards the objective. UNCTAD, DESA and the Regional Commissions could build on the project to map their areas of intervention in support of the MSME sector, as well as capacities and knowledge on the implementation of the EPF components, including in a crisis context, and opportunities for broadening the uptake of EPF components at the regional level based on national needs. UNCTAD, DESA and the Regional Commissions could further consider identifying areas of joint interventions that would trigger complementarities and synergies between the agencies. This could involve collaborating on the development of capacities of all national and sub-national actors, i.e. Governments, MSMEs, other partners (e.g. Chamber of Commerce, Business Incubators, etc.) in line with United Nations Sustainable Development Cooperation Frameworks (SDCF) in respective countries; strengthening or contributing to facilitate access to finance for the MSMEs (including seed money, grants, access to credit, etc.); improving the coordination of MSMEs related policies across ministries; increasing interventions at the local level, such as by supporting NGOs or MSMEs outside of the main cities. The application of a human rights-based approach, gender responsiveness and inclusion of other vulnerable groups (LNOB) should be ensured.

140. Recommendation 2: UNCTAD should continue building on the momentum generated by the project to continue fostering knowledge exchanges and promoting the Entrepreneurship Policy Framework. UNCTAD, DESA and the Regional Commissions should identify means to more meaningfully continue to share good practices and lessons learned on the demand-driven support they provide to the MSME sector, including on areas such as green/circular economy; innovation policies and ecosystems; MSME formalization and strengthening MSME capacities for the empowerment and leadership of women, youth and groups in vulnerable situations. UNCTAD and the Regional Commissions could also propose organizing regional events and/or a global conference to promote entrepreneurship policy and further advance the goals and impact of the Surge project. Furthermore, learnings from the experiences with the Surge about external partnerships should encourage UNCTAD to promote the EPF and entrepreneurship development to other UN agencies also engaged in this area (e.g. ILO, FAO, UN Women, etc.), to development banks, or to related initiatives such as the recent Global Accelerator on Jobs and Social Protection. UNCTAD could also consider joining and supporting events that promote the achievement of SDG 8.3 as an avenue to further promote the EPF.

141. Recommendation 3: The DA-PMT should develop a clear framework for assessing the costs and benefits of implementing a global or inter-regional project versus regional projects. As a global crisis can affect regions and countries differently, global or joint projects should be developed only when there are clear benefits of joint implementation. Thus, a global or inter-regional vs. regional response would not be a priori decision, but a result of a clear assessment. A specific guideline or framework should be developed for this. Some of the assessment criteria could include the need or demand from member States for an integrated response; the range of common versus entity-specific activities and added value of complementary interventions (i.e. in terms of geographical coverage, reach of the target population, capacity, and/or coverage of multiple reinforcing technical areas, or networks and partnerships); capability to execute joint interventions (e.g. in terms of time, resources, logistics, and flexibility to pivot interventions without bureaucratic procedures); coordination costs (which increase with the number of participating UN entities); project inception modalities and governance and knowledge management requirements to facilitate collaboration and knowledge sharing; scaling and sustainability plans; expected results of spreading resources versus concentrating on fewer countries; and so forth. The time taken to conduct such assessments will increase project coherence and effectiveness and facilitate the identification of the financial and human resources needed for project implementation. The network of DA Focal Points is a key existing asset for this assessment.

77

142. Recommendation 4: Implementing UN entities should ensure that they have a comprehensive Results Framework for the entire project as well as an adequate monitoring plan, with indicators that are designed to support the ongoing monitoring. The results framework should have one objective and ideally have one outcome per cluster/workstream. The objective should state the intended goal of the project, describe the overall achievement targeted by the project, involving a process of change aimed at meeting the needs of identified beneficiaries, and reflect the overall funding available to the project. Each objective should include reference to the project’s beneficiaries and its substantive focus. The objective should not attempt to explain the ways in which the project intends to achieve the objective (i.e. it should not include the word ‘through’ or describe the internal work of the UN using verbs such as ‘support’, ‘facilitate’ or ‘contribute’). The outcomes (OCs) should describe the changes that are expected to occur as a result of the completion of outputs. The OCs should be achievable within the project’s timeframe and budget, and should be specific enough to be measured by the associated indicators of achievement. The indicators of achievement (IAs) should provide measures for monitoring progress towards achieving the OCs and reporting on them after completion of the project. Every indicator needs to provide clearly defined baselines, units of measurement and targets, detailing the quantity, quality and timing of expected results. The monitoring, evaluation, accountability, and learning (MEAL) system should also be designed to capture HRBA, gender and LNOB aspects. In phased interventions or during project execution, any changes in the logframe if/when pivoting activities should be clearly explained to the wider team.

143. Recommendation 5: Implementing UN entities should ensure that sufficient resources are allocated to project coordination, technical collaboration, and partnership building. The absorption capacity of implementing entities can be challenged by crisis response projects which add to the planned programme of work. This is further compounded by projects that come with an extensive UN partnership and a global scope. Sufficient resources should be dedicated to global coordination and to building global partnerships with strategic stakeholders (e.g., UN organisations engaged in supporting the MSME sector; development banks). Capacities should also be directed to supporting technical collaboration and the staff implementing interventions, including towards synergy or liaison with the UNCTs. When designing the project, implementing UN entities should consider featuring coordination and partnership-building in the Theory of Change or logframe of the project. Tools to support continuous connections and knowledge exchange, and to ensure institutional memory should be part of the response package, such as a project website, SharePoint space for all team members, and a Yammer network or Teams channel. The integration of cross-cutting aspects (HRBA, gender responsiveness, LNOB) also requires expertise with sufficient and dedicated time and resources. Guidance could be development to project managers on how to do this.

78

Annexes

Annex 1: Project’s Theory of Change .....................................................................................................................................

Annex 2: Evaluation matrix.......................................................................................................................................................

Annex 3: Data collection instruments ...................................................................................................................................

Annex 4: List of documents reviewed ...................................................................................................................................

Annex 5: List of individuals interviewed ...............................................................................................................................

Annex 6: Evaluation surveys ....................................................................................................................................................

Annex 7: Evaluation Terms of Reference ..............................................................................................................................

79

Annex 1: Project’s Theory of Change

80

Annex 2: Evaluation matrix

Relevance

Key questions Suggested measures or evidence Suggested sources and methods

● To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID-19?

● Evidence of needs assessments (surveys, reports, minutes from consultations, workshop reports, etc.).

● Country contexts. ● Reference to the Surge project in country,

programmes, or COVID-focused UN evaluations. ● Reference and opinion of UN staff and partners. ● Opinion of staff from the implementing UN entities,

including Project Design Team, Project Steering Committee, and Task Forces.

● Opinion of external partners, including other UN organizations, Governments, other partners and stakeholders.

• Desk review: surveys, minutes from consultations, workshop reports, country assessments, sectoral analysis, SERPs.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

● To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g. COVID-19 Socio-Economic Response Plan)?

● Reference to SERPs in project progress reports and outputs.

● Reference to the Surge project in UN PRODOCS and in country, programmes, or UN/UNCT annual reports.

● Reference to the Surge project in country, programmes, or COVID-focused UN evaluations.

● Opinion of UN staff and external partners.

• Desk review: Progress reports, project outputs, SERPs.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

• Coherence

Key questions Suggested measures or evidence Suggested sources and methods

● To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

● Evidence of collaboration or joint outputs with other UN departments or programmes; evidence of other UN departments or programmes referring to or using MSMEs project’s outputs.

• Desk review: Strategies and workplans of participating UN

81

● Evidence of synergies between implementing UN entities and overlaps avoided; evidence of joint activities.

● Evidence of the project being mainstreamed in the workplans of other departments or programmes within implementing UN entities.

● Evidence of the project being reflected in the annual workplans and time commitments of staff from implementing UN entities.

● Opinion of staff from the implementing UN entities, including Project Design Team, Project Steering Committee, and Task Forces.

● Opinion of external partners, including other UN organizations, Governments, other partners and stakeholders.

● Reported instances of improvements in the coordination of the response to the COVID crisis by implementing UN entities.

● Evidence of adaptive management and adjustments in the course of project implementation.

entities, PRODOCs, progress reports, evaluations.

• Interviews: Staff from participating UN entities, UNCTs.

● To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID- 19 in delivering socio-economic support to Member States?

● Number and types of other UN agencies contributing to project implementation.

● Evidence of partnerships with other UN agencies; evidence of collaboration or joint outputs with other UN departments or programmes; evidence of other UN departments or programmes using outputs from the MSMEs project.

● Evidence of the project being mainstreamed in the workplans or programmes of other UN agencies.

● Opinion of staff from other UN agencies, including UNRCOs and UNCTs.

● Opinion of external partners, including Governments and other stakeholders.

● Reported instances of improvements in the coordination of the response to the COVID crisis by other UN agencies.

● Evidence of adaptive management and adjustments in the course of project implementation.

• Desk review: Strategies and workplans of participating UN entities, PRODOCs, progress reports, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

82

• Efficiency

Key questions Suggested measures or evidence Suggested sources and methods

● How well coordinated was the process for the response among the entities implementing the joint project?

● Evidence of project monitoring meetings; technical meetings of the project task Forces.

● Evidence of joint activities, synergies between participating UN entities, overlaps avoided; project mainstreaming in participating UN entities’ workplans.

● Opinion of UN staff, including Project Design Team, Project Steering Committee, Task Forces, UNRCOs and UNCTs.

● Opinion of external partners, including Governments and other partners and stakeholders.

● Reported instances of improvements in the coordination of the response to the COVID crisis by participating UN entities.

● Time taken to develop and deliver planned outputs; evidence of delays.

● Evidence of adaptive management and adjustments in the course of project implementation.

• Desk review: Meeting minutes, progress reports, workplans, PRODOCS.

• Interviews: Staff from participating UN entities (e.g. Project Design Team, Project Steering Committee, Task Forces), UNCTs, external partners.

• Case study: UNCT staff and partners.

● How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

● Reported impact according to project monitoring meetings; technical meetings of the project task Forces.

● Opinion of UN staff, including Project Design Team, Project Steering Committee, UNRCOs and UNCTs.

● Opinion of external partners, including Governments and other partners and stakeholders.

● Speed of implementation; time taken to develop and deliver planned outputs; evidence of delays.

● Evidence of adaptive management and adjustments in the course of project implementation.

• Desk review: Meeting minutes, progress reports, workplans, PRODOCS.

• Interviews: Staff from participating UN entities (e.g. Project Design Team, Project Steering Committee, Task Forces), UNCTs, external partners.

• Case study: UNCT staff and partners.

• Effectiveness

83

Key questions Suggested measures or evidence Suggested sources and methods

● To what extent did the programme (Development Account) and project governance and management structures and processes enable, or hinder, the effective implementation of the joint project and the achievement of its results?

● Evidence of project monitoring meetings; technical meetings of the project task Forces.

● Programmes and projects monitoring and evaluation reports.

● Opinion of SHS staff, partners, and stakeholders; ● Evaluators’ expert judgement drawing on all evidence

sources.

• Desk review: Meeting minutes, progress reports, workplans, PRODOCS.

• Interviews: Staff from participating UN entities (e.g. Project Design Team, Project Steering Committee, Task Forces), UNCTs, external partners.

● To what extent has the project contributed to the expected outcomes as enunciated in the project document?

● Evidence of project outputs and reference to project’s outputs in policies, SDG related reports, and other national publications on MSMEs and entrepreneurship (e.g., UN, private sector, CSOs/NGOs, etc.).

● Outcome indicators: All; level of achievement of planned outcomes.

● Programmes and projects monitoring and evaluation reports.

● Opinion of UN staff, partners, and stakeholders. ● Evaluators’ expert judgement drawing on all evidence

sources.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations, Empretec centers data and reports, data collected by UNCTAD via its e- regulation and e-registration portal.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

● How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

● Evidence of uptake, use, and influence of project outputs.

● New approaches and policies adopted and capacity and resources to sustain these.

● Reported country level uptake and institutionalisation of new approaches introduced as a result of ILO COVID response.

● Opinion of UN staff, Governments, and other partners and stakeholders.

● Evaluators’ expert judgement drawing on all evidence sources.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

84

● What innovative approaches or tools, if any, did the response use, and what were the outcomes and lessons learned from their application?

● Use made and outputs of innovations such as online or blended modalities at country level, gamification and other interactive techniques.

● Engagement in short term measures outside of normal activities.

● Outcome indicators: IA 4.1., IA 4.2. ● Opinion of UN staff, Governments, and other partners

and stakeholders. ● Evaluators’ expert judgement drawing on all evidence

sources.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

• Sustainability

Key questions Suggested measures or evidence Suggested sources and methods

● What measures were adopted to ensure that outcomes of the response would continue after the project ended?

● Reported contribution made through the project towards impact in each of the outcome areas for intended beneficiaries and in shaping national policies

● Governments and other beneficiaries’ perceptions of impact/likely impact

● Governments and other beneficiaries’ perception of strengthened capacity.

● New approaches and policies adopted and capacity and resources to sustain these.

● Reported country level uptake and institutionalisation of new approaches introduced as a result of ILO COVID response.

● Evidence of new arrangements, partnerships and funding being put in place to support the scaling up of work implemented or given new prominence as a result of the pandemic.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Gender, human rights and disability

Key questions Suggested measures or evidence Suggested sources and methods

85

● To what extent were perspectives on gender equality, protection of human rights and reaching underserved groups integrated into design and implementation of the project? What results can be identified from these actions?

● Proportion of vulnerabilities disaggregated surveys, assessments.

● Evidence of human rights conventions and strategies (e.g. CEDAW, CRC, CESCR, CRPD, etc.) referred in progress monitoring reports.

● Evidence of project outputs mainstreaming HRBA principles and LNOB.

● Extent to which partnership agreements and project activities include specific measures advancing gender equality, inclusion, human rights.

● Evidence of vulnerable groups or their representative organisations involved in project implementation and monitoring.

● Outcome indicators: IA 1.5, IA 1.6, IA 1.7, IA 1.8, IA 2.1, OP 3.1, IA 3.2, IA 5.3; level of achievement of planned outcomes.

● Opinion of UN staff, Governments, and other partners and stakeholders.

● Evaluators’ expert judgement drawing on all evidence sources.

• Desk review: Project outputs, progress reports, assessment questionnaires, evaluations.

• Interviews: UN staff and partners.

• Survey: UNCTs, external partners and stakeholders.

• Case study: UNCT staff and partners.

86

Annex 3: Data collection instruments

Relevance

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

1. To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID- 19?

● To what extent has the project been articulated with countries’ priorities to respond to the COVID- 19 crisis?

● How did you identify the needs for the project activities or products you delivered in target countries?

● Did you specifically assess the needs of women and vulnerable groups? If positive, how?

● Is there any evidence of such needs (e.g. demands from Member States, surveys, minutes from consultations, workshop reports, market assessments, past evaluations)?

● To what extent has the project been articulated with countries’ priorities to respond to the COVID- 19 crisis?

● How were the needs of national beneficiaries identified?

● Did you specifically assess the needs of women and vulnerable groups? If positive, how?

● Who was consulted and what evidence is there (e.g. surveys, minutes from consultations, workshop reports)?

● Did the Surge Project respond to a national or sectoral agenda or priority?

● Were your needs assessed and did they inform project activities? If positive, how?

● To what extent did the training activities, knowledge products, or technical support delivered by the Surge project respond to your learning needs?

● To what extent were the training activities, knowledge products, or technical support delivered by the Surge project relevant to your work?

● To what extent did the training activities, knowledge products, or technical support delivered by the Surge project provide you the opportunity to share your knowledge and skills with other participants?

● Were you overall satisfied with the training activities, knowledge products, or technical support

87

delivered by the Surge project?

2. To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g. COVID-19 Socio- Economic Response Plan)?

● How did the Surge project’s products and activities align with the SERPs?

● To what extent was the project aligned with UN priorities to LNOB by reaching the most vulnerable and the protection of human rights?

● How did the Surge project’s products and activities delivered in your country respond to the SERP?

● To what extent was the project aligned with UN priorities to LNOB by reaching the most vulnerable and the protection of human rights?

● To what extent was the project aligned with UN priorities to LNOB by reaching the most vulnerable and the protection of human rights?

Would be primarily assessed through a desk review of a sample of SERPs.

Coherence

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

3. To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

● What were the other key activities that your department or other departments in the organization carried out to support MSMEs during the period 2020-2022? Which departments implemented those activities? How did you synergize these activities with the Surge project?

● Is there any evidence of joint activities or outputs? Is there any evidence of

● Did you cooperate with different departments from the project UN entities? If positive, did these UN entities synergize their inter- departmental collaboration while working with you?

● What were the good practices? What could have been done differently?

● Did you receive support (from different departments of the implementing agencies or) from several implementing entities?

● If positive, is there any evidence of complementary activities designed to maximize synergies and avoid overlaps? What were the good practices?

● What could have been done differently?

88

complementary activities designed to maximize synergies and avoid overlaps?

4. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non- Secretariat) to COVID- 19 in delivering socio- economic support to Member States?

● Were there any other UN organisations outside of the core project partners that carried out activities to support MSMEs during the Covid crisis (2020- 2022)? Which ones and how did you synergize these activities with the Surge project?

● To what extent did the project support other efforts by UN entities to support Member States intending to mitigate socio-economic impacts of COVID-19?

● Is there any evidence of joint activities or outputs? Is there any evidence of complementary activities designed to maximize synergies and avoid overlaps? What were the good practices?

● What could have been done differently? Are there any UN partners with which complementarities could have been strengthened (e.g. ILO, UNDP)?

● To what extent did the project support other efforts by UN entities to support Member States intending to mitigate socio-economic impacts of COVID-19?

● Among the range of activities that your organization carried out in the country to support MSMEs during the period 2020-2022, did you synergize any of these interventions with the Surge project?

● Is there any evidence of joint activities or outputs? Is there any evidence of complementary activities designed to maximize synergies and avoid overlaps? What were the good practices?

● What could have been done differently?

● Did you receive support from other UN organisations? If positive, is there any evidence of complementary activities designed to maximize synergies and avoid overlaps with the project’s implementing entities? What were the good practices?

● What could have been done differently?

● In your opinion, how do you assess the level of synergies and complementarity of the project with the interventions of other UN organisations in the country? (Likert: 6 levels)

● In your opinion, were there any UN organisations with which the Surge project could have increased synergies or avoided overlaps at country level (if positive, please specify which UN organisations): (Open ended).

89

Efficiency

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

5. How well coordinated was the process for the response among the entities implementing the joint project?

● How did you coordinate project activities with the other entities? Were there any opportunities for any joint work contributing to reducing implementation costs or scale benefits?

● To what extent did the project benefit from resources (structure, personnel, partnerships) made available by ongoing projects?

● What were the good practices in terms of coordinating the Surge project?

● What were the challenges? What could have been done differently?

● Did the UN entities coordinate adequately project implementation?

● What were the good practices? What could have been done differently?

● Did you receive support from several implementing entities?

● If positive, is there any evidence of complementary activities designed to maximize synergies and avoid overlaps? What were the good practices?

● What could have been done differently?

6. How did the three- phase budgeting and programming approaches impact the efficient delivery of the project?

● Did the phased approach come with a sufficient level of predictability to ensure synergies with the work of your agency? Did the phased approach prevent the project from formulating a long-term vision? Did the phased approach facilitate the formulation of

● Were the activities of the UN entities sufficiently predictable to ensure proper synergies with the work of your agency at country level?

● Did the phased approach facilitate any adaptive management? What were the good practices?

● Were project activities sufficiently predictable to ensure proper synergies with your organisation’s agenda of work? Could you properly plan the collaboration and joint work with the UN entities?

● Did you have a clear vision about what the UN

90

interventions across a pathway from crisis response, to recovery, and development?

● Did the phased approach facilitate any adaptive management? What were the good practices?

● What could have been done differently? What lessons learned from each of the three phases supported the continuation and continuous improvement of the project?

● What could have been done differently?

entities intended to deliver and the expected effects of their interventions?

Effectiveness

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

7. To what extent did the programme (Development Account) and project governance and management structures and processes enable, or hinder, the effective implementation of the joint project and the achievement of its results?

● Was the programme modality (Development Account) adequate to respond to a crisis? What were the strengths and weaknesses of the programme modality (Development Account) in a crisis context?

● What were the challenges faced by the project’s governance and management structures and processes?

● How do you assess the project governance and management structures and processes?

● What worked well? Are there any good practices that could be systematized?

● What could have been improved/what would you propose doing differently next time?

● How do you assess the project governance and management structures and processes?

● What worked well? Are there any good practices that could be systematized?

● What could have been improved/what would you propose doing differently next time?

91

● What were the good practices and lessons learned from project implementation?

8. To what extent has the project contributed to the expected outcomes as enunciated in the project document?

● What would you highlight as the most significant outcomes of the project in the following areas (as relevant)? (i) Improving national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship MSME promotion in post-COVID- 19 resurgence; (ii) Improving resilience and competitiveness of MSMEs in post-COVID-19 resurgence; (iii) Facilitating MSMEs registration and formalization; (iv) Improving MSMEs access to finance; (v) Increasing MSMEs access to innovation and technology; (vi) Enhancing MSMEs access to markets.

● What examples or evidence would demonstrate such outcomes?

● Who were the primary beneficiaries of the project?

● What would you highlight as the most significant outcomes of the project in the following areas (as relevant)? (i) Improving national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship MSME promotion in post-COVID- 19 resurgence; (ii) Improving resilience and competitiveness of MSMEs in post-COVID-19 resurgence; (iii) Facilitating MSMEs registration and formalization; (iv) Improving MSMEs access to finance; (v) Increasing MSMEs access to innovation and technology; (vi) Enhancing MSMEs access to markets.

● What examples or evidence would demonstrate such outcomes?

● Who were the primary beneficiaries of the project?

● What would you highlight as the most significant outcomes of the project in the following areas (as relevant)? (i) Improving national capacities on formulating and implementing enabling policies on green, resilient and inclusive entrepreneurship MSME promotion in post-COVID- 19 resurgence; (ii) Improving resilience and competitiveness of MSMEs in post-COVID-19 resurgence; (iii) Facilitating MSMEs registration and formalization; (iv) Improving MSMEs access to finance; (v) Increasing MSMEs access to innovation and technology; (vi) Enhancing MSMEs access to markets.

● What examples or evidence would demonstrate such outcomes?

● Who were the primary beneficiaries of the project?

● In what ways did the training activities, knowledge products, or technical support delivered by the Surge project influence your perspectives?

● To what extent did the training activities, knowledge products, or technical support delivered by the Surge project enhance your skills and/or knowledge of the subject matter?

● To what extent did the training activities, knowledge products, or technical support delivered by the Surge project enlarge your professional network?

● Were you able to use/apply in your work the knowledge or skills acquired through the training activities, knowledge products, or technical support delivered by the Surge project?

● Were there measurable changes in your activities and performance when you got back to your workplace that you can attribute to the training

92

activities, knowledge products, or technical support delivered by the Surge project?

● To what extent have the changes in your performance and new level of knowledge or skills sustained over time?

9. How did the response contribute to the participating country Governments’ responses to COVID- 19, especially in the area of MSME resurgence?

● What examples would you highlight showing that the project contributed to any of the following achievements (as relevant): (i) MSMEs sustained and growing their businesses, including through exporting and integration in value chains; (ii) Businesses newly established, newly registered/formalized MSMEs; (iii) Jobs sustained and created; including with regard to women and youth entrepreneurs; (iv) Policies and measures developed and implemented by governments to facilitate the MSME resurgence in the immediate term and to enhance their shock resilience and competitiveness in a longer term.

● What were the unintended positive and

● What examples would you highlight showing that the project contributed to any of the following achievements (as relevant): (i) MSMEs sustained and growing their businesses, including through exporting and integration in value chains; (ii) Businesses newly established, newly registered/formalized MSMEs; (iii) Jobs sustained and created; including with regard to women and youth entrepreneurs; (iv) Policies and measures developed and implemented by governments to facilitate the MSME resurgence in the immediate term and to enhance their shock resilience and competitiveness in a longer term.

● What were the unintended positive and

● What examples would you highlight showing that the project contributed to any of the following achievements (as relevant): (i) MSMEs sustained and growing their businesses, including through exporting and integration in value chains; (ii) Businesses newly established, newly registered/formalized MSMEs; (iii) Jobs sustained and created; including with regard to women and youth entrepreneurs; (iv) Policies and measures developed and implemented by governments to facilitate the MSME resurgence in the immediate term and to enhance their shock resilience and competitiveness in a longer term.

● What were the unintended positive and

● To what extent have the knowledge and skills you acquired though the training activities, knowledge products, or technical support of the Surge project contributed to improving the performance or results of your organization?

● To what extent did the knowledge and skills you acquired though the training activities, knowledge products, or technical support of the Surge project contribute to your ability to advance sustainable development.

● In what ways have the training activities, knowledge products, or technical support delivered by the Surge project contributed to: (i) Improving national capacities on formulating and implementing enabling

93

negative effects of the project?

negative effects of the project?

negative effects of the project?

policies on green, resilient and inclusive entrepreneurship MSME promotion in post-COVID- 19 resurgence? (ii) Improving resilience and competitiveness of MSMEs in post- COVID 19 resurgence? (iii) Facilitating MSMEs registration and formalization? (iv) Improving MSMEs’ access to finance? (v) Increasing MSMEs’ access to innovation and technology? (vi) Enhancing MSMEs’ access to markets? (vii) Engaging and sustaining women and youth entrepreneurs and other vulnerable and marginalized groups?

● Please, share concrete

examples of the types of policy or economic improvements, contribution to advancing sustainable development, or other entrepreneurship impacts that resulted from the skills and knowledge you acquired though the training activities, knowledge products, or technical support

94

delivered by the Surge project

10. What innovative approaches or tools, if any, did the response use, and what were the outcomes and lessons learned from their application?

● In which areas was the Surge project innovative?

● What were the most promising practices that participating UN entities should consider institutionalizing or replicating?

● In which areas was the Surge project innovative?

● What were the most promising practices that participating UN entities should consider institutionalizing or replicating?

● Was there anything innovative or unique with the Surge project that participating UN entities should consider institutionalizing or replicating?

● Was there anything innovative or unique with the Surge project that participating UN entities should consider replicating or scaling? (open ended)

Sustainability

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

11. What measures were adopted to ensure that outcomes of the response would continue after the project ended?

● Has the project advanced partnerships amongst project participants, national institutions and the MSMEs in support of sustainable results?

● How did the project contribute to leveraging funding and financing of government and other resource partners (multilateral, bilateral, etc.)?

● What were the enabling factors that contributed to making the project transformative?

● Has the project advanced partnerships amongst project participants, national institutions and the MSMEs in support of sustainable results?

● How did the project contribute to leveraging funding and financing of government and other resource partners (multilateral, bilateral, etc.)?

● What were the enabling factors that contributed to making the project transformative?

● Has the project advanced partnerships amongst project participants, national institutions and the MSMEs in support of sustainable results?

● How did the project contribute to leverage funding and financing of government and other resource partners (multilateral, bilateral, etc.)?

● Do you find that UN entities managed adequately the need for immediate response and the search for longer- term effects?

● To what extent did the Surge project contribute to promoting better preparedness for future crisis? (Likert scale)

95

● What were the enabling factors that contributed to make the project transformative?

● To what extent did the project contribute to promote better preparedness for future crisis?

Gender, human rights, and disability

Key questions Interview questions for UN staff from partner

entities

Interview questions for implementing partners (UNCTs, Empretec, etc.)

Interview questions for target beneficiaries

(Governments, MSMEs, etc.)

Survey questions for target beneficiaries

(Governments, MSMEs, etc.)

Comments

12. To what extent were perspectives on gender equality, protection of human rights and reaching underserved groups integrated into the design and implementation of the project? What results can be identified from these actions?

● To what extent have women and other vulnerable and marginalized groups engaged, including outreach and selection process, types of leadership roles, decision-making, among others?

● To what extent have project activities been designed and implemented with particular attention to the needs and contexts of women and other vulnerable and marginalized groups?

● To what extent have Member States engaged and integrated human

● To what extent have women and other vulnerable and marginalized groups engaged, including outreach and selection process, types of leadership roles, decision-making, among others?

● To what extent have project activities been designed and implemented with particular attention to the needs and contexts of women and other vulnerable and marginalized groups?

● To what extent have national partners engaged and integrated

● To what extent have women and other vulnerable and marginalized groups engaged, including outreach and selection process, types of leadership roles, decision-making, among others?

● In what ways have the activities affected women and other vulnerable and marginalized groups in their lives?

● In what ways were women and other vulnerable and marginalized groups affected by COVID-19 restrictions and how was

● To what extent have women and other vulnerable and marginalized groups benefited from the training activities, knowledge products, or technical support delivered by the Surge project? (Likert: 6 levels)

● Any comments? (open ended)

● Disaggregated analysis of survey data

96

rights protections in their project activities?

human rights protections in their project activities?

that addressed in the project’s efforts?

● To what extent have national partners engaged and integrated human rights protections in their project activities?

97

Annex 4: List of documents reviewed

● Callo-Müller M. V. 2020. Micro, small and medium enterprises (MSMEs) and the digital economy. ESCAP. Bangkok.

● ECLAC. 2020. Sectors and businesses facing COVID-19: Emergency and reactivation, Special Report n.4. Santiago, Chile.

● ESCAP. 2021. Rethinking MSME Finance in Asia and the Pacific: A Post-Crisis Policy Agenda. Bangkok. ● ESCAP. 2021. The Role of Competition Policy in Strengthening the Business Environment for MSMEs in the ASEAN

Region. 11 February 2021. Bangkok. ● ESCAP. 2022. Bangladesh Startup Ecosystem Assessment Report. Bangkok. ● ESCAP. 2022. MSME Access to Finance: The Role of Digital Payments, MSME Financing Series No.7. Bangkok. ● ESCAP. 2022. Policy guidebook for MSME development in Asia and the Pacific, 2nd edition. Bangkok. ● ESCAP. 2022. Small and Medium-Sized Enterprises Lending Approaches: The Role of Banks in Asia. MSME

Financing Series No. 6. Bangkok. ● ILO. 2020. COVID-19 and the impact on agriculture and food security. Geneva. ● ILO. 2018. Women and Men in the Informal Economy: A Statistical Picture, Third Edition, Geneva. ● ILO. 2020. Contagion or starvation, the dilemma facing informal workers during the COVID-19 pandemic.

Geneva. ● ILO. 2021. ILO Monitor: COVID-19 and the world of work. Seventh edition. 25 January 2021. Geneva. ● ILO. 2021. ILO Monitor: COVID-19 and the world of work. Seventh edition. 25 January 2021. Geneva. ● ILO. 2022. Independent High-Level Evaluation of ILO’s COVID-19 response 2020-22. Geneva. ● IMF. 2020. World Economic Outlook Update: A Crisis Like No Other, An Uncertain Recovery. Washington. ● IMF. 2021. World Economic Outlook Update. January 2021. Washington. ● OECD. 2020. Global Outlook on Financing for Sustainable Development 2021. Paris. ● UN OCHA. 2020. Global Humanitarian Response Plan for COVID-19. Geneva. ● UN Women. 2020. Guidance Note for Action: Supporting SMEs to Ensure the Economic COVID-19 Recovery is

Gender-Responsive and Inclusive. UN Women Regional Office for Asia and the Pacific. Bangkok. ● UNCTAD. 2020. Concept note: Global Initiative towards post-Covid-19 MSME sector Phase I. April 2020.

Geneva. ● UNCTAD. 2020. COVID-19 Response. Global Initiative towards post-Covid-19 MSME sector: MSME Surge

Project. Phase II Project Proposal. July 2020. Geneva. ● UNCTAD. 2020. COVID-19: Firmer Action Needed to Better Protect Consumers. Geneva. ● UNCTAD. 2021. COVID-19 Response. Global Initiative towards post-Covid-19 MSME sector: MSME Surge

Project. Phase III Project Proposal. February 2021. Geneva. ● UNCTAD. 2021. Global Initiative towards post-Covid-19 resurgence of the MSME sector. MSME surge project -

Addendum Phase 3 Project Proposal. Geneva. ● UNCTAD. 2021. How COVID-19 affects MSME access to markets and competition: A review of key issues and

recommendations for future action. Geneva. ● UNCTAD. 2021. Impact of the COVID-19 pandemic on trade and development: Recovering, but unevenly -

Situation as at 31 March 2021. Geneva. ● UNCTAD. 2022. Cross-border Trade in the Pre- and Post-Pandemic Environment Evidence from Malawi, the

United Republic of Tanzania and Zambia. Geneva. ● UNCTAD. 2022. Entrepreneurship & innovation in the new health economy. Geneva. ● UNCTAD. 2022. Final Report: Global Initiative towards post-Covid-19 resurgence of the MSME sector.

September 2022. Geneva. ● UNCTAD. 2022. Guidance for action: Supporting MSMEs’ recovery from the COVID-19 pandemic. Prepared for

the trade competition commission of Thailand. Geneva. ● UNCTAD. 2022. The COVID-19 pandemic impact on micro, small and medium sized enterprises: Market access

challenges and competition policy. Geneva. ● UNCTAD. 2023. MSME Surge Project. Empretec: Impact Assessment Survey Report. Geneva. ● UNDESA. 2018. Micro-, Small and Medium-sized Enterprises (MSMEs) and their role in achieving the Sustainable

Development Goals. New York. ● UNDP. 2020. UNDP Briefing Note Gender COVID-19. New York. ● UNECA. 2022. The role of digitalisation in strengthening capacities of Micro, Small and Medium-Size Enterprises

(MSMEs) in Southern Africa to take advantage of the AfCFTA. ECA Sub-Regional Office for Southern Africa (SRO- SA) & ECA Digital Centre of Excellence. Addis Ababa.

● UNECE. 2020. Guidelines and best practices for micro-, small and medium enterprises in delivering energy- efficient products and in providing renewable energy equipment. Geneva.

98

● UNECE. 2020. Guidelines and Best Practices for MSMEs to assure resiliency and progress towards a circular economy in sustainable resource management and critical raw material supply chain solutions. August 2020. Geneva.

● UNECE. 2020. The Impact of COVID-19 on trade and structural transformation in Georgia. Geneva. ● UNEG. 2008. Code of Conduct for Evaluation in the UN system. New York. ● UNEG. 2020. Ethical Guidelines for Evaluation. New York. ● UNIDO. 2020. COVID-19, Implications and Response : Digital Transformation and Industrial Recovery. Vienna. ● United Nations Moldova. 2020. COVID-19 Socio-economic Response and Recovery Plan. June 2020. Chisinau. ● United Nations. 2017. Resolution adopted by the General Assembly on 21 December 2016. 71/221.

Entrepreneurship for sustainable development. A/RES/71/221. General Assembly. New York. ● United Nations. 2019. Resolution adopted by the General Assembly on 20 December 2018. 73/225.

Entrepreneurship for sustainable development. A/RES/73/225. General Assembly. New York. ● United Nations. 2020. A UN framework for the immediate socio-economic response to COVID-19. April 2020.

New York. ● United Nations. 2020. Resolution adopted by the General Assembly on 2 April 2020. A/RES/74/270. General

Assembly. New York. ● United Nations. 2020. Shared Responsibility, Global solidarity: Responding to the socio-economic impacts of

COVID-19. March 2020. New York.

99

Annex 5: List of individuals interviewed

Redacted for confidentiality purposes.

100

Annex 6: Evaluation surveys

The evaluation carried out six external surveys to inform the assessment of the relevance, coherence, and effectiveness of the project and its contribution to outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups. The surveys targeted participants to project activities and beneficiaries. The lists of contacts were provided by UNCTAD, DESA, ESCAP, ECLAC, UNECA, and UNECE. The surveys were composed, for some part, of questions that were common to all six questionnaires, and for other parts, of questions specific to each of the 5+1 project outcomes. The questionnaires were made available in English. They were translated in Spanish for outcomes 3 and 5 as target recipients were primarily in the Latin America and Caribbean region. The surveys were anonymous and remained open for 2 weeks, from Thursday 20 July to Friday 4 August. Two reminder messages were sent to increase the response rate. The surveys were launched to a combined list of 1454 persons, with 89 messages bouncing back. Altogether, the surveys compiled feedback from 133 respondents. All survey questions were optional. Questionnaires partially completed were kept in the batch of results when they contained information that was judged relevant, credible and meaningful. Statistics were calculated on the basis of the number of valid responses per question and not on the basis of the overall number of respondents to the surveys. The overall response rate to the email surveys is circa 9.7%. The surveys present the opinion of those who responded but not of the entire list of recipients of the questionnaires nor of all the beneficiaries of the project.

Outcome Email addresses Bounced back Valid email Valid responses Percentage

1A 574 38 536 52 9.70% 1B 258 24 234 40 17.09% 2 21 0 21 4 19.05% 3 219 4 215 11 5.12% 4 98 1 97 13 13.40% 5 284 22 262 13 4.96%

Total 1454 89 1365 133 9.74% Relevance of the project 1. To what extent do you agree with the following statements?

Strongly agree

Agree Somewhat agree

Somewhat disagree

Disagree Strongly disagree

Do not know

Total

The project activities responded to my priorities and were relevant to my work

54 58 15 1 2 1 2 133

The project complemented interventions by other international organizations working in my country/region

36 56 23 4 3 1 7 130

101

2. Did the project increase synergies or avoid overlaps with other agencies/entities working in this area?

Please explain.

● A lot of organizations were conducting projects regarding COVID, however this one was the only one providing actual information on practical ways to mitigate some challenges arising from the pandemic.

● As every and each project in Armenia there are overlaps

● Auemntaron las sinergias, al publicarse reformasa las leyes laborales y fiscles.

● Aumento mis criterios previos al proyecto

● Aumentou a sinergia com outras entidades à medida que deixou bem clara a importância das parcerias.

41%

44%

11%

1% 1%

1% 1%

The project activities responded to my priorities and were relevant to my work

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

28%

43%

18%

3% 2% 1%

5%

The project complemented interventions by other international organizations working in my country/region

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

102

● Have no detail information

● Here, it is more of overlaps with other agencies.

● I am not aware that there was good mutual communication between the job holders during the execution of the project.

● I believe the project will increase synergies and complement other initiatives in place

● I can say that any project makes an effort to enhance synergies and do away duplications if there is to optimize the resources and maximize the positive impact.

● I did not see avoiding of overlaps with other agencies working in this area especially at country level.

● I think the project definitely increased synergies with other domestic agencies / entities working in each specific area.

● I think this project increase synergies with other project

● IDEP; PNUD HUB REGIONAL DAKAR

● Increased synergies

● It created synergy as Seda provided a number of virtual trainings which is still continuing to date.

● It focused on areas that affected our common clients. However, there is need to make a follow up and implement the recommendations.

● It increase synergy because from my view point I think the manpower strength of this organisation will stand out if the y are innovative ,

● It increased synergies

● It increased synergies and avoided overlaps

● It increased synergies in the sense that it adequately complemented what had been on the ground by the activities of other agencies.

● It increases synergies

● It is still in progress.

● It really increase synergy fill in gaps in the others

● It was helpful for avoiding overlaps

● Like womens ministery

● more strategic alliances

● Neither yes or no because I have seen it or positive impact on me

● No

● No awareness

● No comment

● No it did not

● No.

● Nos proporcionó elementos para poder considerar el comportamiento en éste tipo de eventualidades temporales

● Not sure. Communication between agencies is not always the best.

● Not yet because the understanding among the companies and leaders is not concrete to work hand in hand however strengthen the relationship and strong collaboration among the companies is still under consideration

● Our country SME office

● Partnerships and cooperation within the different parties involved was evident. This includes Donors, Investors, Grant Managers, Incubators, Accelerators, Chambers of Commerce, Hubs, the UN and the Governments of the various countries involved.

● Permitió por sobre todo tomar conocimiento de las realidades regionales de las pymes, y sus dificultades u oportunidades para la mirada exportadora y las herramientas y brechas existentes

103

como barreras a superar y que en la mayoría de los casos es posible extender los casos de éxitos o superaciones de un país a la región

● Project increase synergies, gives hope especially with the financial support. It was really helpful. Covid -19 severely damaged the whole life emotionally, physiologically, financially. It was really good that this project was there during the Covid-19 difficult times.

● project increased synergies with and complemented internal activities in own company

● Project never visible in the communities

● Se dieron conexiones y nuevas sinergias que resultaron muy positivas

● Se evitó solapamientos, ya que se cuidó esta parte en la capacitación.

● si porque nos ayuda a tomar decisiones para formular proyectos para las Mypes en nuestro pais

● Strong synergies were evident, probably due to strong collaboration between the Agencies.

● The inter-institutional cooperation is important.

● The project activities can not overlap even if having similar set of objectives since it is a needed necessity for developing countries.

● The project avoids overlaps with other agencies.

● The project did create synergies and was generally applicable to many situations

● The project encouraged synergies with other agencies including the ILO in supporting and promoting formalization as well as the UNDP in supporting private sector development and SMEs resilience building.

● the project has increase synergies with other agencies

● The project has increased synergies rather than overlapping other activities in this area.

● The project increase the synergies of the efforts with other agencies and entities by focusing areas of intervention and rationalizing resources

● The project increased synergies

● The project increased synergies with between Government Agencies and the Private Sector whereby innovative ideas were explored and cutting edge solutions were developed.

● The project increased working relationships with other agencies. The project also covered some of the areas which Government might have resources to reach.

● The project looked at all the problems of the MSMEs and contributed to the summary of information in one place.

● The project was avoid overlaps with other agencies.

● The project was not only important source of information for the Moldova's Government, but also it was largely complementary to the work of few agencies, such as UNDP, UNCTAD, UN Women etc and was helpful in the process of CCA analysis, Gender assessment etc. Particularly, the project recommendations were presented during the Socio-Economic task force chaired by UNDP , the Economic Council under Prime-Minister and development partners coordination platform co-chaired by the UN and World Bank Group in Moldova. some of the findings were further adopted as activities under the UNCTAD and UNDP projects (notably regarding customs procedures and SME support).

● the project was used to make recommendations as part of a study on the MSME business environment in Madagascar

● The project was very important session

● This programme increased the synergies amongst the relevant ministries/institutions that works on MSMEs

● To build capacities to MSM"s

● To considerable extend did avoid overlap, what is important is to bring MSME center stage especially in developing economies and that unique role has been played by this attempt

● we had synergies and networking

● yes

104

● yes

● yes great synergies avoiding overlapping between UN RECs and DESA in NYC created new partnerships within and outside UN system

● yes it did

● Yes it did. Our collaboration with Regional Economic Commissions of the UN was very helpful, UNESWA and UNECLAC were already incorporating some aspects of competition work and SMEs and this project enabled us to collaborate more and share our expediencies like never before. We also closely collaborated with UNESCAP in their SMEs and also to organize the DA project closing event in Bangkok.

● yes it increased synergies with other agencies working in that area

● Yes it increases synergies

● Yes it was complimentary

● Yes right

● Yes the project enhanced synergies with other agencies through collaborations and partnerships to provide relevant support to MSMEs

● Yes the project increases the synergies of Empretec Centre Benin and chamber of commerce and Industry in Benin

● Yes,

● Yes, it did and help a lot

● Yes, it did increase synergies. Especially with NGOS who are impacting on skills acquisition, they also made impact but UNTACD surpassed them all

● Yes, it did. Information on the issues of access to markets and disruption of supply chains supported competition agencies in enforcing the law to support MSME sustainability.

● Yes, it increased synergies since several agencies were involved to implement the project.

● Yes, it was structured in a close coordination with other implementing agencies and key national counterparts

● Yes, MSME is the most hit by COVID-19 and need special focus

● Yes, proper planning of involved actors boosted synergies and avoided overlaps.

● Yes, since it was implemented in close collaboration with the UN Regional Economic Commissions and with the member States counterparts fully involved.

● Yes, the project increased synergies and the multi partner approach worked well.

● Yes, the project increased synergies. It created a platform where women who make up the majority MSMEs in developing countries are able to access technology at an affordable cost to enhance their businesses.

● Yes, the timing of the project was excellent, while businesses and other agencies were cautiously distanced due to Covid-19 lockdown, the project was empowering us as development practitioners to prepare for a come back to assist us in supporting our clients recover from Covid-19.

● Yes, it's created prioritisation, definitions and specificities toward gaps and overlaps.

● Yes. Maximize the use of resources in the area.

● Yes. Because the entrepreneurship is an issue that relates to all the agencies

● Yes. It led to knowing how one could use the current computer advancement in pushing business including customers participation

● Yes. The project complemented other studies conducted at national level for specific target group.

● Yes. The project was specific to the needs of women entrepreneurs

● Yes. In some ways.

● Yes… increased synergies and helped to strengthen actions.

105

Usefulness of the project 3. To what extent do you agree with the following statements?

Outcome 1A Strongly agree

Agree Somewhat agree

Somewhat disagree

Disagree Strongly disagree

Do not know

Total

The project provided access to information and best practices on government support measures for post COVID-19 recovery

35% 39% 18% 2% 4% 2% 0% 51

The project improved my knowledge on entrepreneurship/MSMEs policies design and implementation in the context of post COVID-19 recovery

39% 39% 12% 4% 6% 0% 0% 51

The project contributed to identifying new/improved policy measures for MSME promotion in post COVID-19 resurgence

43% 33% 14% 6% 4% 0% 0% 51

Outcome 1B Strongly agree

Agree Somewhat agree

Somewhat disagree

Disagree Strongly disagree

Do not know

Total

The project provided information useful for starting a new business or expanding a business venture

36% 41% 8% 3% 8% 0% 5% 39

The project improved my knowledge on entrepreneurship and business development

32% 45% 11% 3% 5% 0% 5% 38

The project contributed to identifying new/improved measures to increase business sales

42% 26% 18% 3% 5% 0% 5% 38

Outcome 2 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project provided useful information on MSMEs formalization/registration

36% 41% 8% 3% 8% 0% 5% 3

The project improved my knowledge of how to increase the number of businesses registering

32% 45% 11% 3% 5% 0% 5% 3

The project contributed to identifying new/improved measures to decrease the administrative costs of starting a business

42% 26% 18% 3% 5% 0% 5% 4

106

Outcome 3 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project provided useful information on financial inclusion and MSMEs access to finance

30% 50% 20% 0% 0% 0% 0% 10

The project improved my knowledge on accounting and reporting, including on the SDG reporting, and improved my capacity to manage financial resources

50% 20% 20% 10% 0% 0% 0% 10

The project contributed to identifying new/improved measures for financial inclusion and MSMEs access to finance

40% 40% 10% 10% 0% 0% 0% 10

Outcome 4 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project provided useful information on MSMEs access to innovation and technology

75% 25% 0% 0% 0% 0% 0% 12

The project improved my knowledge on how to increase MSMEs access to innovation and technology

75% 25% 0% 0% 0% 0% 0% 12

The project contributed to identifying new/improved measures to increase MSMEs access to technology

50% 33% 17% 0% 0% 0% 0% 12

Outcome 5 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project activities and publications provided useful information on MSMEs access to markets

38% 54% 8% 0% 0% 0% 0% 13

The project improved my knowledge on how to increase access to markets, including export and integration into value chains

54% 31% 15% 0% 0% 0% 0% 13

The project activities contributed to identifying new/improved measures for consumer protection and competition

46% 46% 8% 0% 0% 0% 0% 13

The project activities contributed to identifying new/improved measures on

33% 33% 25% 0% 0% 0% 8% 12

107

agricultural quality and food loss reduction

3-B. To what extent do you agree with the following statements?

Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project activities were accessible for women, youth, people with disabilities, or other vulnerable groups

33% 37% 21% 3% 2% 0% 4% 126

The project activities and content reflected the perspectives of women, youth, people with disabilities, or other vulnerable groups

33% 35% 26% 2% 1% 0% 2% 126

33%

37%

21%

3% 2% 0%

4%

The project activities were accessible for women, youth, people with disabilities, or other vulnerable groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

108

4. What (if anything) would you describe as innovative or unique from the project that the UN should consider

replicating or upscaling? Why?

● A simplified micro lending business method, such as funding a local small businesses with a day to day transaction of lend to collect daily loan offer with affordable interest rate.

● All.

● Ampliar programas de apoyo a zonas de vulnerabilidad a terceros países, o buscar cooperación es que llegue a los más necesitados e ir dándoles un acompañamiento

● As women are involved with this project, so UN should consider replicating.

● Business that clearly accommodate those with disabilities

● Capacitance communities to engage in entrepreneurship

● Considero que debe darle continuidad a las capacitaciones para que los usuarios hagamos mejor uso de la información financiera

● Continue supporting the local women led and youth led initiative to increase accessibility of services by women and girls in hard to reach communities.

● Direct training to entrepreneurs on appropriate technology application and innovation

● Diversification of business opportunity innovations and mentoring of Entrepreneurs

● During project designation, the target beneficiaries were not fully involved hence still some gaps which need more attention. some of the areas are…; Sustainability component is not there and exit strategy

● E-Commerce, promoting and upscaling global entrepreneurship especially for marginalized and vulnerable groups

● Everything is good

● excellent actions in the SDGs

● Finding, after training participants, small fund should be made available to start practicing what was learnt in the class

● Good governance in private sector to make engine of growth

● Government should already implement what has been proposed

● I don't know

33%

35%

26%

3% 1% 0%

2%

The project activities and content reflected the perspectives of women, youth, people with disabilities, or other vulnerable groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

109

● I live in a rural and the poorest province in my country. I was impressed by the reach of the project because while my country (the capital city) is hosting many UN regional offices, UN programmes are unheard of in these parts, though I suspect they are visible elsewhere in the country and region.

● Implementation of capacity-building training manpower , creativity to this various group,

● improving capacity-building

● Improving EE/RES secondary legislation for all countries

● Innovation simply means new methodology in doing things but on the aspect of activities of international organization in a particular country, the people that benefit almost everything are people at high, not people at lower, and whereas people at the lower class is most needed in the programme.

● innovations from women

● Innovative digital finance solutions and regulatory frameworks to support both financial inclusion and growth of SMEs, especially those owned and led by women. It is a new perspective for private sector.

● Innovative or unique from project that UN should consider they must teach also at local high school to give light to our future leaders

● it is an innovation project as it has increased women participation in business activities post-covid 19 era.

● It is necessary to give a valid and uniform assessment of the data (through the UNFC and UNRMS system), which is necessary for planning and understanding the issues in the preparation of strategic documents. Therefore, the application of the UNFC and UMRMS system for all resources, at the national level.

● la digitalizacion en el mundo globalizado

● La inclusión de jóvenes, para el fortalecimiento de sus habilidades y destrezas con una temporalidad más temprana.

● Large scale multifaceted project allowing for rapid intervention

● Las recomendaciones para tratar económicamente este tipo de eventualidades

● Let's the poor BREATHE

● Los emprendimientos liderados por mujeres, con criterios de triple impacto: Económicamente rentable, socialmente inclusivo y medioambientalmente amigable.

● More online activities

● N/A

● Need to upscale countrywide best practices to encourage or handhold MSME as an information brochure

● No comment

● Nothing at all, the project was very satisfying

● Nothing to say

● Opportunity or skills sharing from neighbouring countries. Forging business relationships and Empretec relationships.

● organize face-to-face courses for policymakers to support small and medium enterprises

● Our company benefited from Covid-19 by contracting testing & certification of medical Covid facial masks

● Platform or some sort of reimbursement for under privileged individuals who want to be part of this initiative

● policy dialogues hybrid dialogue inclusion of private sector

● Practically of the topics that were discussed

● Reducing border barriers, increasing non-refundable investment support for small and medium-sized enterprises in developing and underdeveloped countries, and improving cooperation between countries

110

● Should highly concentrate on youth and women empowerment especially on agricultural technology since the unemployment rate in the country is still high and is one of the sector which can help to reduce poverty yet still considering the best practices to reduce pollution of the environment

● Still figuring out on that

● Supporting small business for access to grants and Entrepreneurship training.

● Targeting people with disabilities and the vulnerable groups

● The best practices of project outcomes delivered in other countries should be supported and encouraged for adoption in other countries.

● The combination of independent study, YouTube videos, exercises, moderated online participation and tests facilitated greater engagement with the material, learning from the facilitator, and peer learning

● The cost of tools and others types for calculate

● The focus on MSMES and the practical approach undertaken allowed for impactful outcomes, which is not always the case of other technical cooperation projects.

● The global initiatives towards post covid 19 resurgence...its unique in tackling the novelty covid 19 and should be improved upon for future resilience and R n D by UN and lessons learn should be noted

● The number of times the trainings are done

● The online courses, webinars, and information dissemination.

● The opportunity for the usage of online tools for the dissemination of EMPRETEC programme to impact large participants

● The project, I interacted with was on formalization of MSMEs through the Cooperative business model and to small extent topics on Enterprise development were touched.

● The project's focus on capacity-building and skill development for MSMEs is vital for their sustainable growth, emphasis on sustainable and inclusive development, and the project's efforts to leverage technology and digital solutions for MSMEs are forward-thinking.

● The role competition policy in the economic recovery of MSMEs. Reason: MSMEs continue to face challenges related to their interaction with bigger counterparts. For instance, suppliers of agricultural produce to bigger multinational buyers. Traditional competition law rules do not present effective solutions. Consider issues of abuse of superior bargaining power and abuse of buyer power.

● The studies might contain innovative and/or unique conclusions and recommendations, so the UN should consider the best ones for replicating or upscaling.

● The training exercises to improve knowledge

● The uniqueness of this project, time span and evolution was specially in terms of allowing us as project managers to bring in new ideas and try them out. I have done projects before and also tried to implement new ideas, but this project was a clean slate and a lot of room for innovation. it was challenging but very enlightening. Such room to introduce ideas should be considered in projects as opposed to straight jackets. In the spirit of the new normal, projects should evolve and fit with the times.

● The whole approach was innovative, enabling one stop shop for mechanisms and best practices towards post-Covid-19 resurgence of the MSME sector.

● the work dedicated to the analysis of structural barriers to trade was very relevant and helped to identify measures necessary for facilitation of exports. Specifically, conducting flash studies to maintain evidence-based feedback from MSMEs on barriers to trade and access to external markets is crucial, especially in the context of current supply chain deterioration context in ECA region caused by the war in Ukraine. Furthermore, the work dedicated to profiling of women-led MSMEs was insightful and helped to develop legal amendments supportive to women employability.

● UN Projects

111

● UN should consider upscaling the Empretec behavioural approach to entrepreneurship skills development due to its high impact on motivation and confidence of participants and success of their business ventures.

● Undertaking a needs analysis and developing programs based on the needs of the MSMEs

● Upscaling new starts ups and contribute to identifying new/improved measures to increase businesses

● Upscaling the development of investment attraction mechanisms for energy system

● Use of online platforms. It helps people in MSME sector to even participate whilst near work.

● Women entrepreneurship can be upscaled to more people living in rural areas.

● yes

● YES ; SDGs is my occupation professional

Since completion of the project 5. Please share examples on how you were able to use the knowledge or skills acquired through the project in

your work.

● Access to information in one place was important to me.

● Acquired the best practices to the use and reduction of pollution while maintaining high level of Production

● An important project that I worked on, I highlighted key issues that are vital for the MSMEs whereby we provided entrepreneurial training, business counselling support and access to credit facility which helped the business to survive during and past the COVID pandemic.

● As a hoúsewife and a farmer, The acquired knowledge is helping me

● As an import-export retail business that I'd like to develop, I haven't yet applied the acquired knowledge toward my business.

● as part of my consultancy activities and with my students, including public administration officials in vocational training

● Asesorias a pequñas empresas y personas fisicas empresarias, par dar cvumplimiento a sus obligaciones fiscales y de información financiera para la obtenión de financiamienmtos. Cumplimiento en disposiciones de actividades conm enfoque a la sustentabilidad y cuidados ecoplogicos.

● Aun no he realizado un proyecto percé que haya sustentado las capacitaciones pero si comparto los conocimientos

● Better appreciation of challenges of MSMEs in accessing markets and finance which I incorporated into my analysis of competition law cases.

● By explaining deeply for the people at work how opportunity is being ma(d)e use of effectively and efficiently.

● coming up with easy women’s problems in farms

● COMPLETION AND CAPACITY KNOWLEDGE GREEN TECHNOLOGIES

● Conducting surveys and interviews with entrepreneurs, especially in rural areas, gave a deeper understanding of the existing problems of small businesses.

● De várias formas, uma delas seria a busca de parcerias com outras órgãos e entidades para assistir as micro e pequenas empresas no processo de exportação

● Desde la academia, socializo los temas tratados y como se puede adoptar soluciones de otras latitudes, previa adaptación a nuestro entorno o país. En el ámbito de las consultorías damos a conocer a clientes que en su gran mayoría no solamente son pymes sino también empresas familiares, respecto a los desafíos, puntos de nivelación necesarias y oportunidades para la

112

exportación de sus productos. Esto de de vital importancia habida cuanta que estamos en una region tri nacional, conocida como las tres fronteras. Ciudad del Este, Paraguay; Foz de Iguazú, Brasil y Puerto Iguazú Argentina.

● Design and implementation of new courses for MSME about internationalization and digitalization in my country

● Diversification of business opportunity innovations

● education for sustainable development

● En mi trabajo mejoro mis criterios de aplicación de la normativa

● En mis proyectos de crecimiento empresarial.

● Establish priority policies for women-owned and female-intensive businesses and social enterprises

● For future errands

● Giving charities to the needy people

● Goal setting was one of my best Pec, after the training l set meaningful , reasonable and achievable goals, . My friend invited me to join and play an online that will pay double in 30 days but I remembered being taught how to take calculated risk, l turn it down only for the scheme to fail after two weeks, my business money remained intact because l apply what l learn.

● Health and safety should be our own responsibility

● I am developed with accounting method

● I attended training & workshops

● I have attended the training and applied through my independent business development consultancy assignments by providing further trainings, information, and advice

● I have been able to design andupscale my projects within the circular economy space

● I have learned to utilize the E-view and SPSS analytical tool in identifying policy trends and priorities in the MSME sector in The Gambia

● I have not.

● I incorporated the skills into my organisation policies to advance youth meaningful employment and community development

● I just established an NGO which focuses on Climate change

● I learnt on how to reach out to prospective clients through the use of technology such as phone and internet to advertise my small business using whatApps.

● I managed to access GIS software and intend to use it in agribusiness interventions.

● I managed to understand the key problems that arise in the supply chain of critical mineral raw materials during crisis situations (pandemics, etc.), especially in conditions when there is no uniformity in their available data (in terms of quantities, availability, import-export, needs, etc.).

● I see solutions in any problems or situations. And it has improved my working abilities

● I was able to protect my self from covid and help my family gain from my skill too

● Implementar protocolos de bioseguridad en las asociaciones o empresas

● Important to use skill acquired especially for women entrepreneurs and employees how easily converging can enable to get them credit , skill up gradation and special needs yet being in competitive world.

● Improved my appreciation of synergies and partnerships in development.

● Increased the level of communication using internet. More active collaboration with partners to create consortium and participate in new tenders in area we didn't participate before. Expand area of activities.

● Information sharing om border formalities

● Innovation and best practices

● Installed wifi in my village home using solar to ensure I continued working in spite of remote location and lack of electric grid

113

● Introducing net metering for PV Autoproducers

● It enabled me to go for computer training

● Like Business Development Service (BDS) Centre, all knowledge acquired through the project will continuously impact the delivery in terms of training for the capacity-building of MSME in Benin

● Looking for innovation to solve emerging society issues is a tool for success

● Material generated through the project was of sufficient interest to Member States that it resulted in multiple invitations to present the work at various forums, and this enhanced links between policymakers and the UN.

● Mothers in headhouse

● My contribution has been in Energy Sector ( SME)

● N/A

● No capital to invest on the knowledge

● None

● online course

● Only in small entities that need prepare financial statements to share to Banks.

● Our company benefited from testing and certification of Covid-19 related facial masks

● Por mi actividad profesional, revisamos información de varias actividades económicas y las divulgaciones de los efectos del covid no pueden ser generales, porque a unas actividades económicas la pandemia hizo crecer sus operaciones comerciales

● Programme development and management - applied the knowledge and skills gained in initiatives crafted for MSME development for nationwide implementation Mainstreaming sustainability principle - sustainability principles were embedded in several MSME development initiatives of the office, ensuring that environmental, social, and economic considerations are integrated into our frameworks.

● Propagation

● Report findings were publicly disseminated amongst Development Partners (via different modalities and platforms) and were subsequently incorporated into their project design. The key findings were included in the CCA while certain policy recommendations utilized to provide inputs for EU annual report.

● Roll-out new training material developed during the project and disseminated through ad-hoc online sessions to national counterparts

● Several included mechanisms were subsequently utilized on certain level in my work environment and daily activities, for example, available state support for the MSMEs.

● Shared as a best practice to other partners that we are working with and rallied the critical role of Cooperatives in formalization

● sharing my knowledge with women entrepreneurs to uplift their businesses..

● Support decision-makers in the country in the elaboration of initiatives, and measures for the development of female entrepreneurship.

● The analytical study generated policy recommendations, business operations, communication, and financing options that might be used in future work.

● The examples of businesses, how they adjusted to the situation gave me new ways of thinking how to improve during challenging times.

● The knowledge gain was highly considered helpful in the areas health emergencies and livelihood survival greatly & the should shared through the programme help migat

● The knowledge has been of great help in sourcing, grouping and keeping up with marketing. And secondly, it has greatly improved my view on record keeping.

● The ocean economy is not very well explored in my area. The ethnic groups have always avoided water out of cultural and spiritual reverence. My development programmes never included the ocean economy until after this project. I have started an awareness initiative that presents the ocean as a

114

possible and lucrative source of livelihood. I believe the initiative is making a breakthrough because locals (though still very few) are beginning to lodge application for fishing licenses to the authorities.

● The online training course was a good example to highlight . It was possible to gather participants in an area which has not been covered before., the link between competition and SME policies.

● The products developed correspond to useful tools when supporting MSMEs in specific areas, such as competition law and policy.

● The project provided references to us in supervising the partnership between MSME and large/medium companies.

● The study helped us refocus on the new needs of MSMEs

● Through training.

● To be able to conduct trainings online, has been my best rewarding experience.

● To teach students

● We are able to understand and tackle similar challenges in future as we are better informed. We are more resilient and adaptable than before.

● We have organized workshops in relation to the UNCTAD MSME project in Thailand both national and regional agencies.

● We organizing all of training and expos by online and conducting IT technology for our operations

● We used the skills learned in undertaking needs analysis to undertake a needs analysis for our capacity-building programs

● Women capacity-building is important to acquire skills through knowledge.

● Working with FAO and UNDP in developing National Horticulture and Agro Processing Strategies

● Yes onboarding some companies to ecommerce platform. This initiative was lead by my self through skilled gained in the training

6. What are the challenges, if any, in applying the knowledge or skills acquired through the project to your

work?

● Lack of support from the body at the end of the training. So the knowledge is not practicable in the wider market

● Some of the policy and technology that was discussed cannot be accessed locally

● Access to full process of the project

● Access to new suppliers was difficult, as they require higher quantities to be requested. Creation of association facilitated to solve the issue.

● Accessibility to the most under developed rural areas.

● ampliar el conocimiento y plasmarlo en proyectos factibles de acuerdo a la realidad de cada pais

● Application is always not easy at first but I pull through

● beyond an online evaluation like what is currently done here, set up an on-site follow-up programme after the training

● Budget constraints in implementing initiatives

● Business environment in my country with limitations for MSME

● Capacity-building, adequate financial support and mentoring

● Changing of focal points in government counterparts and partners

● Coming back to business after a long vacation is terable

● education for sustainable development

● Falta de conocimiento de terceros u a la vez creen que no sería necesario

● Few institutions/enterprises/decision-makers/experts contribute and facilitate sharing of good practices.

115

● Financial access

● finding buyers of the product

● For me about the job description in every profession

● For Thai MSMEs, it is required a number of knowledge and skills to confront the obstacles of doing business in the post Covid.

● Funding is the biggest challenge. I am unable to go as far as I wish to because even pitching a request for funding is time consuming with too many technical requirements.

● Get the community mindset on board

● Getting gadget to use especially on disabilities peoples

● Governments and professional colleagues are not interested.

● GREEN ECOMIE GREEN INDUSTRIAL PROCESS ...

● Healthy is our daily routine for every I our capacity-building

● If skill training will not be available.

● Information availability, financing models

● Insufficient guidance and strategic planning by senior management.

● It was a unique situation, people were in finding difficult to normal the new social behaviour as results of covid nigh

● It's still premature to outline any challenges.

● lack of funds to be able to reach out to all of the entrepreneurs.

● Lack of infrastructure & unskilled people merged to work with me slow down the job

● Limited country data availability to conduct analysis for effective policy decision-making process

● Limited resource envelope and lack of practical skills.

● Limited resource in undertaking effective surveys that will be used to guide a programme design

● Limited resources

● Limited resources and skills to implement strategies to the full capacity

● Logistics

● Los cambios bruscos que se tienen que efectuar para atender oportunamente este tipo de eventualidades de acuerdo con las necesidades de las empresas

● Los retos es cambiar la manera de hacer el trabajo aplicando los lineamientos de la niif para pymes

● Low competencies of SMMEs in adopting new technologies for attaining quick recovery from the effects of the Pandemic. Many SMMEs still look up to Government for support.

● Mainly the biggest challenge is the financial aspect of new startups.

● Master the tools

● N/A

● Need of repository of country-wise concessions or hand holding for MSMEs at one place

● No challenges

● No difficulties

● no problem

● Non

● None

● None so far.

● None.

● Not accessed

● not really

● Not yet

116

● O maior desafio seria a criação de um ambiente colaborativo para oa implementação de políticas públicas voltadas ao incentivo das exportações para micro e pequenas empresas

● One challenge was to promptly adjust the lab equipment and train staff for testing medical facial masks

● Opportunity seeking and taking initiative, l have seen an opportunity l would have taken the initiative but financial challenge caused a set back.

● Our governments intervention about policies

● project didn’t finish

● Que sea útil en la toma de decisiones.

● Resources to reach more traders on how they can do business the time of COVID -19.

● Societies of womens

● Some business shuttered when the COVID hit the world.

● Subsidies process

● Technical and financial capacity limitations

● Technical facilities

● The biggest challenge was persuading respondents to provide factual data and answer the questionnaire.

● The challenge is we raised high expectations from our member states to do activities together and with no funds for the continuation of the work, it is frustrating both ways.

● The challenge was to classify critical raw materials in a uniform and recognizable way through the realization of the project and make an assessment regarding their availability and needs for the health sector, which was realized most often in conditions of insufficient available data.

● The challenges are coordination and synergies amongst relevant agencies/entities.

● The cost of internet charges is currently high in my home country. high internet intermittent or rather bad internet signal.

● The main challenge concerns the establishment of strong partnership for the funding to support the delivery of EMPRETEC programme in developing countries

● The only challenge derives from the very specific focus of the work programme I coordinate, which doesn’t necessarily allow for extensively addressing general and broad MSMEs challenges.

● The only challenge is funds.

● The policy recommendations were not costed that creates difficulty in promoting their implementation amongst UN agencies or other development partners.

● Their uselessness

● Total y plenamente posible desde el ámbito académico estoy proponiendo temas que hacen a la necesidad de la potenciación de las pymes, desde sus digitalización, formalización y actualización en temas claves para fomentar la exportación. Y desde el ámbito de la consultoría y desde la cámara de empresarios de Ciudad del Este y Alto Paraná estamos promoviendo un Diplomado en Comercio Exterior dirigido a empresarios pymes y empresas familiares.

● Training Ministries, Agencies and Regulatory bodies

● Trasmitir adecuadamente los conceptos de sustentabilidad.

● We have been able to apply the knowledge.

7. Please share examples, if any, of how the project included or supported women, youth, people with disabilities or other vulnerable groups.

● Accounting and budges

117

● All projects that will be implemented has to include Gender and Development (GAD) and must be inclusive.

● All socially and economically advantaged groupings such as the women have encouraged to form cooperatives and access funds constituency development funds to alleviation and improve their welfare.

● By spreading the information to people in need of it.

● CIRCULAR ECONOMIE OF RECYCLE WASTE SOLID

● Community project on understanding the need for non pharmaceutical adherence practice.., proper information about the viral nature, the strains ,transmission of covid and also fight disinformation due to lack of information.

● Como complemento a la mirada exportadora y desarrollo digital de las pymes, también impulsamos y socializamos los criterios de sostenibilidad con triple impacto: económico, social y ambiental, como política de las empresas privadas y dentro del mismo se enfatiza la oportunidad inclusiva para mujeres, jóvenes, y personas con discapacidad, u otros grupos vulnerables.

● conscientization in education for sustainable development (SDG5 , SDG10)

● Differently able were able to seek required skills and seek for sustainable employment.

● Diversification of business opportunity innovations

● Door to door activities, service was brough close to people and hence elder and disable women and girls and men were able to access it

● Empowerment

● Enhanced business models and resilience

● gender equality in panel discussions inclusion of gender dimension in report

● Have no detailed information

● Humanity and responsibility

● I don't know because I can see any positive update on us

● Iam vice chairperson and cofounder of Autism Rwanda so here I tried to educate women's and men's in this organization how to use devices for people with disabilities

● I'm looking forward to get disabled volunteers and include the in creating a clean society

● In Benin the selection of the beneficiaries includes creterias which has eased the inclusion of women, youth and other vulnerable groups

● In rural surveys and surveys, more than 50% of the building materials trade is run by women

● It increased the level of knowledge

● It was important to invite women-in-bussness in our activities . A PhD student used our output to finalize a thesis of women in finance, in Moldova.

● knowledge and technical skills

● La invitación fue generalizada, sin condicionar condiciones.

● N/A

● NA

● New tools to support to MSME owned by women

● No aplica

● No los hubo.

● No tengo conocimiento cuantas mujeres incluyo

● None

● Not in my cooperative sector

● not really

● One of key targets of the project was to ensure at least 30% of women participation.

● Our testing lab for testing and certification of medical facial masks included female staff

118

● Over 50 per cent of participants in activities supported by this project are women

● Participation

● Personally as a woman, l was really empowered, l saw myself as a redefined entrepreneur, l had become more efficient in my work

● Por la pandemia y la falta de movilización libre, propicio innovar estos medios de divulgación que facilitó la divulgación de esté tipo de información

● Priority will be given to support first and the level of support will be higher than that of a regular business

● Project participants got starter packs and were able to uplift their livelihoods.

● Project supported women, youth and vulnerable families by promoting EE/RES

● Proyectos privados con PERU LNG en Peru se viene ejecutando productos productivos y con nuevas propuestas para mejorar la comercializacion y articulacion

● Research on women's entrepreneurship enabled policymakers and ministerial staff in Member States to have a better understanding of how support given to women in business pays off.

● Still doing researches on that

● The innovative aspects of doing things was the best part of the training, with Data your market will sell.

● the necessary trainings given to them are useful to them

● The project conducted various capacity and skills training activities were beneficiaries mainly targeted women, youth and vulnerable groups.

● The project had a gender component.

● The project included an analysis of women's participation in "Small and Medium Enterprises" projects, which were represented both in the field of critical raw materials and the supply chain during Covid-19, and which are expected in the post-Covid -19 period. This analysis was evaluated very positively.

● The project included and looked at those target groups and can be a starting point for some more detailed activities

● the project produced analytical piece on the impact of Covid-19 on trade and business development prospects of women-led companies which fed into subsequent policy action adopted by the Government in relation to childcare as a part of women empowerment and employability.

● The project was supported for the women and youth through capacity-building training

● The project would also cover on areas of " COVID-19 Seed recovery for women and Youth"

● The said group had been able to get access to entrepreneurship training and access to credit facility.

● The survey sampled enterprises from women, youth and PWDs

● There was a deliberate effort to include women and youth among the participants

● There were Covid Relief funds disbursed to women and youth MSMEs to cushion them during the crisis. In Kenya for example the group WomenWork in partnership with Kenya Chamber of Commerce and 4G capital ensured we got the funds directly to our phones through mobile money.

● There were specific trainings targeting women entrepreneurs of developing countries in East Africa, which understood that these MSMEs also needed to shift from informal trading to formal businesses.

● these people are highlighted in the recommendations made but the implementation is up to the government

● Through knowledge sharing seminars and workshops

● Training of women entrepreneurs in Malaysia/ Empretec training in the context of Cross-border trade activities/Training on Farming as business/E-Regulation platforms allowing for formalisation of informal sector.

● Upskilling of youth and women businesses

● Use of sign language

119

● We do have a success story from a woman from Zimbabwe who participated in the online course: see her story. link here. https://unctad.org/news/how-make-small-businesses-developing-countries- more-competitive

● we have not any special project

● While my farming project supports mostly women and youth, the new initiative is more popular among men.

● Women Empowerment

● Women, youth, people with disabilities are we support them

Results of the project 8. To what extent do you agree with the following statements?

Outcome 1A Strongly agree

Agree Somewhat

agree Somewhat

disagree Disagree

Strongly disagree

Do not know

Total

The project contributed to formulating and implement enabling policies on green, resilient and inclusive entrepreneurship for MSME promotion in post COVID-19 resurgence

32% 39% 15% 10% 2% 0% 2% 41

Outcome 1B Strongly agree

Agree Somewhat

agree Somewhat

disagree Disagree

Strongly disagree

Do not know

Total

The project activities contributed to expanding my business venture or to starting a new business

15% 42% 18% 0% 12% 0% 12% 33

The project supported my business by increasing sales

15% 27% 24% 3% 12% 3% 15% 33

The project activities contributed to the creation of new jobs

18% 39% 21% 3% 12% 0% 6% 33

Outcome 2 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project contributed to facilitating MSME registration and formalization

100% 0% 0% 0% 0% 0% 0% 3

The project activities contributed to decreasing the administrative costs of starting a business

0% 100% 0% 0% 0% 0% 0% 2

The project contributed to increasing the number of businesses registering

50% 0% 50% 0% 0% 0% 0% 2

120

Outcome 3 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project contributed to improving financial inclusion and MSMEs access to finance

11% 44% 33% 11% 0% 0% 0% 9

Outcome 4 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project activities contributed to increasing MSMEs access to innovation and technology

27% 45% 27% 0% 0% 0% 0% 11

Outcome 5 Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project contributed to new/improved measures on consumer protection and competition

11% 33% 56% 0% 0% 0% 0% 9

The project contributed towards new/improved measures on agricultural quality and food loss reduction

13% 25% 50% 0% 0% 0% 13% 8

The project contributed to increase MSMEs access to markets, including export and integration into value chains

13% 63% 25% 0% 0% 0% 0% 8

8-B. To what extent do you agree with the following statements?

Strongly agree Agree Somewhat

agree Somewhat

disagree Disagree Strongly disagree

Do not know Total

The project contributed to improving the resilience and competitiveness of MSMEs in post-COVID-19 resurgence

38% 35% 16% 4% 3% 2% 3% 106

The project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups

32% 32% 28% 2% 3% 0% 3% 105

121

9. Please share any examples of how the project has contributed to the resurgence of MSMEs, policy or

economic improvements, or advancing sustainable development in your country.

● A través del sistema de registro, importante para conocer cómo se desarrollan económicamente las

empresas.

● Access to finance and creation of programs so that MSMEs pivoted to continue with their operations.

● After that , Most policies at work has aspects protection and awareness

● As always in Armenia nothing ever helps.

● By applying the knowledge acquired

37%

35%

16%

4% 3% 2%

3%

The project contributed to improve the resilience and competitiveness of MSMEs in post COVID-19 resurgence

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

32%

32%

28%

2%

3% 0%

3%

The project contributed to better outcomes for women and youth entrepreneurs and other vulnerable and marginalized groups

Strongly agree

Agree

Somewhat agree

Somewhat disagree

Disagree

Strongly disagree

Do not know

122

● By developing good guidelines for MSMEs to assure resilience and progress towards a circular economy in sustainable resources management and critical raw material supply chain solutions.

● By touching the concern people.

● City of Harare developed a MSMEs policy which now recognize the importance of informal sector and has included use of technology as one of the building blocks.

● Con éste tipo de proyectos apoyados por la ONU, desarrollados por especialistas son de mucha ayuda para los que quieran y estén interesados para aplicarlos en beneficio de las actividades económicas de mi pais

● conscientization in education sustainable development in companies

● Diversification of business opportunity innovations especially into Renewable energy

● Economic improvements through Income generating activities

● El proyecto desde mi percepción, lo que ha logrado es aportar herramientas de singular importancia para el fortalecimiento de las empresas que han sobrevivido a la pandemia, a entender mejor los dolores del mercado, y las oportunidades y barreras para la exportación.

● Empowerment

● En mi trabajo no tengo acceso para poder dar un ejemplo

● GREEN JOBS. GREEN TECHOLOGIES

● I don't know

● I joined the project as a freelance development practitioner, and was not commissioned by my government. I don't think any representative from my country joined the project. However, I do engage in multi stakeholder forums, workshops and conferences where I make a point of raising issues of SMME resilience for inclusive development.

● In my country, I see a lot of businesses are moving into trading online, thus increasing their market.

● influenced Thai SME policy and national strategic plan

● it contributed with easier access to supporting mechanisms, as well as tailored guidelines by country for the MSMEs in order to overcome all challenges from the new environment caused by the pandemic.

● It's really improved the small through new methods of engagement using available technology and innovations to drive market ...also in the areas of supply chain innovation to deliver goods and services

● logistic service is more developed

● Maybe for those who are already in business

● mejorar los ingresos economicos de las familias, contar con capital, mejorar los costos de produccion

● More recently in my country Kenya, various funding opportunities are now available to MSMEs through the convenience of mobile phone. This includes the Women Enterprise Fund, the Hustler Fund, and Group loans. Grant Managers such as the KCIC Group have ensured sustainable development in our country by promoting and funding sustainable projects throughout the country.

● MSME is going to be back bone for developing economies. Focus on policies to encourage them especially guide them to explore eCommerce advantages

● MSMEs started using e-commerce and this lead the development of digitalization and consequently facilitate business operations

● Muchos mipymes se reinventarse y a la vez entendieron la importancia de que hay que estar preparados ante cualquier eventualidad, no sobrevive el más sabio u cauteloso, sino quien tenga una idea innovadora , la desarrolle y sepa como llegar al consumidor

● My appreciation of MSME challenges in markets contributed to effective enforcement. We were able to progress complaints faster and also require large buyers to review oppressive contracts to support sustainability of their MSME suppliers.

● N/A

123

● na

● New skills were mastered in the conditions of the pandemic, and certain types of risks of natural disasters and pandemics began to be considered in a valid way, which also influenced the introduction of new jobs for developing and solving this problem.

● No comment

● Not reached affected communities

● Not yet

● Only in cases the entity has the obligation to prepare financial statements to acces to loan in a local bank.

● Our government poor credit methods

● our project was a good example for resurgence MSME, after that, we will prepare for the future the sustainable strategy for vulnerable groups.

● Project allowed for distribution of fiscal incentives through e-Regulation platform El Salvador. EPF strategies developed for beneficiary countries and recommendations on implementations included measures specifically oriented towards MSME rsurgence

● Promoting olive biomass pomace for heating and hot water energy demand

● Relevant

● Revamping the Waste management system through collaboration with government in the sanitation of the environment in Calabar, Nigeria.

● Se logro la reorentacion operativa de PYME al establecer politicas decontratación depersonal

● Several online events were organized to share success stories of Empretec participants during COVID 19 restrictions which raised awareness of others on business opportunities during the pandemic.

● sharing knowledge

● so far there is no significant contribution however sensitization to the communities and to companies that have great impact on the green technology were the targets for training how they can contribute towards the climate emission

● Some traders they have knowledge on how to control resources for business and looking ways on how to find start-up Capital

● Still doing research on that

● successful project to put up

● The country shifted from somewhat backward thinking and started to focus on ideas that will positively impact the society more especially in the rural areas

● The focus of the project was to support MSMEs to overcome the constraints enhanced by the pandemic, so enhancing resilience in a sustainable manner was a constant goal.

● The Ministry of Agriculture is coordinating National strategies meant to empower farmers towards sustainable production and sustainable Value Chain Development. Women, Youth and People with disability are being supported to ensure NO ONE IS LEFT BEHIND

● The outcomes of the conducted studies helped to develop policy recommendations provided to the government in support of economic improvements and advancing sustainable development of the country.

● The partnership involved the Microsoft Small Enterprise Authority of Kenya and hence got practical feedback from the study

● The project enabled better access to information and contributed to the increase in the number of MSMEs

● The project had a positive impact on MSMEs, there were a lot of job creations reported.

● The project has trained entrepreneurs to be more efficient and confident in their work thereby contributing to economic development

● The project jobs The project activities contributed to the creation of new jobs Strongly agree The project activities contributed to the creation of new jobs Agree The project activities contributed to

124

the creation of new jobs Somewhat agree The project activities contributed to the creation of new jobs Somewhat disagree The project activities contributed to the creation of new jobs Disagree The project activities contributed to the creation of new jobs Strongly disagree The project activities contributed to the creation of new jobs also the project contributed to improve the resilience and competitiveness of MSMEs in post COVID-19 resurgence here in Rwanda

● The project revealed challenges faced by women-led MSMEs which subsequently were complemented by development of gender-based assessment jointly with the WBG. Furthermore, challenges faced by women in terms of access to ECD facilities (which became more acute during the pandemic lockdown) were addressed by the GoM policy on regulating the ECD facilities in private sector that contribute to women's economic empowerment and employability through regulation of establishment of creches and kindergartens under private companies employing young women with children, as well as facilitating expansion of private and public ECD facilities as one of the measures supportive of women labour participation.

● The project strengthens youth and women's employment opportunities, therefore, improving economic and community development.

● The project supported a Policy Implementation Forum: Supporting Micro Small and Medium Size Enterprises (MSMEs) in the Post COVID-19 Pandemic Era - Promoting MSME Formalization This event was jointly organised by DSDG/DESA and the Ministry of Trade, Industry, Regional Integration and Employment (MoTIE) of The Gambia and the UNDP. It brought together high-level policymakers and development partners to review challenges and identify solutions supporting the growth of the MSME sector in The Gambia in the post-COVID-19 pandemic era, taking into account opportunities offered by the African Continental Free Trade Agreement (AfCFTA).

● The results were widely distributed to National stakeholders, some concrete actions were implemented by ODIMM Moldova by launching new support schemes for MSME..

● The SDG’s are simply seen as fancy paperwork that yields funds, without any concrete plans on how to implement them…. Nigeria

● The training opened the eyes of the participants to the potential of MSMEs especially through leveraging ICTs

● there is a lot of development in my country as many businesses were in the verge of closing up during covid but thanks to the project, they survived.

● Through this project the government of the Republic of Zambia under the ministry of small and medium enterprises has greatly improved women participation in business activities through various coorporatives

● to support women with disabilities, students are mobilizing with companies and individuals to collect small equipment such as a sewing machine, knitting needles and balls of wool, raffia etc so that these women can start producing and selling fashion or decorative clothing and accessories

● we establish comprehensive business policies that prioritize women-owned businesses, female- intensive businesses and social enterprises

● Womens inclusive in polca, and governement

Looking forward 10. Are there other ways in which you think the UN could support the MSMEs sector in your country?

● I have a startup business in agricultural exports.

● Physical training and accelerator programme would be good

● 1. conducting relevant trainings on the implementation of MSME sector in the country. 2. development and sending of press releases on the report to government agencies.

● Access to finance for MSME, increase to financing resource

125

● Access to finance that may serve as post-training support for MSMEs

● be closer to local reality beyond best practices because there is no one best way but all depends

● Building the capability of BSOs, especially where the function of trade has devolved to regions, provinces and even lower forums. The Federal bodies in Pakistan are no longer representative of the MSMEs, especially no support for sectoral specialization.

● By establishing a zero-rated network programme that connects entrepreneurs with their immediate market/community, linked to other up to the international community

● By Funding Start up organizations and continue to provide classes for quality results.

● By provide some seed money

● By providing various training on how to manage businesses

● By sending technological tools

● By sharing knowledge and capacity development

● Collaborate with national governments to promote the implementation of best practices, in particular, to disseminate best practices in the development of the MSME sector.

● Conducting specific trainings in the field of financing and access to the market

● Considero que si dándole seguimiento con mas capacitaciones para ampliar esos criterios

● Continue supporting and mentoring MSMEs

● Design and develop policies for businesses in specific industries

● Developing BDS services/Facilitating business through e-tools/allowing for smooth implementation of new sustainability reporting standards introduced by ISSB.

● Direct engagement with private sector and capacity-building of private in promoting good governance

● Ejemplificando mas la información tomando como base la actividad económica para que sea comparable la información y poder medir el impacto en nuestro pais

● Enlightenment and also empowerment

● Grassroots propagation

● Ground -top approach which will break and limits the barriers of state bureaucracy and other means of engaging the people ..who will be directly affected

● I don't know.

● I hope to continue the indicative

● implementation of pilot projects with training and financial support for rural businesses

● In addition to training, the project could invite MSMEs to submit proposals for financial support so that there is a more practical element to the training

● In collaboration with UNCT to develop an agile policy paper that would highlight short- and long term measures necessary to stimulate economic growth (and improvement of livelihoods) amidst regional war-related crises and trade disruptions.

● In Moldova, a majority of micro, small, and medium enterprises (MSMEs) owned and led by women operate without access to resources and benefits. Assistance could be provided to the ministries and, agencies and to the Secretariat of the Prime Minister on the development of policies and initiatives in support of entrepreneurship, especially for women, youth, and the elderly, of digital finance solutions, transition to sustainable trade and a circular economy.

● Increase support to improve the business environment as well as to reduce the cost of doing business, including: A. Access to finance/capital/credit B. Support the establishment of the Single Window Business Registration (e-services) for countrywide coverage C. Improve the quality of infrastructure for product development and trade D. Support the development and improvement of business environment-related infrastructures etc.

● Information Dissemination

● more financing for the conscientization in the education for sustainable development

126

● More funding opportunities are required to meet the demand especially with the economic crises being witnessed. Do not tire in doing good.

● More targeted activities or studies, specified to certain issues/challenges on MSMEs will be helpful.

● MSMEs need continuous training to enhance their knowledge and skills and to face new challenges: there should be annual courses on a wide range of issues for their update.

● n/a

● na

● New webinars or courses about the key issues for MSME

● Not specific to UN, but the MSMEs in my country are one sector that needs support in many areas. For example, in my view, they need support on energy planning and management - eg more awareness to energy audits, awareness of the possibilities to improve, utilisation of RES, how the energy influence the price of their service/product etc.

● Project Funding for the ongoing UNDP Supplier Development Programme in Botswana. More MSMEs to be capacitated to be part of the Government and Private sector Supply Chain.

● Provide direct capacity-building support to MSME entrepreneurs, with priorities given to women, youth and groups in vulnerable situations; focusing more on the effective implementation of MSME supporting policies in line with demands of MSME entrepreneurs, instead of supporting the formulation of policies themselves

● Provide support in developing MSME friendly policies, complement government resources through financial and technical support

● Publicizing the work of our entrepreneurs internationally

● purchasing women innovations

● Si

● Si por supuesto. Es una figura de autoridad y relevancia, por lo que genera interés en participar.

● Si, compartiendo información de PYMES que requieran asesorias, en caso de contar con alguna estadistica relacionada.

● si, con apoyo de ONG, instituciones privadas y otros

● Sim.

● soft loans and more accessible training publicity

● Support in establishing technology and innovation hubs, centers, and incubations for youth, women and disabled persons

● The UN would support the women and youth on Start-up capital, cross border traders lost thier capitals during the COVID-19 because of movement restriction and there was no business at all.

● There is a little support

● Through capacity-building

● Through capacity-building and mentoring projects.

● Through direct training of personnel on concrete examples, but including personnel in government sectors where, in accordance with procedures, the relevant issues are dealt with and where there are "fully trained and qualified professional workforce", in order to improve the effectiveness and efficiency of their work and fully realize the principles of sustainable development in the shortest period.

● Through organised registered cooperatives

● To open more international trading opportunities (business linkages)

● To provide grants and access to credit facility. To provide vocational training and value chain training.

● To support the underserved people MSMEs and marginalized communities' economic growth, the UN should collaborate with grassroots NGOs that promote employment and community development. The local NGOs are closer to the people and understand their challenges.

● Totalmente, a más de los programas que ya tienen desarrollado para aplicar, considero que la vía de llamados a proyectos de co creación, con amplia participación de los actores de cada región del

127

pais, es de fundamental importancia. Pues solo con el involucramiento participativo de los actores de cada región geográfica es posible generar confianza, luego adherencia al proyecto y finalmente sentirse orgullosos de los logros del trabajo en equipo, participativo, que no solo busca el resurgimiento de las pymes, sino que además siembre la cultura de sostenibilidad en línea con los ODS, y la cultura de la legalidad mitigando los casos de corrupción e impunidad, a través de las buenas prácticas empresariales y personales como ciudadanos. Me suscribo para emprender y caminar este sendero de la mano de ustedes. Considero que el llamado a presentar proyectos de co creación es el camino correcto e infalible.

● UN can do more on business development services, including as a follow and complementary to behavioural methodology of the Empretec programme. More efforts could be done to bring together and coordinate MSME related policies and activities of different agencies at a national level.

● UN should develop core team to discuss with developing economies countries the best practices across glove for MS ME and help them to have online real time repository of MSMEs, their issues and resolution mechanisms. UN can support this activity.

● work more with chambers of commerce

● yeah UN should allow local national and INGO to acquire small funding instead and support accept taking risk organization if the country wants move together localization agenda.

● Yes

● Yes

● Yes This is because the COVID-19 affected a lot of MSMEs especially we at the northern part of Ghana

● Yes by presenting international benchmarks for EE/RES through workshops and supporting countries for secondary legislation for EE/RES

● Yes keep training members of MSMES

● yes there are many 1.funding 2.couching after trainings will be useful. 3.capacity-building 4. study tours to learn from other countries.

● Yes through the trainings and expertise availability for the better and proper enforcement and participation of the policies

● Yes through training in entrepreneurship.

● Yes,

● Yes, because most of us were train without finding so we plead that next time small Grant be given for a start up

● Yes, in the universities

● Yes, Kenya. A review to establish how public policy and competition policies support or create challenges to MSME competitiveness and sustainability.

● Yes, most beneficial of this coordinators don't lodge the beneficiary account. I am on opinion that individual account should be credited direct. Because I didn't benefit from the programme too

● Yes, pick up from the recommendations of the project and and start engagements with stakeholders that promote and develop the MSE sector such as the Micro and Small Enterprise Authority in Kenya

● Yes, targeted resources for capacity-building and training on how such models as cooperatives can be suitable vehicles for formalization.

● Yes. Huge amount of options. Starting with eliminating useless jobs that we have in local UN. Some projects has admin costs that never have given any result.

● Yes. It's necessary to give stability to the incubators. And may be that UN can promote and support that initiative face the government

● Yes. Using technologies requires not only capacity. Resources are needed to purchase the necessary gadgets. On top of this, piloting a project in the country can increase replication potential if results are positive. Learnings and practice can help shape policy at national level. If UN can fund a pilot project where learnings are generated, that is ideal in Zimbabwe.

128

Final questions 11. In what type of organization do you work?

Type of organisation Total

Government 28

Micro, Small or Medium Enterprise (MSME) 17

Other private sector, Business or industry association, Commercial Registry Office 20

Other (please specify) 43

Other (please specify)

● Academe

● Academe

● Academic

● Association

● CAD Internazionale ● Centro de Estudios Tecnología e Investigación: Es una consultora privada pero vinculada a temas

de investigación con la Universidad Católica

● Consulting Company

● CONTADOR PUBLICO INDEPENDIENTE

● Cooperative Development

● Cooperatives

● EMPRETEC CENTRE

● Farmer ● Former civil servant of the Ministry and now retired and independent consultant in the Mining

sector

● Former Head of Branch at UNCTAD responsible for this project until 1 May 2021

● H

● Have worked in sector now advisory role

● I am a Business Advisor

● Independent business development consultant

● INGO

● International NGO working in Sustainable Development

● International Organisation

● International organisation

● International Organization

● International Organization

● International organization

● ITC

● NGO

● NGO /Expert

● None

● Nongovernmental Organization

● Public University

● Research institution

● SOCIOLOGIST in OR at FRANCE

129

● Think tank

● UN

● Un

● UN

● UN

● UN agencies

● UNDP

● United Nations

● United Nations entities

● University 12. What country do you represent?

Country Total

Albania 2

Armenia 3

Bangladesh 1

Bhutan 1

Botswana 1

Brazil 2

Burkina Faso 1

Cameroon 1

Colombia 1

Dominican Republic 1

El Salvador 2

Ethiopia 2

France 1

Gambia 3

Germany 1

Ghana 1

Guatemala 2

Honduras 1

India 2

Indonesia 1

Italy 1

Kenya 7

Kyrgyzstan 1

Lesotho 3

Macedonia 2

Madagascar 2

Malawi 1

Mexico 2

Moldova 3

Mongolia 1

130

Namibia 1

Nigeria 10

Pakistan 1

Paraguay 1

Peru 2

Philippines 5

Rwanda 2

Senegal 1

Serbia 2

Somalia 1

South Africa 3

South Sudan 1

Swaziland 1

Sweden 1

Tajikistan 1

Thailand 3

Togo 1

Uruguay 1

USA 2

Venezuela 1

Vietnam 1

Zambia 3

Zimbabwe 3

Total 101 13. What is your gender?

Gender Total

Female 42

Male 62

Other/Prefer not to say 1

Total 105 14. Do you identify as part of any of the following groups?

Type of vulnerable group Total

Youth 30

Indigenous 17

LGBTQ+ 0

Person with disabilities 2

Survivor of violence (domestic, gender-based, other) 2

Other vulnerable group (please specify) 24

131

Other vulnerable group (please specify)

● Above youth 1

● Academic 1

● Adilts 1

● Adulto mayor 1

● Adulto maytor 1

● Cross Border Traders 1

● Estado con alto índice de marginalidad económica y tecnológica. Oaxaca, México. 1

● Executive 1

● Grassroots women 1

● N/A 1

● NA 1

● No 4

● Non of the above 1

● None 2

● Pashtun Ethnic Minority 1

● People living with facial palsy 1

● Previously disadvantaged individual 1

● SIMPLE CITIZEN 1

● Single Mother 1

● Smallholder farmer 1 15. Are there any final comments that you would like to share about the project to help inform this

evaluation?

● A well curated and well executed project. We hope to have more opportunities like this.

● Agradecer al proyecto por habernos incluido

● Any

● Continue the good work.

● Does it matter? Have you ever considered anything said? Nope? What a surprise.

● en Uruguay para acceder a financiamientos se requiere información contable de acuerdo a NIC. La iniciativa es muy buena para que el microempresario cuantifique resultados u ordene sus finanzas internamente , pero a nivel país debe regirse por régimen reglamentado gracias por la generosa oportunidad que me brindaron y el excelente trabajo desarrollado

● Great programme it was

● I am hosting regular talks and training workshops and would be delighted to have a speaker from your office, and hopefully even a collaboration

● I enjoyed managing the project.

● I really need a financial support to support my MSMEs in the northern part of Ghana. Majority of their businesses have collapsed and it is making cost of living very difficult.

● I think everything is good

● In my opinion, in the implementation of UN projects, the majority of experts who could be engaged or active are not too interested in directly participating (lack of general knowledge, language, personal - direct engagement, somewhat weaker departmental communication in the state and private sector, etc.), which ultimately (in my opinion) affects the level of quality and the final design of the project.

● Is a good information for me

132

● It would be helpful for us if we are able to make partnership with UN Women or any UN agency.

● Keeping in touch with MSMEs

● Let's the poor BREATHE

● Many thanks and keep up the good work....while improving lives

● Many thanks for supporting for this work

● Me encantó la experiencia.

● Me ha gustado la capacitación, tal vez en otras similares se podría trabajar también con espacios de ILab, aplicando Productos Mínimos Viables, a través de interacciones de los participantes de diferentes países. Otra sugerencia es que puedan dar espacios para fortalecer los temas aprendidos en los 3 cursos con becas de CEPAL, generando llamados a proyectos de co creación en donde los participantes de los cursos puedan conseguir mayor permeabilidad respecto a los temas aprendidos, y así generar la sinergia necesaria con recursos y presencia de marca de ustedes. Todo lo aprendido puede complementarse con el uso de herramientas AGILE, y aplicando tecnologías de bajo costo y alto impacto para permear en forma sinérgica al campo o regiones de toda la riqueza aprendida en los 3 cursos.

● more successes

● More training The evaluation should not take a longtime to come evaluators to avoid the risk of forgetting the content

● N/A

● NA

● No

● None

● None

● Not really

● Not sure

● Personalmente agradezco por tomarme en cuenta para esta escuesta

● possibility of collaboration

● Share the project

● Thank you for the opportunity to participate in the training, and for the follow up

● Thanks for considering sharing the survey to get my feedback

● Thanks for the opportunity

● Thanks for the survey.

● Thanks for this program

● That all

● The project and upcoming successors should be of long-term with possible long-term financing to support infrastructure development, Capital funding support and long-term Capacity-building activities.

● the project helped to reveal real problems for small businesses and develop recommendations for overcoming them

● The project was a great opportunity to promote and support the MSME sector and Empretec network during COVID 19. Wish there were more projects like this one.

● The project was pro-poor and appropriate.

● The project would also cover on areas of sustainability, institutional support for continuety of the project, supporting material and equipment.

● The projection should initiate inclusive capacity-building's courses / sessions within regions and/or grassroot communities.

● The UN should incorporate NGOs into the project for fruitful outcomes.

133

● There are lots of information to share with you but it is confidential, that is one on one.

● This is a wonderful initiative, please continue with it as it sparks the slumbering creativity to some of us. Also, it shaping the communities to be better

● This project should deal with the individual groups not there heads by so doing you are dealing with this group of people directly

● Those trainings should be conducted in-person or hybrid inorder to make them more interactive, focused and effective

● training of smses

● Unicamente agradecer el apoyo que brindan

134

Annex 7: Evaluation Terms of Reference

Evaluation of the United Nations Development Account 12th tranche “Global Initiative towards post-Covid-19 resurgence of the MSME sector” (2023W)

TERMS OF REFERENCE

A. BACKGROUND A1. ABOUT THE DEVELOPMENT ACCOUNT

The Development Account (DA) is a mechanism to fund capacity development projects of the 10 economic and social entities of the United Nations Secretariat, namely: the Department of Economic and Social Affairs (DESA), the Economic Commission for Africa (ECA), the Economic Commission for Europe (ECE), the Economic and Social Commission for Asia and the Pacific (ESCAP), the Economic and Social Commission for Western Asia (ESCWA), the Economic Commission for Latin America and the Caribbean (ECLAC), the United Nations Conference on Trade and Development (UNCTAD), the United Nations Environment Project (UNEP), the United Nations Human Settlements Project (UN-Habitat) and the United Nations Office on Drugs and Crime (UNODC).

The DA provides capacity development support to developing countries in their implementation of the 2030 Agenda for Sustainable Development, as well as recommendations and decisions made in intergovernmental processes and relevant governing bodies. The DA-funded projects build on the mandates, individual technical capacities and comparative advantages of the respective implementing entities, while providing those mostly non-resident entities with the ability to operationalize their knowledge and know-how to deliver capacity development support at regional, sub-regional and country levels.

The Under-Secretary-General (USG) for Economic and Social Affairs is designated as the Project Manager of the Development Account with responsibility for overall coordination, programming, monitoring and evaluation, as well as for reporting to the intergovernmental bodies. The Project Manager is supported by the DA Steering Committee, who advises him/her on strategic policy and project-support matters.166 The Project Manager is also supported by the DA Project Management Team (DA-PMT) located within the Capacity Development Programme Management Office (CDPMO) of DESA, which assists with all aspects of the management of the DA, in particular with regard to programming, monitoring, evaluation and reporting. DA-PMT also liaises with the DA Focal Points in the implementing entities, who are most often the head of the entity’s unit responsible for project planning, project management, capacity development or technical cooperation, on all aspects of the management of DA-funded projects.

In response to the COVID-19 crisis, the Development Account has funded five short-term joint projects to help developing countries alleviate the socio-economic impact of the pandemic, including the project on Global initiative towards post-COVID-19 resurgence of the MSME sector (2023W).

A2. ABOUT THE PROJECT

166 The DA Steering Committee is composed of five members with one member representing each of the following implementing entities and key stakeholders: 1. DESA; 2. the regional commissions; 3. UNCTAD; 4. UNEP, UN-Habitat and UNODC (on a rotational basis); and 5. the Programme Planning and Budget Division (PPBD) of the Office of Programme Planning, Finance and Budget of the Department of Management Strategy, Policy and Compliance.

135

The COVID-19 crisis is plunging the global economy into a deep recession and micro, small and medium enterprises – which play a major role in emerging economies – are amongst the hardest hit. Trapped in economic stagnation due to large-scale lockdowns, millions of MSMEs have become the most vulnerable to COVID-19 within the private sector. Compared with large firms, small businesses have fewer resources and lower capacities to cope with the abrupt economic shocks economies are currently facing. With more than two-thirds of the global population employed by MSMEs, the unprecedented outbreak of the pandemic has vividly shown how tightly their activities are woven into the economic and social fabric of the world, as well as their critical role in social and economic resurgence.

The objective of the project is to develop and implement capacity-building tools for governments and micro, small and medium enterprises (MSMEs) to facilitate the resurgence and strengthen the resilience of MSMEs in developing countries and economies in transition. These capacity-building tools will seek to mitigate the economic and social impact of the global COVID-19 crisis and to facilitate the contribution of MSMEs to the SDGs implementation.

The project was designed based on the request for assistance for MSMEs from more than 50 Member States, including countries in Latin America and the Caribbean, Asia and the Pacific, Africa, Europe and the Arab regions, as well as intergovernmental demands and resolutions on COVID-19.

The project is jointly implemented by UNCTAD, DESA, ECA, ECE, ECLAC, ESCAP and ESCWA, and builds upon the comparative advantages of the participating agencies to provide immediate advice, capacity-building and support to governments and MSMEs during the ongoing global pandemic. The project is structured in five clusters that address the most critical areas of the MSME recovery. Broadly, the roles and lead entities for each of the clusters and workstreams are as presented in the following table:

Table 1. Project clusters and leads

Project cluster/workstream Lead agency Overall coordination UNCTAD Project cluster/workstream 1 Entrepreneurship and business skills promotion UNCTAD

Project cluster/workstream 2 Business facilitation/formalization

UNCTAD/DESA

Project cluster/workstream 3 Access to finance/financial literacy ESCAP

Project cluster/workstream 4 Access to technology and innovation UNECA

Project cluster/workstream 5 Access to markets UNECE

The beneficiary countries cover different geographical regions, as shown in Annex 2. The expected outcomes, indicators of achievement, and outputs are presented in the project results framework (Annex 3). The project was developed and implemented under three phases. A new set of outputs was designed or added at each of the three phases of the project.

Under the three-phase approach, the project budget was approved by phase. In 2021, when the phase 3 budget was discussed, the Development Account faced a funding gap. To bridge the gap, in November of the same year, the five joint projects were requested to reduce their proposed phase 3 budget by 1 million USD, which led to the curtailment of certain planned activities. For this project, the budget was reduced by $310,000.

Overall, a total of $4,490,500 was allocated under this project. Concretely, UNCTAD received $2,671,000, ECLAC received $134,000, ESCAP received $240,000, ECA received $467,000, ECE received $448,500, ESCWA received $370,000 and DESA received $160,000.

136

The project started its implementation in May 2020 and was scheduled to conclude on 31 March 2022, but received approval in February 2022 for an extension until 30 June 2022.

B. EVALUATION OBJECTIVES AND SCOPE

B1. EVALUATION PURPOSE, OBJECTIVES AND SCOPE The present evaluation will constitute a terminal evaluation of the Project. Terminal evaluations are mandatory for all DA-funded projects with a value above $1 million. The evaluation will be largely guided by the UN Development Account Project Evaluation Guidelines, issued in October 2019 and the evaluation policies of the implementing entities, in particular, UNCTAD, which leads the evaluation.

The main purpose of the evaluation will be to support accountability for results, and to enable learning.

This terminal evaluation of the project has the following specific objectives:

- Assess the results and establish the link between achievements and activities of the intervention;

- Assess the response delivery and external coordination167, including the extent of gender, human rights and disability mainstreaming; and

- Identify good practices and lessons learned from the project that could feed into and enhance the implementation of related interventions.

The primary intended users of the assessment are the management of the implementing entities. The evaluation will also provide accountability to project beneficiaries and member States. Furthermore, the evaluation will form a key input to the programme-level evaluation of the DA’s response to COVID-19 to be initiated by the CDPMO/DESA. The programme-level evaluation will entail: a synthesis of the terminal evaluations of five COVID-19 joint DA projects, including this project; a review of relevant 10th and 11th tranche DA projects; and a programme-level assessment. The primary audiences of the programme-level evaluation will include the DA Steering Committee, the DA-Programme Management Team (DA-PMT), and the management of the implementing entities. The results of the programme-level evaluation will also be presented to the General Assembly, through the biennial progress report on the implementation of the DA.

The evaluation will cover the duration of the project from May 2020 to 30 June 2022, covering all phases, clusters and activities.

B2. EVALUATION CRITERIA AND QUESTIONS The evaluation will assess the Project’s performance against the main criteria of relevance, efficiency, effectiveness, coherence, sustainability, gender, human rights and disability. In particular, the evaluation is expected to address a number of questions under the following criteria168:

Table 2: Evaluation criteria and tentative questions

167 The OIOS COVID-19 response evaluation protocol identifies the following three cross-cutting focus areas: 1) response delivery; 2) external coordination (or “Delivering as one”); and 3) business continuity. “Response delivery” is further defined as consisting of delivery of: 1) the existing mandate needed to implement previously mandated activities in the new environment created by the pandemic; and 2) the COVID-19-specific response (health and non-health) needed to address the pandemic specifically. See OIOS (October 2020), “COVID-19 Response Evaluation Protocol”, para 3-4. 168 The evaluation questions were developed as part of the “Proposed approach, scope and questions to the Evaluation of the United Nations Development Account’s Response to COVID-19”, which was jointly developed by the DA-PMT and select implementing entities in the spring of 2020. The document is designed to guide both the terminal evaluations of the five COVID-19 joint projects and the programme-level evaluation of the DA’s response to COVID-19, and is expected to be updated later in 2022 to reflect the confirmed approaches and timelines for the terminal evaluations of the five projects.

137

Relevance 1. To what extent was the project designed to target the new needs and priorities of participating countries as a result of COVID-19?

Relevance 2. To what extent was the project aligned with the COVID-19 socio-economic responses of the participating countries (e.g. COVID-19 Socio-Economic Response Plan)?

Efficiency 3. How well coordinated was the response among the entities implementing the joint project?

Efficiency 4. How did the three-phase budgeting and programming approaches impact the efficient delivery of the project?

Effectiveness 5. To what extent did the programme (Development Account) and project governance and management structures and processes enabled, or hindered, the effective implementation of the joint project and the achievement of its results?

Effectiveness 6. To what extent has the project contributed to the expected outcomes as enunciated in the project document?

Effectiveness 7. How did the response contribute to the participating country Governments’ responses to COVID-19, especially in the area of MSME resurgence?

Effectiveness 8. What innovative approach or tool, if any, did the response use, and what were the outcomes and lessons learned from its application?

Sustainability 9. What measures were adopted to ensure that the outcomes of the response would continue after the project ended?

Coherence 10. To what extent was the project complementary to, and coordinated with, other work undertaken by the implementing entities?

Coherence 11. To what extent has the project been coordinated with, and complementary to, the response of other UN entities (Secretariat and non-Secretariat) to COVID-19 in delivering socio-economic support to Member States?

Gender, human rights and disability

12. To what extent were gender, human rights and disability perspectives integrated into the design and implementation of the project? What results can be identified from these actions?

C. EVALUATION APPROACH AND METHODOLOGY The evaluation will be a transparent and participatory process involving the Project’s implementing entities and key stakeholders. It will be conducted based on gender and human rights principles and adhere to the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation.

The evaluation will apply a mixed-method design, including a combination of quantitative and qualitative analysis to inform findings.It is anticipated that travel of the evaluation team may take place in support of elaborating case study/ies, as well as to meet key project stakeholders in Geneva (UNCTAD and ECE). The selection of potential case study/ies and travel requirements will be developed as part of the inception report.

Following a preliminary documentation review and a limited number of inception meetings with the core project team, the Evaluation Team will develop an inception report for the evaluation, which will include the finalized overall scope and focus of the evaluation, evaluation questions and methodology, including

138

information on data sources and collection, sampling, key indicators, stakeholder mapping, selection of case study/ies, survey design, and the evaluation timeline.

The tentative methodology for the evaluation is presented in Table 4.

Table 3: Tentative methodology for the assessment

a) A desk review of Project documents, including documents/data related to:

o Project-level planning, implementation and results achievement, including but not limited to:

▪ Concept note, Phase 2 project proposal, and Phase 3 budget and outputs

▪ Progress report for Phases 1 and 2 (both financial and substantive/narrative report)

▪ Final report (both financial and substantive/narrative report)

▪ Meeting minutes, including the minutes of the bi-weekly/monthly DA network meetings

▪ Monitoring reports

▪ Information on non-DA resources, financial and in-kind, brought in by the participating entities

▪ Information on resources, financial and in-kind, contributed by partners/donors (including

information requested under the “supplementary funding” section in the progress reports,

which is often incomplete)

▪ Beneficiary/user feedback collected, including, but not limited to, workshop survey results,

user feedback on publications, advisory services, guidelines, methodology documents, etc.

▪ Requests for assistance/services received

▪ List of activities completed and details about each activity, including but not limited to:

- Agenda, participant lists (name, title, division/unit, organization, country, gender,

email address), report and any outcomes document, for each workshop/meeting

- Description of each advisory service, beneficiaries (including contact details of the

contact persons) and any outputs/deliverables produced

- List and description of tool(s), research papers, policy briefs, studies published and

information on how each product was disseminated and/or used, list of

recipients/users of the product (e.g., dissemination lists)

▪ Documentation related to broader projects or sub-projects of the participating entities of

which the Project or its component(s) has constituted an integral part or which are linked to

and/or build upon/succeed the work undertaken as part of the Project

▪ Documents and literature related to the Project context

▪ Relevant web and social media metrics related to the outputs of the project;

o Project strategic documents, including but not limited to:

▪ General Assembly's Resolution on Global Solidarity to fight the coronavirus disease 2019

(COVID-19) (A/RES/74/270);

▪ Secretary General's report on "Shared responsibility, global solidarity: Responding to the socio-

economic impacts of COVID-19";

▪ UN Framework for the immediate socio-economic response to COVID-19;

▪ 2021 Programme budget and mandate of implementing entities;

▪ COVID-19 Socio-Economic Response Plan of participating countries.

b) Questionnaires/surveys (in appropriate languages in addition to English) to relevant stakeholders in countries participating in a sample of project activities;

c) Telephone, online or in-person interviews with key stakeholders, including but not limited to:

139

o Project Coordination Team and project focal points of implementing entities

o DA-PMT

o DA focal points in participating entities

o Sample of UN Resident Coordinators/Country Teams, as appropriate

o Sample of key global partners

o Sample of country-level stakeholders (mainly stakeholders from key beneficiary countries)

d) Case Study/ies, which may include for example a detailed examination of a particular intervention, or of project activities at a regional or national level.

In addition to assessing the mainstreaming of gender, human rights and disability perspectives in the design, implementation and monitoring of the Project (evaluation question 12), the evaluation will integrate these perspectives in the management of the evaluation, data collection and analysis, as well as the development of the evaluation report. Gender balance will be given full consideration in the composition of the Evaluation Reference Group, elaborated in Section D1 (Evaluation management), and the Evaluation Team. Data collected and analyzed in the course of the evaluation will be disaggregated by gender to the extent possible and whenever appropriate, and the evaluation findings, conclusions and recommendations to be presented in the evaluation report will reflect a gender analysis.

The evaluation will be carried out according to the UNEG ethical principles and standards.169 The evaluators should demonstrate behavioural independence, impartiality, credibility, honesty, integrity and accountability in conducting the evaluation/assessment to avoid biasing the findings. The evaluators must also address in the design and conduct of the evaluation procedures to safeguard the rights and confidentiality of information providers. The Evaluation Manager will be responsible for ensuring that the Evaluation Team conducts the work assignments without any undue interference from those who were responsible for the implementation of the Project.

D. ORGANIZATION OF THE EVALUATION

D1. EVALUATION MANAGEMENT The independent final project evaluation will be managed/coordinated by UNCTAD’s Independent Evaluation Unit, with the support of an Evaluation Advisory Committee (EAC) that comprises a representative each of the evaluation units of the partner entities (DESA, ECA, ECE, ECLAC, ESCAP, ESCWA) and the Evaluation Officer with the CDPMO/DESA. The EAC primarily serves a quality assurance function and facilitates support to the Evaluation Team as necessary.

An Evaluation Reference Group (ERG) consisting of a representative from each UN partner entity (e.g., the DA Focal Point for each entity) and the DA-PMT will review and contribute inputs to key steps in this evaluation such as the TOR and draft final report.

Both the EAC and the ERG commit to submitting substantive comments on a timely basis, and comments will be invited on a ‘non-objection’ basis (no response = agree) so that the process is not delayed for an unnecessarily long time.

An independent Evaluation Team will be convoked to undertake this assignment. The Evaluation Team (ET) is responsible for conducting the evaluation, applying the methodology as appropriate and for producing the evaluation report. All team members, including the Team Leader, will participate in briefing and debriefing meetings, discussions, and will contribute to the evaluation with written inputs for the draft and final report. A selected number of the evaluation team members will participate in the mission travel(s) if applicable. The ET and the Evaluation Manager will agree on the outline of the report, in consultation with the EAC early in the evaluation process. The ET will develop its own evaluation tools and framework, within

169 UNEG (2020), Ethical Guidelines for Evaluation; UNEG (2008), Code of Conduct for Evaluation in the UN system

140

the available timeframe and resources. The team is fully responsible for its report, which may not reflect the views of any of the implementing entities of the project. The evaluation report is subject to quality control by the Evaluation Advisory Committee and clearance by the Evaluation Manager, as set out above.

The Team Leader guides and coordinates the team member(s) in their specific work, discusses their findings, conclusions and recommendations and prepares the draft and the final report, consolidating the inputs from the team member(s) with his/her own. The members of the evaluation team should possess a mix of evaluation skills and technical or sectoral/thematic knowledge relevant to the evaluation. In putting together the team, adequate linguistic, geographic and gender representation will also be key considerations.

The Evaluation Team will be provided full access to all project reports, documentation, and stakeholder lists and contact information. The Project Coordination Team are required to submit to the evaluation manager project documentation, including data and information residing with the other participating entities, in the last month of the project if possible, if not, immediately following the completion of the project, as well as support the evaluation process, including through facilitating the evaluators’ access to the project’s beneficiaries and other key stakeholders.

The roles and responsibilities in the evaluation process are described below:

Evaluation Manager (UNCTAD) will: ● Prepare the draft evaluation TOR and revise/finalize based on inputs received

● Prepare the TOR for each member of the Evaluation Team (Team Leader, Team Member and Expert(s))

● Recruit and manage the Evaluation Team

● Backstop the evaluation process, including supporting the development and administration of surveys,

support outreach of the evaluation team to project stakeholders, and access to secondary data listed in

Table 3.

● Oversee/provide quality assurance to the evaluation and the development of the evaluation report

● Facilitate the work of the Evaluation Advisory Committee and the Evaluation Reference Group

● Be responsible for clearance of the evaluation report

● Support the development of a management response to the evaluation report, including an implementation

plan

● Organise a virtual workshop on evaluation findings and lessons learned.

Evaluation Advisory Committee comprises a representative each of the evaluation units of the partner entities (ESCWA, ECA, ECE, ECLAC, ESCAP, DESA) and the Evaluation Officer with the CDPMO/DESA. The EAC primarily serves a support and quality assurance function. Specific responsibilities of the EAC include: ● Review and approve the evaluation TOR;

● Advise on the selection of the evaluation consultant(s) to ensure that the selection is based on the required

skills and qualifications;

● Support facilitating access from their respective entities to relevant project documentation and

stakeholders;

● Review and comment on the inception and evaluation reports; and

● Monitor and conduct periodic follow-ups on the implementation of evaluation recommendations

addressed to the parties within their entities.

Evaluation Reference Group, consisting of a representative from each UN partner entity (e.g., the DA Focal Point for each entity) and the DA-PMT,170 will review and contribute inputs to key steps in this

170 While multiple representatives of DA-PMT may attend meetings of the Global Reference Group, reflecting different roles held by each staff in relation to the Programme, DA-PMT will provide one consolidated written input as part of the review of the draft inception report and the draft evaluation report.

141

evaluation such as the TOR and draft final report. The ERG’s key function is to enhance the relevance, credibility and transparency of the evaluation process. Specific responsibilities include: ● To review the draft evaluation ToR and provide substantive feedback;

● To facilitate access from their respective entities to relevant project documentation and stakeholders;

● To review the draft evaluation report and provide substantive feedback, including coordinating feedback

from other sections, units and offices from headquarters and from the field to ensure quality and

completeness;

● To participate in the validation meeting of the final evaluation report;

● To play a key role in disseminating the findings of the evaluation and implementation of the management

response.

Project Coordination Team will: ● Facilitate the Evaluation Team’s access to relevant Project documentation and stakeholders, including

through:

o Collecting and compiling requested data and information from the participating entities, as

requested by the Evaluation Manager

o Providing an updated list of stakeholders, and facilitating access to the sample of stakeholders that

the Evaluation Team may wish to interview

o Facilitating the administration of questionnaires to workshop participants in the participating

countries

o Ensure the cooperation and contribution of the relevant staff of the implementing entities to the

evaluation process, as requested

● Lead the preparation of a response to the recommendations directed to the participating entities, including

an implementation plan

DA-PMT will: ● Participate in the Evaluation Reference Group

● Provide guidance on the allocation of the evaluation budget

● Organize a virtual meeting with DA focal points to discuss the key lessons from this evaluation as well as from other COVID-19 joint project evaluations and how to incorporate them in future programming, implementation, monitoring and evaluation of DA-funded projects.

D2. EVALUATION TIME FRAME The evaluation will be conducted from December 2022 to August 2023.

The evaluation process will involve five phases with the tentative timelines as below in Table 5 (the timelines may be adjusted should any exigencies arise):

Table 5: Evaluation phases and tentative timelines

Phase Timelines

1. Preparation

August 2022 – December 2022 ● Preparation and finalization of evaluation TOR ● Establishment of the Evaluation Reference Group ● Recruitment of the Evaluation Team ● Prepare package of documents required by the

Evaluation Team

2. Inception

December 2022 -March 2023

142

● Preliminary documentation review and preparation of inception report by the Evaluation Team, including development of data collection instruments (questionnaires/ surveys, interview guides)

● Brief visit to Geneva (3 days) to meet with UNCTAD Evaluation Unit and key project stakeholders (UNCTAD and ECE)

● Draft inception report due: 10 February 2023 ● Evaluation Manager review and inception report

revision by Evaluation Team: 13 - 22 February 2023 ● Reviews by Project Coordination Team, project focal

points of implementing entities, and Evaluation Advisory Committee (in parallel): 23 February – 7 March 2023

● Draft final inception report due: 14 March 2023 ● Final inception report approved: 20 March 2023

3. Data collection and analysis March- May 2023

● Desk review of remaining Project documents, including requesting additional documentation

● Online surveys of stakeholders ● Interviews with stakeholders ● Data analysis and triangulation

4. Report preparation and reviews May – July 2023

● Data analysis and triangulation ● First draft evaluation report due: 16 June 2023 ● Evaluation Manager review and report revision by

the Evaluation Team: 19 – 28 June 2023 ● Reviews by Project Coordination Team, project focal

points of implementing entities, Evaluation Advisory Committee and Evaluation Reference Group (in parallel): 29 June – 11 July 2023

● Revised draft evaluation report due: 18 July 2023 ● Final evaluation report with annexes: 25 July 2023

5. Dissemination and follow-up August 2023 and onwards

● Presentation to the Project Coordination Team, project teams of implementing entities and development and approval of a management response, including an implementation plan for recommendations

● Virtual workshop on evaluation findings, lessons learned and follow-up with the DA Focal Points: April 2023

D3. EVALUATION TEAM DELIVERABLES The Evaluation Team will be composed of a team of three consultants (evaluators), namely Team Leader, Team Member and a Gender and Human Rights (HRGE) Expert who also plays the role of Team Member. The two Team Members will report functionally to the Team Leader. The Team Leader will report to the Evaluation Manager. Each of the Evaluation Team has a set of deliverables as described below:

143

Deliverables for Team Leader and Team Member

o Initial review of key Project documents (preliminary document review)

o Preparation of an inception report with a finalized evaluation scope and focus, evaluation questions and methodology, including information on data sources, sampling and key indicators, stakeholder mapping/analysis, selection of case study/ies, as well as survey design

o Desk review of remaining Project documents

o Data collection and analysis based on the finalized methodology

o Preparation of an evidence matrix presenting a summary of evidence collected through each data collection method by evaluation question

o Development of a draft evaluation report, based on the template presented in Annex 1, for review by the Evaluation Manager, Project Coordination Team, project focal points of implementing entities, the EAC and the ERG

o Revision/finalization of the evaluation report, including all annexes, based on comments received

o Preparation of a 3-page summary of the evaluation report and a presentation (PPT) on key findings, conclusions and recommendations

o Presentation of evaluation report and discussions with relevant stakeholders such as Project Coordination Team, project teams of implementing entities, DA focal points of participating entities and DA-PMT.

Deliverables for Gender and Human Rights Expert/Team Member

o Initial review of key Project documents (preliminary document review), including identifying gender equality, human rights and disability inclusion dimensions and issues for consideration;

o Preparation of an inception report with a finalized evaluation scope and focus, evaluation questions and methodology, including information on data sources, sampling and key indicators, stakeholder mapping/analysis, selection of case study/ies, as well as survey design. Where applicable, gender equality, human rights and disability inclusion considerations will be integrated in the evaluation scope of analysis; evaluation criteria and questions design; methods and tools, and data analysis techniques;

o Desk review of remaining Project documents;

o Data collection and analysis based on the finalized methodology which would be gender sensitive;

o Preparation of an evidence matrix presenting a summary of evidence collected through each data collection method by evaluation question;

o Development of a draft evaluation report, based on the template presented in Annex 1 of the Terms of Reference for the evaluation, for review by the Evaluation Manager, Project Coordination Team, project focal points of implementation entities, the EAC and the ERG. The analysis of gender equality, human rights and disability inclusion dimensions and issues should be integrated in the report as an independent section and to the extent possible, these issues should be mainstreamed throughout the report, including in the evaluation findings, conclusions and recommendations.

o Revision/finalization of the evaluation report, including all annexes, based on comments received;

o Preparation of a 3-page summary of the evaluation report and a presentation (PPT) on key findings, conclusions and recommendations.

o Presentation of evaluation report and discussions with relevant stakeholders such as Project Coordination Team, project teams of implementing entities, DA focal points of participating entities and DA-PMT.

144

E. COMMUNICATIONS AND DISSEMINATION PLAN: The results from the evaluation including key lessons learned, best practices and recommendations will be shared widely with participating entities, partners and stakeholders, and member States. In particular, the following modes of communication could be used:

e) A workshop with all relevant stakeholders to present the key findings, recommendations and lessons learned. The evaluation report will be presented at a workshop attended by the implementing entities, the DA-PMT and other relevant stakeholders for discussion and validation. The implementing entities will be given the opportunity to present their management response, including an implementation plan for the recommendations;

f) A separate virtual meeting will be held with the DA focal points to discuss the key lessons from the evaluation as well as from other COVID-19 joint project evaluations and how to incorporate them in future programming, implementation, monitoring and evaluation of DA-funded projects and projects.

g) A copy of the final evaluation report will be published on UNCTAD’s website and the websites of the partner implementing entities, as appropriate; and

h) The key findings from the evaluation report will also form a key input to the programme-level evaluation of the DA’s response to COVID-19 to be initiated by the CDPMO/DESA.

i) Other communication briefs and products will be produced as appropriate.

  • Lists of figures, tables, and boxes
  • List of Acronyms and Abbreviations
  • Executive summary
  • 1. Introduction
  • 2. Description of the Project
    • 2.1 Background
    • 2.2 Project objectives and expected accomplishments
    • 2.3 Project strategies and key activities
    • 2.4 Beneficiaries and target countries
    • 2.5 Key partners and other key stakeholders
    • 2.6 Resources
    • 2.7 Link to the Sustainable Development Goals (SDGs)
    • 2.8 Innovative elements
  • 3. Evaluation objectives, scope, and questions
    • 3.1 Purpose and objectives
    • 3.2 Evaluation scope, criteria, and questions
  • 4. Methodology
  • 5. Findings
    • 5.1 Relevance
    • 5.2 Coherence
    • 5.3 Efficiency
    • 5.4 Effectiveness
    • 5.4 Sustainability
    • 5.5 Gender, Human Rights, and Leave No One Behind
  • 6. Conclusions
  • 7. Recommendations
  • Annexes
    • Annex 1: Project’s Theory of Change
    • Annex 2: Evaluation matrix
    • Annex 3: Data collection instruments
    • Annex 4: List of documents reviewed
    • Annex 5: List of individuals interviewed
    • Annex 6: Evaluation surveys
    • Annex 7: Evaluation Terms of Reference

EVAL_Trade_E293EvalReport_June2024

This evaluation assesses the relevance, effectiveness, efficiency and relevance of the UNECE Geneva E293 Project entitled “Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system”, (further project or E293)). Project was formulated and managed by Economic Cooperation and Trade Division.

Languages and translations
English

1

Evaluation Project E 293: Strengthening the capacity of Central Asian

countries to implement trade facilitation measures and better integrate into the international rules-based trading system

Report completed in June 2024 Evaluation conducted by: DUSAN ZUPKA

2

ACKNOWLEDGEMENTS: The evaluator extends his gratitude to those stakeholders who contributed with their insights during the evaluation of E293 project. Their expertise and dedication have been instrumental in shaping this report. I would like to express my gratitude to all persons met, interviewed, and who contributed to this evaluation through electronic survey, particularly the E293 project manager in UNECE Economic Cooperation and Trade Division for cooperation and support during the entire evaluation. This report was commissioned by UNECE. The findings, conclusions and recommendations of this report are those of the external evaluator and do not necessarily reflect the views of the UNECE.

3

Table of Contents List of acronyms and abbreviations ................................................................................................ 4

Executive summary ......................................................................................................................... 5

I. Introduction ............................................................................................................................... 11

1. Description of the project ...................................................................................................... 13

2. Evaluation objectives, scope and questions .......................................................................... 19

3. Evaluation methodology ....................................................................................................... 21

II. Evaluation findings .................................................................................................................. 25

1. Relevance .......................................................................................................................... 25

2. Effectiveness ..................................................................................................................... 32

3. Efficiency .......................................................................................................................... 38

4. Sustainability ..................................................................................................................... 45

III. Conclusions and recommendations......................................................................................... 51

1. Conclusions ........................................................................................................................... 51

2. Recommendations ................................................................................................................. 53

Annexes......................................................................................................................................... 54

4

ADB Asian Development Bank ECTD Economic Cooperation and Trade Division EQ Evaluation Question ESCAP Economic and Social Commission for Asia/Pacific ILO International Labour Organization ITC International Trade Center NGO Non-Governmental Organization NTFB National Trade Facilitation Body OECD/DAC Organization for Economic Cooperation and

Development/Development Assistance Committee PMU Programme Management Unit PROJECT

UNECE Project E293: Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system

SDGs Sustainable Development Goals SPECA United Nations Special Programme for the Economies

of Central Asia TCPF Technical Cooperation Project Form TFA Trade Facilitation Agreement TFIG Trade Facilitation Implementation Guide TOR Terms of Reference UN/CEFACT United Nations Centre for Trade Facilitation and

Electronic Business UNDA United Nations Development Account UNECE United Nations Economic Commission for Europe WGT Working Group on Trade WTO World Trade Organization

5

This evaluation assesses the relevance, effectiveness, efficiency and relevance of the UNECE Geneva E293 Project entitled “Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system”, (further project or E293)). Project was formulated and managed by Economic Cooperation and Trade Division. This division supports UNECE member States in designing and establishing institutions, policies, processes and initiatives with a strong role for good governance to build innovative, competitive, and inclusive societies to progress towards the goals set out in Agenda 2030 in the areas of innovation policy, infrastructure investment (e.g. public- private partnerships) and trade (including trade facilitation).1 The report looks at the implementation approach and the results of E 293 during the period May 2018 – December 2023. The evaluation has been commissioned by the United Nations Economic Commission for Europe (UNECE) to the independent evaluator in January 2024. The evaluation report includes evaluation findings, conclusions and recommendations. It aims to promote learning, ensure accountability and inform decision-making by assessing if and how far trade facilitation measures have been integrated in the target countries and whether the intended results have been achieved. It also identifies the areas where it has been effective in adding value, and provides recommendations for the future implementation of similar projects. Brief overview of the project The five Central Asian countries (Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, Uzbekistan) plus Afghanistan and Azerbaijan participate in the UN Special Programme for the Economies of Central Asia (SPECA). When the Project was designed these countries were among the last in the world to accede to the international rules-based trading system coordinated by the World Trade Organization (WTO). It was also recognized that the WTO Trade Facilitation Agreement (TFA) would make a significant contribution to facilitating trade in the region and its integration with the economies of Europe and Asia. The objective of the project was to strengthen national capacities of the beneficiary countries (the Central Asian countries and Azerbaijan) to implement trade facilitation measures and better integrate into the international rules-based trading system. The technical cooperation project form (TCPF) established two expected results for achieving the main objective. These expected accomplishments were defined as follows:

1 https://unece.org/economic-cooperation-and-integration; Trade | UNECE; https://unece.org/sites/default/files/2023- 05/ECTD-Booklet_Final.pdf https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fun.mdrtor.com%2Ftrack%2Flink%2Fkey%2 F84209-3716-44500-160-3414- 261949%2Fsignature%2F72f21e1a418734f8c07a5766a7cc4929%2Fuserid%2Fc84869d9600ec122806763a53d5207 a2&data=05%7C02%7Celisabeth.tuerk%40un.org%7C579f1ec24a6d4439a47608dc84741360%7C0f9e35db544f4f6 0bdcc5ea416e6dc70%7C0%7C0%7C638530881287500144%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj AwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=KAk7JHj63DI afjYXzmBRDCRTHeI0aPTA%2Bvb1%2FuKPtyU%3D&reserved=0;

6

1. Enhanced capacities of policymakers and experts in National Trade Facilitation

Committees (NTFCs), regulatory agencies and other relevant stakeholders, to develop and implement Trade Facilitation (TF) policies and measures, notably in the area of WTO Trade Facilitation Agreement (TFA) Articles 10.1; 10.3; 10.4; and 23.2, in which UNECE and its subsidiary body UN/CEFACT have a comparative advantage.

2. Improved capacities of policymakers and experts to measure progress in the implementation of TF, and to achieve internal and cross-border policy coherence for the simplification and harmonization of trade procedures.

Evaluation purpose, objectives, scope, and intended users/audiences The broad purpose of this evaluation is to assess the extent to which the objectives of the UNECE project E293 “Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system” were achieved. The report looks at the extent of the relevance, effectiveness, efficiency and sustainability achieved by the project implementation activities in strengthening capacities of the target countries to implement trade facilitation measures and better integrate into the international rules-based trading system.2 The evaluation also assessed any impacts the project may have had on progressing human rights, gender equality, disability inclusion, climate change and disaster risk reduction in the context of this engagement. The evaluation finally looked at the activities repurposed to address the impact of the COVID-19 crisis, and assessed, where relevant, UNECE’s COVID-19 early response through this project. The evaluation process included activities needed for conducting fair and independent assessment of if, and how were the objectives, expected accomplishments, planned activities and estimated costs delivered/respected as established in the UNECE Technical Cooperation Project Form. The evaluation was initiated in January 2024 and concluded in April 2024. Based upon the Terms of Reference, the evaluation covers the full implementation time of the project, from January 2018 to December 2023 and its impact in targeted UNECE member States, in particular Azerbaijan, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan. The results of the evaluation will be used in the planning and implementation of future activities of the UNECE Economic Cooperation and Integration Subprogramme. Findings, conclusions and recommendations of this evaluation will be used, when possible, to: • Improve direct project’s follow up actions, implementation of products by project beneficiaries and dissemination of the knowledge created through the project. • Assess the gaps and further needs of beneficiary countries in the area of this project. • Formulate tailored capacity building projects to strengthen the national capacity in enhancing trade facilitation regulations in beneficiary countries.

2 Evaluation TOR

7

The intended users/audience of this report is the UNECE secretariat and member States, in particular government representatives and other stakeholders from beneficiary countries with active engagement in trade facilitation policy making and practical application, private sector, development organizations and practitioners in trade facilitation. Evaluation methodology The evaluation of E293 project is structured around the four OECD/DAC evaluation criteria (relevance, effectiveness, efficiency, and sustainability). In summarizing the findings for each of the OECD/DAC evaluation criteria, the evaluation links them to UNECE’s Economic Cooperation and Trade Division normative machinery3 and its objectives as well as a strategy set out in the UNECE Proposed programme budget for 20234. The construction of the evaluation methodology was guided by the TORs aiming at fair assessment of the degree of achievements of E293 overall objective and its expected results. The evaluator used a mixed-methods design that uses a variety of evaluative methods and techniques to address the key evaluation questions set out in the TOR. The evaluation methodology was intended to be flexible so that it could be adjusted to take into consideration frequent changes of project management responsibilities as well as the different needs of recipient countries under the programme’s umbrella and better understand how the project has adapted to these different circumstances. In other words, the interview protocol was built on the same building blocks—it means the key evaluation questions—to allow for data collected to be compared during its analysis but while using the tool, it was very specifically adapted to the context and type of respondent. Furthermore, the evaluation was guided by normative guidelines and principles based on good practices in managing, conducting and using evaluations. The evaluation process and methodological approach followed the principles set forth in the UNECE Evaluation Policy.5 Moreover, it has been performed in line with the Norms and Standards for Evaluation and respecting the Ethical Guidelines for Evaluation published by the United Nations Evaluation Group (UNEG).6 The inputs for the evaluation report were collected by several tools such as:

- Desk research of existing trade facilitation documentation, literature and data on the Project.

- Electronic survey, designed by the evaluator, targeting key stakeholders and beneficiaries.

3 https://unece.org/sites/default/files/2023-05/ECTD-Booklet_Final.pdf 4 https://unece.org/sites/default/files/2022-05/PPB%202023_Sect%2020_ECE.pdf 5 ECE Evaluation Policy, adopted by ECE EXCOM in December 2021: https://unece.org/sites/default/files/2021 6 https://www.unevaluation.org/document/detail/2866

8

- Online interviews of diverse stakeholders and beneficiaries with the public and private sectors, international organizations, national authorities and academia.

Summary of key findings 1. Evaluation criteria concerning the relevance of E293 were fully achieved in relation to its

responsiveness to beneficiary countries needs and alignment with the program of work of the UNECE and partially achieved in relation to its alignment with global/regional priorities, Sustainable Development Goals and, consideration of gender, human rights, disability perspective and climate change integration into the design and implementation of the project.

2. Evaluation criteria concerning the effectiveness of E293 implementation were fully achieved in relation to alignment of the project design with the needs of beneficiary countries and harmonization and coherence of activities with those of other partners operating within the same context and, partially achieved in relation to the extent of implementation of the planned activities required for the achievement of the project objective/expected results and overcoming of the challenges/obstacles to achieving expected results.

3. Evaluation criteria concerning the efficiency of E293 implementation were fully achieved in relation to adequacy of the project resources for attaining its planned results and, partially achieved in relation to the degree of achievement of the planned results on time, organization of all activities efficiently and extent of efficiency in using the resources economically.

4. Evaluation criteria concerning the sustainability of E293 results were fully achieved concerning measures adopted to ensure that project outcomes would continue after the project ended as we as regarding extent to which the partners and beneficiary countries “own” the outcomes of the project and the likelihood of the stakeholders’ engagement continuation, scaling up, replication and institutional strengthening.

Summary of key conclusions

1. Overall, the project responded to beneficiary countries needs as well as to the expectation of trade facilitation stakeholders in Central Asia. It was well aligned with development priorities of recipient countries’ governments and other stakeholders’ needs and helped support these countries to support their international obligations in contributing to several SDGs.

2. The project was fully aligned with the program of work of UNECE. Its approach and main target groups were in line with the UNECE mandate, mission and goals.

3. Key aspects of the project activities’ consistency with the sustainable development goals were well explored, while gender, human rights and disability perspectives have barely been touched. The evaluation did not note any evidence concerning the integration of gender, human rights and disability perspectives in the project document.

4. Several shortcomings linked to the interrupted continuity of the project affected the evaluation’s ability to fully assess the effectiveness and efficiency of the project. Another

9

shortcoming was the absence of a framework enabling a structured analysis of progress with respect to timing of delivery of expected activities and outputs.

5. The project was consistent with the interventions of other actors in the same areas, ensuring complementarity and coordination of the activities implemented. Regular dialogue with majority of the agencies supporting trade facilitation enhancement in Central Asia during the SPECA Working Group on Trade meetings and various capacity building seminars facilitated harmonization of efforts in trade facilitation with other partners operating in the same context.

6. The limited strategy for overcoming the obstacles/challenges that the project had to face resulted in large delays in the implementation of the planned activities and an extension of the project’s duration from two to six years.

7. In practical terms, the confirmation of sustainability was manifested in the improvement of the legal basis for trade facilitation in several Central Asian countries in the last 6 years (see example from Tajikistan below), as well as in the commitment of these countries in the preparation of national trade facilitation road maps (completed in Tajikistan and Kyrgyzstan and work in progress in Uzbekistan) aligned with the SPECA regional trade facilitation strategy and road map and the creation of national trade facilitation committees (Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan).

Summary of key recommendations

1. UNECE, through its three core functions – development of norms, standards and legal instruments; hosting of a convening platform; and technical cooperation across a number of relevant sectors - to continue trade facilitation related assistance to SPECA participating States, in particular with regards to standards and best practice recommendations for trade facilitation and electronic businesses to support Governments efforts to fulfill their obligations stemming from the WTO Trade Facilitation Agreement and/or WTO accession negotiations.

2. In the future, it is necessary to gradually move focus of UNECE trade facilitation capacity building activities from SPECA Working Group on Trade centered activities to training workshops and hands-on training at the national level in five Central Asia countries including all line ministries involved in export/import activities.

3. Future trade facilitation activities should include a more robust discussion/awareness raising concerning trade impacts on sustainable development, such as SDG 5 (gender equality) or SDG 13 (climate action), etc., to better integrate considerations of cross- cutting issues such as gender, human rights, disability perspectives, climate change. The COVID-19 crisis has made it more difficult for women to participate in economic activities and trade. Focusing on gender sensitive challenges should be further supported and type of assistance broadened.

4. In any scenario for future UNECE support to its member States in trade facilitation, UNECE internal management should be strengthened to ensure sound project management from

10

planning through delivery to reporting. The future projects’ planning should therefore explicitly consider this vital function and specify how this is to be ensured.

11

This report constitutes the final deliverable of the evaluation of the UNECE Geneva E293 Project entitled “Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system”. It presents results based on an in-depth document analysis (focused on the Evaluation Questions), online interviews of stakeholders and results of the Electronic Survey covering the entire implementation period 2018-2023.

Trade facilitation has been widely recognized as a key factor in economic development policy, due to the realization that it can generate major economic benefits in terms of competitiveness and efficiency at a relatively low cost. Trade facilitation can greatly enhance the integration of developing and transition economies into the global economy. Developments in trade facilitation at the WTO and other international fora have brought this subject to the highest level of political decision-making.7

The WTO Trade Facilitation Agreement (TFA) has become, upon entry into force in 2017, a major driver for the implementation of trade facilitation instruments developed and supported by UNECE and other international organizations. With trade facilitation on national political agendas, many countries are now faced with the challenge of translating this broad concept into specific implementation strategies that can achieve the expected results and economic benefits. The task is even more challenging in landlocked regions such as Central Asia. Trade facilitation in Central Asia has a potential to underpin supply chains, reducing time and cost of moving goods among the Central Asian countries and around the world and promote more inclusive trade.

UNECE is an international organization with experience and capacity to support implementation of trade facilitation measures. It is particularly well placed to support the implementation of some of the key measures in the WTO TFA. Working through its subsidiary body, the United Nations Centre for Trade Facilitation and Electronic Business (UN/CEFACT), it is one of the bodies producing standards and best practice recommendations in trade facilitation and electronic business globally. It has produced close to 40 trade facilitation recommendations on such topics as trade document formats and procedures, codes for trade and transport, and the implementation of Single Window systems for export and import clearance. UN/CEFACT also develops relevant standards such as the only global standard for Electronic Data Interchange UN/EDIFACT, XML schemas, core components, and others. These tools are of primary importance for allowing SMEs and economies in transition to benefit from trade facilitation. The standards, best-practice recommendations and other tools offered by UNECE and UN/CEFACT support the implementation of a number of measures in the TFA.

7 E293 project concept note

12

Background to the evaluation, including the reason for the evaluation and the time frame of the evaluation Established in 1947 by the United Nations Economic and Social Council (ECOSOC) and based in Geneva, Switzerland, the United Nations Economic Commission for Europe (UNECE) is one of the United Nation’s five Regional Economic Commissions. UNECE has 56 member States in Europe, North America and Asia. UNECE promotes pan-European economic integration in line with the UN’s Sustainable Development Goals (SDGs) and Agenda 2030. One of UNECE’s six divisions, the Economic Cooperation and Trade Division (ECTD) assists UNECE member States to better integrate into the world economy and to harness trade (including trade facilitation), innovation and infrastructure financing and investment for the sustainable development of the UNECE region. As already mentioned above, UNECE as one of the international organizations with experience and capacity to support implementation of trade facilitation measures by providing know-how, capacity building support and preparing relevant publications and guideline. According to OECD, “Trade facilitation refers to a specific set of measures that aim to streamline and simplify the technical and legal procedures for goods entering or leaving a country to be traded internationally. The concept covers the full spectrum of border procedures, from the electronic exchange od data about shipments, to the simplifications and harmonization of trade documents, to the possibility of appealing administrative decisions by border agencies8”. The E293 project started in 2018 and it was formally closed in December 2023 (after multiple no-cost extension). The evaluation of the project was commissioned by the UNECE which constitutes a customary approach for all international cooperation activities implemented by this organization with reference to its Evaluation Policy9.. The evaluation activities were initiated in January 2024 and completed in April 2024 by delivering the independent evaluation report by evaluator to UNECE management. In accordance with the evaluation TOR, the results of the evaluation will be used in the planning and implementation of future activities of the UNECE Economic Cooperation and Integration Subprogramme. Findings of this evaluation will be used, when possible, to:

● Improve direct project’s follow up actions, implementation of products by project beneficiaries and dissemination of the knowledge created through the project in recipient countries and beyond.

● Assess the gaps and further needs of the Central Asian countries in the area of the implementation of trade facilitation measures and better integration into the international rules-based trading system.

8 Source: OECD 2023 9 https://unece.org/sites/default/files/2021- 12/Item%2010_ECE_EX_2021_35_Rev1_Evaluation%20Policy_as%20adopted.pdf

13

● Formulate tailored capacity building projects to strengthen the national capacity in enhancing innovation in the project focus area.

Purpose and objectives of the evaluation, and the primary users/audiences According to the Terms of Reference (TOR), the overall objective and purpose of the evaluation is to assess the extent to which the objectives of the UNECE project E293 were achieved. The evaluation assesses the degree of achievements of the relevance, effectiveness, efficiency and sustainability of the project in strengthening capacities of its target/recipient countries to implement trade facilitation measures and better integrate into the international rules-based trading system. Attention was also paid to consideration of impacts the project may have on progressing human rights, gender equality, disability inclusion and climate change in the context of the UNECE engagement in the context of the E293 project. As envisaged by TOR, the evaluation also looked at the activities repurposed to address the impact of the COVID-19 crisis, and assessed, where relevant, UNECE’s COVID-19 early response through this project. Expected primary users/audience of the evaluation findings, conclusions and recommendations are the E293 project team and the ECTD management as well as UNECE as a whole (including top management). Financing donors and partners of the Programme are also foreseen to be key users of the report, in particular the donor (Russian Federation). These are not only important due to their financial and in-kind support but also important stakeholders in building consensus on the need for further enhancing of trade facilitation measures and state-of-art approaches.

Central Asia is a fast-growing region endowed with rich natural resources. While social and economic fragility are common challenges, the Central Asian countries also share trade and investment opportunities which have the potential to yield social and economic progress. The five Central Asian countries (Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, Uzbekistan) plus Afghanistan and Azerbaijan participate in the UN Special Programme for the Economies of Central Asia (SPECA). When the E293 project was designed these countries were among the last in the world to accede to the international rules-based trading system coordinated by the World Trade Organization (WTO). It was recognized that the WTO Trade Facilitation Agreement (TFA) would make a significant contribution to facilitating trade in the region and its integration with the economies of Europe and Asia.

14

The objective of the project was to strengthen national capacities of the beneficiary countries (the Central Asian countries and Azerbaijan) to implement trade facilitation measures and better integrate into the international rules-based trading system.

As already mentioned above the overall objective of the project was to strengthen national capacities of the beneficiary countries (the Central Asian countries and Azerbaijan) to implement trade facilitation measures and better integrate into the international rules-based trading system. The technical cooperation project form (TCPF) established two expected outcomes for achieving the main objective. These expected accomplishments/outcomes were defined as follows:

1. Enhanced capacities of policymakers and experts in National Trade Facilitation Committees (NTFCs), regulatory agencies and other relevant stakeholders, to develop and implement Trade Facilitation (TF) policies and measures, notably in the area of WTO Trade Facilitation Agreement (TFA) Articles 10.1; 10.3; 10.4; and 23.2.

2. Improved capacities of policymakers and experts to measure progress in the implementation of TF, and to achieve internal and cross-border policy coherence for the simplification and harmonization of trade procedures.

The following indicators of achievement were formulated in the E 293 concept note:

Indicator of achievement IA1.1. - At least two countries established and/or maintained national trade facilitation bodies;

Indicator of achievement IA1.2. - At least four national policy documents (e.g. decree on establishing a National Trade Facilitation Committee, a strategy for the implementation of international trade facilitation standards or other document) promoting trade facilitation formulated;

Indicator of achievement IA2. - One pilot benchmarking system for measuring progress in trade facilitation developed. The project targeted low and middle-income countries with economies in transition. These countries participate in the United Nations Special Programme for the Economies of Central Asia (UN SPECA), promoting implementation of trade facilitation measures and better integration into the international rules-based trading system. The UNECE has been aiming to advance the capacities of the member states in trade facilitation for many years. Therefore, it is well situated to support the implementation of some of the key measures contained in the WTO TFA. Working through its subsidiary body, the United Nations Centre for Trade Facilitation and Electronic Business (UN/CEFACT), UNECE has produced a long list of trade facilitation recommendations on such topics as trade document formats and procedures, codes for trade and transport, and the implementation of Single Window systems for export and import procedures

15

Project strategy The strategy of the project was to focus on the four areas covered by the WTO Trade Facilitation Agreement, in which UNECE and UN/CEFACT have comparative advantage: 1. National Trade Facilitation Committees (TFA Art.22.2); 2. Streamlining documents and documentary procedures (including digitalization) (TFA Art. 10.1); 3. International Standards for Trade Facilitation (TFA Art. 10.3) and, 4. Single Window (TFA Art. 10.6). The overall concept was to prepare training materials on these subjects and use them to help the Central Asian countries’ transition economies to engage meaningfully in trade facilitation activities. The project was meant to support: (1) the SPECA Working Group on Trade, and (2) the finalization and implementation of the SPECA Trade Facilitation Strategy10 and Roadmap11 to implement this Strategy, developed in the SPECA WG on Trade and adopted by the SPECA Governing Council on 21 November 2019 in Ashgabat. Project key activities The UNECE Executive Committee (EXCOM), which is entrusted with the approval of extrabudgetary contributions, approved the Technical Cooperation Project Form (TCPF) related to E293 project on 26 September 2017. TCPF established the following activities for achieving the final goal of the project and its expected outcomes: A1.1. Adaptation and development of training materials and modules for national TF courses to cover the relevant target areas in the TFA for the implementation of WTO TFA provisions related to Articles 10.1 on formalities and documentary requirements; 10.3 on use of international standards; 10.4 on single window; and 23.2 on national trade facilitation committee (NTFC); A1.2. Organization of wo capacity-building seminars for members of NTFBs, regulatory agencies, and other relevant stakeholders on the development of national trade facilitation roadmaps, including for the implementation of the WTO targeted TFA measures;

10(https://unece.org/fileadmin/DAM/SPECA/documents/gc/session14/SPECA_Trade_Facilitation_Strategy_English. pdf 11(https://unece.org/fileadmin/DAM/SPECA/documents/gc/session14/Roadmap_for_Implementation_of_the_SPE CA_TF_Strategy_English.pdf)

16

A1.3. Organization of one sub-regional SPECA capacity building and experience sharing conference to strengthen cooperation on the implementation of TF measures and supporting international standards, including on the aspects of Single Window interoperability; A1.4. Translation of the training materials developed under A1.1 into Russian; A2. l. Development of training materials on benchmarking mechanisms to measure progress and achieve policy coherence in the implementation of TF measures, based on the UN/CEFACT Trade and Transport Facilitation Monitoring Methodology (TTFMM) in the beneficiary countries; A2.2. Organization of two capacity-building seminars for NTFBs on the development of national benchmark mechanisms to measure progress and achieve policy coherence in TF based on the UN/CEFACT TTFMM methodology; A2.3. Translation of learning materials and training courses into Russian; A2.4. Update and maintenance of the UNECE Trade Facilitation Implementation Guide (TFIG) by incorporating project results and lessons learned and the new learning materials, including through the development of new itineraries providing a step-by-step approach to the implementation of TF measures in the project target areas, and the drafting of case stories; A2.5. Organize a concluding Conference (in the context of the UN/CEFACT Forum) on regulatory cooperation for TF to disseminate project’s outcomes and share lessons learned from project target countries.

The Technical Cooperation Project Form determined as the project target group the senior trade policy makers, national experts in trade policy and trade facilitation, representatives of the business community, representatives of academia. The same documents identified as the beneficiary countries Azerbaijan, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan.

All these countries participate in the UN Special Programme for the Economies of Central Asia (SPECA). The subregion lacks a tangible sense of regionalism. When the project document was drafted the Central Asian countries were among the last in the world to accede to the international rules-based trading system (WTO). The need to implement the WTO Trade Facilitation Agreement (TFA), which was entering into force, was a significant challenge to the Central Asian countries. The trade policy and capacity-building events, as well as the TFA readiness assessments carried out by UNECE with various partners in 2015-2016 have shown the demand and justification for a project such as this in the target countries.

The project stakeholders targeted during the evaluation were the trade facilitation policy makers, government officials, UNECE, other international organizations involved in trade facilitation,

17

private sector and practitioners in trade facilitation, as shown in Table 1 (based on information provided by the project manager). The project was intended to support: (1) the SPECA Working Group on Trade12, and (2) the finalization and implementation of the SPECA Trade Facilitation Strategy13, adopted in Ashgabat, Turkmenistan during the 14th session of the SPECA Governing Council (3) the drafting of a Roadmap to implement the SPECA Trade Facilitation Strategy14. Logically, the key partners/stakeholders were the representative of the five targeted countries participating in the SPECA Working Group on Trade activities. The project’s concept note and the technical cooperation project do not provide the intended stakeholder list and the stakeholder analysis. In the absence of an overview/analysis of different institutions that would have been affected by activities of the project and how these participated and/or benefited from the project, the table 1 contents are based on the discussions/interviews with the UNECE staff responsible for the project management and the documentary review undertaken by the evaluator.

Table 1: Stakeholders mapping

Stakeholder Role in the project Level of influence on the project

Extent to which affected by the

project Governments

Government of Azerbaijan, Kazakhstan, Kirgizstan, Tajikistan,

Turkmenistan Uzbekistan,

Main stakeholders Implementation of TF

approach Monitoring progress

High High

UN and other International Organizations UNECE, ITC, WTO, UNCTAD, UNDP,

UNESCAP

Stakeholder Project leadership, management and

monitoring, technical support to SPECA

High High

SPECA WG on Trade Advice on TF approach/development

High High

12 https://unece.org/DAM/SPECA/documents/gc/session14/VIII._ToR_SPECA_WG_on_Trade_English.pdf 13 https://unece.org/fileadmin/DAM/SPECA/documents/gc/session14/SPECA_Trade_Facilitation_Strategy_ English.pdf 14 https://unece.org/fileadmin/DAM/SPECA/documents/gc/session14/Roadmap_for_Implementation_of_ the_SPECA_TF_Strategy_English.pdf

18

Stakeholder Role in the project Level of influence on the project

Extent to which affected by the

project of TF strategy and

best practices sharing GIZ, USAID Advice on TF

approach and best practices sharing

Medium Low

Private sector Private sector: Trade-

facilitate.com Advice and sharing

lessons learned in TF Medium Low

The project management and implementation counted with the following resources:

1/Human resources which according the TCPF consisted from 4 months/year of one regular budget staff time, supported by the ECTD G staff as required.

2/Training and capacity building know-how.

3/Partnership and collaboration with the stakeholders described above.

4/Monitoring and reporting mechanism supported by the project manager.

5/Communication and outreach including awareness raising, engaging stakeholders, workshops.

6/Financial resources as described in the following table:

Table 2 Project under evaluation, its duration and financial resources

Title of the intervention

“Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system”

Budget of the intervention

USD 352 000

Project Number

E 293

Dates of the intervention

Start date: January 2018 End date: December 2023

19

Trade facilitation activities play an important role in advancing several of the United Nations Sustainable Development Goals (SDGs) by promoting economic growth, innovation, and fostering partnership. Here's how trade facilitation activities are closely linked to various SDGs: SDG 8: Decent Work and Economic Growth: Trade facilitation promotes economic growth by making trade processes more efficient and less costly. This, in turn, creates job opportunities, enhances productivity, and fosters entrepreneurship, thereby supporting decent work and sustainable economic growth. SDG 9: Industry, Innovation, and Infrastructure: Trade facilitation activities contribute to fostering innovation. By improving trade logistics, enhancing connectivity, and promoting technology adoption, trade facilitation supports the development of modern infrastructure. SDG 17: Partnerships for the Goals: Trade facilitation requires collaboration and partnerships among governments, international organizations, the private sector, and civil society. By fostering cooperation and coordination among stakeholders, trade facilitation contributes to achieving the broader SDGs through collective action and shared responsibility.

Purpose and objectives The evaluation assessed the work of the UNECE project E293 “Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system” and extent to which the objectives were achieved. To achieve its objective the evaluation assessed how the four standard DAC criteria, relevance, efficiency, effectiveness and sustainability were reflected in E293 activities.

In addition, the evaluation measured impacts that the project may have had on progressing gender equality and women empowerment in the context of the project implementation. The gender perspective was be addressed through assessment of the cooperation with women policy makers, representatives of academia and national experts in the area of trade in the target countries. Finally, it looked at the activities repurposed to address the impact of the COVID-19 crisis, and assessed, where relevant, UNECE’s COVID-19 early response action through this project. The specific objectives of the exercise were:

 Determine as systematically and objectively as possible the relevance, effectiveness, efficiency and sustainability of the project results in light of its objective and expected accomplishments.

 Assess how the project activities contributed to gender equality and women’ s empowerment and make recommendations on how these considerations can be better addressed in future activities supporting trade facilitation measures.

20

 Identify key findings and conclusions from the project and formulate action-oriented, forward-looking recommendations addressed to UNECE for improving future interventions in trade facilitation cooperation projects.

 It is expected that the evaluation of the E 293 Project would contribute to trade facilitation learning and capacity strengthening in UNECE and behind.

The results of the evaluation will be used in the planning and implementation of future activities of the UNECE Economic Cooperation and Integration Subprogramme. Findings of this evaluation will be used, when possible, to:

● Improve direct project’s follow up actions, implementation of products by project

beneficiaries and dissemination of the knowledge created through the project. ● Assess the gaps and further needs of countries in the area of trade facilitation. ● Formulate tailored capacity building projects to strengthen the national capacity in

enhancing innovation in the field of trade facilitation. The results of the evaluation will be reported to the UN/CEFACT as well as to the UNECE Executive Committee if required. Evaluation scope, criteria and questions The temporal scope of the evaluation includes the entire UNECE E293 project activities in the period 2018-2023, while the geographic scope looks at the extent to which the project has managed to strengthen the capacity of Central Asian countries to implement effective trade facilitation measures. The specific aspects evaluated were: - The materialisation of the expected results and their facilitating/hampering factors. - The contribution to SDGs (specifically to SDG 17) and the mainstreaming of cross-cutting issues. The matching of the needs of the regional (SPECA) and national partners. - The governing mechanism of the Intervention including the effectiveness of UNECE management Attention was also paid to the following categories of project results:

- Project design and its relevance for the target countries priorities - Results framework/Log frame alignment with existing gaps in trade facilitation practices - Achievement of planned results by reviewing the log frame indicators against results - Project implementation process including management arrangements, work planning,

monitoring and reporting - Potential risks to sustainability

The project stakeholders targeted during the evaluation were trade facilitation policy makers, government officials, private sector and practitioners in trade facilitation, as shown in Table 2 above in the text. The Project Concept Note as well as the Technical Cooperation Project Form contain neither a stakeholder list nor a stakeholder analysis. In the absence of an overview of

21

different institutions that would have been affected by activities of the project and how these participate and/or benefit from the project, the table 2 contents is based on the initial discussions with the project management. The evaluation was guided by the planned objectives of the project, its expected accomplishments, planned activities and estimated costs established in the TCPF. It assessed the results achieved by the project by application of the following evaluation criteria: relevance, effectiveness, efficiency and sustainability of all activities implemented during the project duration (2018-2023). The evaluation TORs, evaluation questions and evaluation matrix are attached in Annexes.

The evaluation methodology was constructed in accordance with the TOR. It was established in line with the UNECE Evaluation Policy approved by the UNECE Executive Committee on 16 December 2021,15 completing the administrative instruction for the evaluation in the United Nations Secretariat (ST/AI/2021/3). The methodology was also aligned with the Organization for Economic Co-operation and Development Assistance Committee criteria for international development evaluations.16 The evaluation was carried through the following 3-phased process: Inception, Desk/Research, and Synthesis, which are summarised below: Inception phase This phase consisted of online and face-to-face kick-off meetings with the project management staff, followed by initial documents/data collection/ review and background analysis. Desk/Research phase This phase consisted of an in-depth document analysis followed by semi-structured interviews with selected key stakeholders. This exercise included the identification of information gaps. and of hypotheses to be tested in the field phase. The Desk Note was presented online. Synthesis A final analysis and an overall assessment of findings were carried out in this phase which consisted of clustering and prioritizing lessons learned, drawing conclusion from the evaluation questions and providing recommendations on future actions. The evaluation followed the recommendations of the UNECE Evaluation Policy17, adopted by the UNECE Executive Committee on 16 December 2021, the administrative instruction guiding Evaluation in the UN Secretariat18; and the United Nations Evaluation Group (UNEG) Norms

15 https://unece.org/sites/default/files/2021- 12/Item%2010_ECE_EX_2021_35_Rev1_Evaluation%20Policy_as%20adopted.pdf 16 https://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm 17 UNECE Evaluation policy 18 ST/AI/2021/3

22

and Standards for Evaluation 19 . Human rights and gender equality considerations were integrated at all stages of the evaluation20: (i) in the evaluation scope and questions; (ii) in the methods, tools and data analysis techniques; (iii) in the findings, conclusions and recommendations of the final report. SDGs considerations were also taken into account during the evaluation of the project. The key questions established for the assessment of evaluation criteria have been structured along the main purpose of the evaluation, the Terms of Reference and the line of the enquiry mentioned above into a set of thirteen key evaluation questions. The criteria for undertaking the evaluation are mentioned in the TOR and represent the standard criteria used for project evaluations: relevance, efficiency, effectiveness and sustainability. OECD/DAC glossary of key terms in evaluation and results-based management updated in 2019 provides the definition of these evaluation criteria as follows: “Relevance: The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, global priorities and partners’ and donors’ policies. Efficiency: A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results. Effectiveness: The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Sustainability: The continuation of benefits from a development intervention after major development assistance has been completed.”21 The evaluation used and analysed a wide range of data and information sources. To make the information received from several sources fully beneficial the combination of information gathered were regularly cross-checked and triangulated. Attention was paid to both project process assessment as well as to its outcome evaluation. Process evaluation was important for measuring whether the planned activities were implemented on schedule and in the right way. In addition, outcome/impact evaluation measured whether the expected changes had been achieved by the project activities after project completion. The evaluation process was a summative one in its nature, it means that it focused on: -assessing what happened -asking if the project did what was planned -asking if implementation was done well -asking if the intended changes happened -asking if the changes observed are attributable to this initiative

19 UNEG 2016 Norms and Standards for Evaluation 20 In line with UNEG Guidance contained in Integrating Human Rights and Gender Equality in Evaluations 21 OECD/DAC Network on Development Evaluation, 2019 which completes and updates the traditional OECD/DAC, glossary of key terms in evaluation and results-based management, Evaluation and Aid Effectiveness series, 2002

23

-focusing on learning. The evaluation was conducted based on the following mixed methods to triangulate information:

1. A desk review of all relevant documents, including the project document and information on project activities (monitoring data); materials developed in support of the activities (agendas, plans, participant lists, background documents, donor reports and publications); proposed programme budgets covering the evaluation period; project reports to the donor.

2. Online survey (designed by the evaluator) of key stakeholders and beneficiaries. 3. Interviews (in-person and/or online) of a wide range of diverse stakeholders and

beneficiaries (names and contact information received from UNECE) from both the private and the public sector, academia, and international organisations.

Theory-based approach: The evaluation is theory-based as it was guided by a reconstructed theory of change to assess whether and how Project activities have contributed to expected results. A theory of change explains how an intervention is expected to produce its results. It clarifies the causal pathways behind an intervention’s cause-and-effect logic, which are often just implicitly assumed but not explicitly specified. The theory of change therefore allows to better examine the causal link between delivered outputs and observed outcomes. For the purpose of this evaluation a result chain, or a pipeline model was applied. It represents the theory of change in terms of boxes, as designed below:

Source: The evaluator. Gender equality and human rights The evaluation paid due attention to the principles of a rights-based and gender mainstreaming approach. Both men and women have been consulted in the evaluation to obtain a proper understanding of the different needs and challenges they face in their specific contexts, social and economic challenges and in trade facilitation development.

Inputs

External donor funding, UNECE human resources for monitoring, reporting and management

Activities

Development/translati on of training materials on TF measures, capacity building seminars, experience sharing regional conference, measuring benchmarking system and scoring tool development, update and maintenance of UNECE TF Implementation Guide

Outputs

Four countries, Kazakhstan, Kyrgyzstan, Tajikistan, Uzbekistan established NTFCs ; Participating countries developed and adopted SPECA TF Strategy and Roadmap ; Tajikistan and Uzbekistan developed national TF roadmap. Benchmarking/scoring tools developed.

Outcomes

Enhanced capacities of policymakers and experts in NTFCs, regulatory agencies to develop and implement TF policies and measures/ Improved capacities of policymakers and experts to measure progress in the implementation of trade facilitation and achieve simplification and harmonization of trade procedures

Impact

Strengthened national capacities of beneficiary countries to implement trade facilitation measures and better integrate into the international rules- based trading system in support of achievements of SDGs.

24

Interview guide and online survey questionnaire The interview guide (see Annex) guided the evaluation as the principal analytical tool. This tool was designed already during the inception phase following the initial meetings held with UNECE stakeholders and an initial review of the project documents. It contains the evaluation questions to be considered for all evaluation criteria, as well as the justification criteria and indicators of success. The Interview guide (see Annex) and the Electronic survey questionnaire (see Annex) were fully aligned. Limitations: The evaluation had to overcome a number of challenges, some of which are described below:

- The E293 project started in 2018 and was scheduled for 2 years; in the end, it was formally closed in December 2023, after multiple no-cost extension. Many stakeholders found it difficult to assess all project aspects and to relate the program inputs, outputs and outcomes between the start of the project in 2018 into 2023. For example, consultants stayed with the project for 2 weeks to 4 months at maximum.

- Similarly, the application of the methodology outlined above faced a challenging situation, represented by the low interest/readiness of the project stakeholders (recommended by UNECE) to participate in the interviews or the on-line electronic survey after 6 years (i.e. due to regular turn-over of officials in governments). To overcome this challenge required extraordinary and time-consuming efforts from evaluator in organizing interviews and/or obtaining necessary feedback through the electronic survey.

- The available documentation and monitoring data of the project were quite fragmented. Over the span of six years, the project’s documentation and reporting has been carried out by different people and in varying formats, depending on the type of activity. Even though the project management team put in a lot of work since the start of the evaluation to make data available to the evaluator, the documentation still lacked coherence.

- The evaluated sample of activities has not been randomly selected. It mainly represents activities judged by the project management team to be ‘good examples’ of the project. The main reason for this was the fact that only a small number of interventions were sufficiently extensive and long-standing to offer a reasonable perspective to draw broader conclusions.

- The absence of budgetary provisions for a field-based evaluation mission to at least one of the targeted countries presented another limitation

- Technical issues related to on-line interviews and surveys (poor internet connection, limited knowledge of some stakeholder of the on-line survey procedures etc.)

25

- limited number of stakeholders, if any, that were actively involved at least during 20-30% of the duration of the project

This chapter presents evaluation analysis and findings of the project results and performance in terms of its activities relevance, effectiveness, efficiency and sustainability, according to evaluation criteria and questions determined by UNECE in the evaluation TOR. Findings are based on evidence generated by documentary review, online interviews and electronic survey activities. The findings reflect a systematic and intensive analysis and interpretation of the information and data obtained, and not the subjective judgement of the evaluator.

Evaluation applied the project performance rating marks, to promote learning and accountability as a foundation of the UNECE journey towards establishing itself as a learning and accountable organizations. The following table explains relationships between the rating number and the extent of the evaluation criteria achievements:

Table 3:

Evaluation criteria Rating number

Degree of criteria achievement

Relevance, effectiveness, efficiency, sustainability 1 Criteria not achieved

Relevance, effectiveness, efficiency, sustainability 2 Criteria little achieved

Relevance, effectiveness, efficiency, sustainability 3 Criteria partially achieved

Relevance, effectiveness, efficiency, sustainability 4 Criteria fully achieved

Relevance assesses the consistency of the objectives and expected accomplishments of the E293 project with the needs/priorities of recipient countries, global development priorities, with UNECE corporate goals and comparative advantages and consideration of cross-cutting issues

1.1 Responsiveness to beneficiary countries needs/priorities and timeliness of the project

Stakeholders interviewed and surveyed confirmed that the project met their needs and priorities in improving trade facilitation measures. The project activities were timely as they were aligned to the participating countries trade facilitation needs in the timeframe 2018-2023

The fundamental project rationale was in essence that when the project was designed trade facilitation was widely recognized by the Central Asian stakeholders as a key factor in

26

economic development policy. During SPECA working group on trade sessions, the Central Asian countries repeatedly acknowledge that trade facilitation can generate major economic benefits in terms of competitiveness and efficiency at a relatively low cost. It was also noted that trade facilitation could enhance the integration of the Central Asia countries transition economies into the global economy. Developments related to trade facilitation at the WTO and other international fora (UNCTAD, ITC, SPECA, WB) have brought this subject to the highest level of political decision-making.

The WTO Trade Facilitation Agreement (TFA) adopted in 2017 became a major driver for the Central Asia countries for the implementation of trade facilitation instruments developed and supported by UNECE and other international organizations. Moreover, with trade facilitation on national political agendas, the Central Asia countries were faced by the challenge of translating the WTO TFA broad concept into specific implementation strategies. Considering a long standing UNECE experience in this area, the Central Asia countries expressed interest in receiving UNECE assistance in supporting the implemention of trade facilitation measures and better integrate into the international rules-based trading system. These expressions of interest were made during the sessions of the SPECA Working Group on Trade held in 2016, 2017 and 2018. Conclusions and recommendations of the SPECA Working Group on Trade confirmed the need and relevance of the UNECE support to the Central Asian countries in trade facilitation strengthening22,23,24, as well as a timeliness of such support by UNECE.

22 2016: https://unece.org/fileadmin/DAM/trade/workshop/2016_May_Ashgabat/Report_Eng.doc 23 2017: https://unece.org/fileadmin/DAM/trade/workshop/2017_July_Geneva/SPECA- WG11_Report_Eng.pdf 242018: http://www.unece.org/fileadmin/DAM/trade/workshop/2018_Sep_Kazakhstan/ConclusR ecs-Eng.doc

27

Photo: Participants of the SPECA Working Group on Trade meeting in Uzbekistan, 2024

Project activities were undertaken in close cooperation with the SPECA countries under the SPECA Working Group on Trade. A high proportion of stakeholders from the SPECA participating states' NTFBs were also involved in this work.

In this context, E293 project was designed to assist its participating countries integrate the WTO trade facilitation system by adopting UNECE respective standards, best practices and recommendations (notably single window system and strengthening/establishing trade facilitation bodies).

Rating: 4 1.2 Alignment with global/regional priorities and with Sustainable Development Goals

The compatibility of the project’s activities/results with the regional priorities of the countries of Central Asia was explained above. As far as global priorities are concerned, the main focus of the project was on supporting the SDGs.

Project document and its activities dedicated very little, if any attention to the contribution of trade facilitation activities to other important global priorities, such as Paris Climate Change Agreement 201525 or any others. In the current context, when UNECE's support for climate action is strongly emphasized in several UNECE publications (for example “UNECE Support for Climate action 2023, 2022, 2021”26,27,28) more attention to this area would be expected from the project managed by UNECE. The project did not fulfil this expectation. In this context, it is fair to

25 https://unfccc.int/process-and-meetings/the-paris-agreement 26 https://unece.org/sites/default/files/2023-10/UNECE%20Climate%20Action_Brochure_WEB%20%282%29.pdf 27 https://unece.org/info/UNECE-and-the-SDGs/pub/372540 28 https://unece.org/info/UNECE-and-the-SDGs/pub/361192

28

note that the project was designed shortly after the 2015 Paris Agreement and long before the UNECE Evaluation Policy of 2021.

On the other side, the relationship between the planned activities of the project and the SDGs was well defined in the project document drafting phase. Project Concept Note stipulated that:

“This project will contribute to achieving SDG 17: Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development, in particular its target 17:10 – promotion of the international rules-based trading system in a region, which still faces challenges in its integration into this system, as well as target 17.11: increasing the share of developing countries in international exports. From a broader perspective the project also will contribute to achieving SDG 1: End poverty in all its forms, SDG 9: Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation, and 16: Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all level.”

Project clearly supported the sustainable development goals, namely its following targets:

-17.10 which focuses on promoting a universal, rules-based, open, non-discriminatory, and equitable multilateral trading system under the World Trade Organization (WTO). This includes the conclusion of negotiations under the Doha Development Round29. In this context E293 aimed to create a fair and inclusive global trading environment that benefits all SPECA nations, regardless of their economic status and,

17.11. in supporting SPECA countries to increase their exports by implementation of trade facilitation measures. Some of the SPECA participating states are not yet WTO members and in their accession aspirations the project stimulated their pursue for policy reforms with the goal to harmonize their domestic trade regime as much as possible with the top-tier best practices of the WTO. For example, the project implemented the following activities: (Activity 1.1) drafting training materials on the four targeted trade facilitation measures (national trade facilitation committees or NTFCs -TFA Art.23.2; streamlining documentary procedures – TFA Art. 10.1; using international standards – Art.10.3; and Single Window – Art.10.4) as well as PowerPoint presentations as training tools were prepared during the normal project period. One additional guide on an additional measure (establishing a TF enquiry point – TFA Art.1.3, tailored for the SPECA countries was requested by the countries and was prepared with support from the project. Two itineraries of the UNECE Trade Facilitation Implementation Guide (TFIG), namely WTO Trade Facilitation Agreement (TFA) and SMEs and Trade Facilitation were developed through a grant to the ILO training institute in Turin. In addition, the Single Window itinerary was modified. The webpages of these itineraries have been designed and content have been incorporated. Some new instruments to TFIG have been added including: UN Regional Commissions Survey on Trade Facilitation and Paperless Trade – Regional and Global Reports, Reference Data Model, Technical Notes on Trade Facilitation, Public-Private Partnerships in Trade Facilitation (Recommendation

29 https://www.wto.org/english/tratop_e/dda_e/dda_e.htm

29

41). UNCTAD, ITC and ESCAP pages have been updated as well. The content was translated into Russian. Several broken hyperlinks to external materials have been fixed. ITC-ILO, the grantee of the update and maintenance of TFIG pages conducted the technical part of the activities whereas secretariat staff and consultants jointly coordinated the content development and update.

All this went in parallel with the progress in other non-trade related SDGs that were indirectly benefitting from the increased prosperity and economic growth resulting from the processes mentioned above.

Rating: 3

1.3 Alignment with the program of work of the UNECE and UNECE project added value

UNECE as a multilateral platform facilitates greater economic integration and cooperation among its fifty-six member States and promotes sustainable development and economic prosperity through:

 policy dialogue,  negotiation of international legal instruments,  development of regulations and norms,  exchange and application of best practices as well as economic and technical

expertise,  technical cooperation for countries with economies in transition.

The UNECE contributes to enhancing the effectiveness of the UN through the regional implementation of outcomes of global UN Conferences and Summits. Its terms of reference have been defined by ECOSOC.30

The UNECE region continues to face economic and environmental challenges in addressing the complexity of achieving the SDGs, which remain a source of primary concern to member States. In line with the UNECE mandate/priorities the project addressed a persisting challenge faced by the SPECA countries: implementation of trade facilitation measures in line with the WTO Trade Facilitation Agreement and better integration of these countries into the international rules- based trading system.

Structure of the National Trade Facilitation Committees proposed by UNECE Manual:

30 https://unece.org/objectives-and-mandate

30

The project concentrated on the four measures in the WTO Trade Facilitation Agreement, in which UNECE has decades of experience, mandate and comparative advantage. The project contributed to the capacity of the SPECA participating States and their business community to advance trade facilitation by using the main comparative advantages of UNECE:

- through the project UNECE developed major tools to assist learning about trade facilitation: including four training materials on core areas of trade facilitation and measures in the WTO TFA, which are the areas where UNECE and its subsidiary body UN/CEFACT have advantage: o national trade facilitation committees (NTFCs) -TFA Art.23.2 and UN/CEFACT

Recommendation 4, o streamlining documentary procedures – TFA Art. 10.1 and most standards and trade

facilitation recommendations of UN/CEFACT, as well as standards and other tools for digitalization,

o use of international standards – TFA Art.10.3 and half of the UN/CEFACT Recommendations,

o Single Window – Art.10.4 and UN/CEFACT Recommendations 33-36, o Training materials (PPt presentations, etc.) based on the UN/CEFACT Recommendation

42: Establishment of a Trade and Transport Facilitation Monitoring Mechanism (TTFMM)31 developed with ESCAP and contributions made to the UN Survey on trade facilitation, which produced a special issue on the SPECA subregion32,

o Upon request from the SPECA countries, UNECE developed a Guide on Establishing a Trade Facilitation Enquiry Point (TFA Art.1.3 and UN/CEFACT Recommendation 40: Consultation Mechanisms33) as training tools were prepared during the regular project period.

The project was fully aligned with the program of work of UNECE and its added value was reflected in the following activities led by UNECE:

31 https://unece.org/sites/default/files/2023-10/Rec42-ECE_TRADE_C_CEFACT_2017_8E-Rev1.pdf 32 https://www.unescap.org/kp/2023/untf-survey-2023- SPECA?ref=untfsurvey.org&_gl=1*1f7k8b4*_ga*NzEyODkyNzAxLjE3MDk1NzYwNTU.*_ga_SB1ZX36Y86*M TcwOTgzMzgzOS4zLjEuMTcwOTgzMzg1NC40NS4wLjA. 33 https://unece.org/sites/default/files/2023-10/Rec40-ECE_TRADE_C_CEFACT_2015_9E_Rev1.pdf

31

- UN/CEFACT reviewed the training modules, endorsed them, and published them on its website, for wider use.

- The trainings on these subjects and the support for the implementation of the SPECA Trade Facilitation Strategy made a valuable contribution to the dissemination of the UNECE standards and best practice recommendations on trade facilitation.

Rating: 4 1.4 Consideration of gender, human rights, disability perspective and climate change integration into the design and implementation of the project Consideration of human rights, disability perspective and climate change integration were not key focus area of the project although the importance of these subjects was recognized by a few of the interviewed stakeholders. The project design did not include in its proposed strategy explanation of how the project would contribute to gender equality and women’s empowerment, the realization of human rights, with an emphasis on “leaving no one behind”. The same applies to the integration of climate change consideration. There is no reference made to these aspects neither in the Technical Cooperation Form, nor in the Project Concept Form. This is a weakness of the project considering the World Bank Group study made in 2018 informing that across the world, countries are losing $160 trillion due to the lifetime earnings differences between women and men. It’s a situation that doesn’t just affect women, who’ve been blocked from higher earnings, jobs, and capital. It affects everyone — leaving trillions on the table instead of catapulting gross domestic product growth. In nearly every country today, women face barriers to fully participate in the work force and earn as much as men. The losses in wealth from inequality in earnings between men and women vary by region. The largest losses—each between $40 trillion and $50 trillion— are observed in East Asia and the Pacific, North America, and Europe and Central Asia. The evaluation noted that the gender dimension was reflected in the participation and the discussion held in SPECA WGT session in 2019 in Uzbekistan. Evidence can be found for instance in the report from the Technical Cooperation Seminar on Implementing Specific Provision of the WTO TFA held in Nur-Sultan, from 20-22 May 2019 showing a considerable involvement of women in the discussions, as well as in the follow up activities to this event34. One of the electronic survey responders mentioned that “Gender rights were actively taken into account, because the participation of women in all events organized by E293 was evident.” The gender dimension was taken into account during the evaluation phase reflected in the significant percentage of woman stakeholders in interviews/electronic survey. Another sign of woman's perspective mainstreaming was integrated into the project activities was the evidence that, the key consultants and project management staff in the UNECE were women.

34 Technical cooperation Seminar on Implementing Specific Provisions of the WTO Trade Facilitation Agreement; National Trade Facilitation Committee and a Trade and Transport Facilitation Monitoring and Measuring Mechanism Nur-Sultan, Kazakhstan 20 - 22 May 2019/Report

32

Reports from other SPECA WGT meetings as well as reports from capacity seminars organized as part of the project do not contain sex-disaggregated information about the participants of these events. Lack of sex-disaggregated data and full-scale gender statistics from the key meetings/seminars was a considerable obstacle to asses a gender-based approach during the project implementation. Women’s empowerment and gender equality were not embedded in the project design. Generally, some trade facilitation activities include elements that relate to human rights issues, of which labour-related human rights are most widely found. No performance indicators were defined concerning human rights and climate change integration. On a positive note, UNECE/UN-CEFACT published a white paper “Women in trade facilitation”35, during the project duration. This paper is part of UN/CEFACT’s gender mainstreaming efforts which look to create deliverables that provide support for policy development, data creation and research, standards, recommendations, advocacy and dialogue, resource allocation, planning, implementation, and monitoring of programs aimed at eliminating gender disparities in trade facilitation. Project lacked specific analyses, activities and goals related to other human rights issues. It can be concluded that the mainstreaming issues were not sufficiently embedded in the interventions

Rating: 2,5

Relevance Overall Rating: 3,38

Effectiveness is to assess that to what extent the intervention’s objectives and expected results were achieved for meeting the needs of beneficiary countries and assuring coherence and harmonization with other partners operating within the same context and overcoming challenges faced during the project duration

2.1 Extent of project design and set up effectiveness in meeting the needs of beneficiary countries

As explained already above, Central Asia and Azerbaijan very much needed support in trade facilitation and preparing for accession to the WTO at the time of designing the project. Political will for trade facilitation in all countries, both members of WTO at the time (Kyrgyzstan and Tajikistan) and the acceding countries, had gained momentum. The governments, the business community and the development partner engaging with the region (GIZ, UNCTAD, ITC, etc.) looked at UNECE and UN/CEFACT for cooperation on the areas in which UN/CEFACT had

35 https://unece.org/sites/default/files/2023-10/WhitePaper_Women-TF_Eng.pdf

33

comparative advantage in trade facilitation. The project played a catalytic role in accelerating activities on trade facilitation under the SPECA WG on Trade and other activities on trade facilitation supported by UNECE. Project design was effectively addressing the needs of the Central Asia countries related to accession requirements to WTO TFA in assisting SPECA countries to implement the provisions of this agreement.

For example, the project played an instrumental role in the following activities: (A 1.1) drafting training materials on the four targeted trade facilitation measures (national trade facilitation committees or NTFCs -TFA Art.23.2; streamlining documentary procedures – TFA Art. 10.1; using international standards – Art.10.3; and Single Window – Art.10.4) as well as PPt presentations as training tools were prepared during the normal project period. One additional guide on an additional measure (establishing a TF enquiry point – TFA Art.1.3, tailored for the SPECA countries) was requested by the countries and prepared later with support from the project.

In line with the needs of beneficiary countries as voiced during the SPECA WGT meetings and training seminars, the project supported the preparation of several training materials:

-Training Material on the Implementation of a Single Window (ECE/TRADE/C/CEFACT/2021/INF.5) https://unece.org/sites/default/files/2021- 04/ECE_TRADE_C_CEFACT_2021_INF5E_TrainingMaterialSW.pdf -Training Material on the Use of International Standards (United Nations ECE/TRADE/C/CEFACT/2021/INF.6) https://unece.org/sites/default/files/2021- 04/ECE_TRADE_C_CEFACT_2021_INF6-StandardsTraining.pdf -Training Material on National Trade Facilitation Bodies (ECE/TRADE/C/CEFACT/2021/INF.7) https://unece.org/sites/default/files/2021-04/ECE_TRADE_C_CEFACT_2021_INF7-NTFB- Training.pdf -Guide for National Trade Facilitation Bodies on How to Use UN/CEFACT Standards and Tools (ECE/TRADE/C/CEFACT/2021/INF.8) https://unece.org/sites/default/files/2021- 04/ECE_TRADE_C_CEFACT_2021_INF8-Guide-NTFBs-UNCEFACT-Standards.pdf; https://unttc.org/documents/guide-national-trade-facilitation-bodies-how-use-uncefact-trade- facilitation-standards; https://unttc.org/documents/prakticheskoe-rukovodstvo-dlya- nacionalnykh-organov-po-uproscheniyu-procedur-torgovli-kak These training materials were used in the substantive part of the programmes in at least two large regional seminars: in May 2019 in Nur Sultan (Astana) and July 2020 in Dushanbe, carried out in collaboration with all the SPECA participating States and such international partners as GIZ (Nur Sultan) and OSCE (Dushanbe), for numerous activities in support of, for example, the establishment and functioning of national trade facilitation bodies (committees) and a regional network of such committees. All SPECA countries received assistance for the establishing and functioning of national trade facilitation committees (NTFCs). Four of them (Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan) actually launched such bodies during the period and also established a regional network of these bodies. UNECE provided much information on its

34

experience with these bodies and technical advice on their functioning. This was reflected in the tangible improvement of the performance of the countries in the reports on the implementation of the WTO Trade Facilitation Agreement (https://www.tfadatabase.org/en/implementation) and the UN Survey on Trade Facilitation and Paperless Trade (https://www.untfsurvey.org/).

The training materials and the activities under the project contributed significantly to the development and adoption by the intergovernmental SPECA Governing Council of the Trade Facilitation Strategy.

(https://unece.org/fileadmin/DAM/SPECA/documents/gc/session14/SPECA_Trade_Facilitation_ Strategy_English.pdf), the Roadmap for the implementation of this strategy (https://unece.org/fileadmin/DAM/SPECA/documents/gc/session14/Roadmap_for_Implement ation_of_the_SPECA_TF_Strategy_English.pdf) and their implementation, supported by the project.

Rating: 4

2.2 Extent of implementation of the planned activities required for the achievement of the project objective/expected results

According to the technical cooperation project form the main objective of the project was “to strengthen national capacities of beneficiary countries to implement trade facilitation measures and better integrate into the international rules-based trading system”. The project designers believed that in order to achieve the expected results of the project, it will be necessary to carry out the following:

A1.1. Adaptation and development of training materials and modules for national TF courses to cover the relevant target areas in the TFA for the implementation of WTO TFA provisions related to Articles 10.1 on formalities and documentary requirements; 10.3 on use of international standards; 10.4 on single window; and 23.2 on national trade facilitation committee (NTFC); A1.2. Organization of wo capacity-building seminars for members of NTFBs, regulatory agencies, and other relevant stakeholders on the development of national trade facilitation roadmaps, including for the implementation of the WTO targeted TFA measures; A1.3. Organization of one sub-regional SPECA capacity building and experience sharing conference to strengthen cooperation on the implementation of TF measures and supporting international standards, including on the aspects of Single Window interoperability; A1.4. Translation of the training materials developed under A1.1 into Russian; A2. l. Development of training materials on benchmarking mechanisms to measure progress and achieve policy coherence in the implementation of TF measures, based on the UNICEF ACT Trade and Transport Facilitation Monitoring Methodology (TTFMM) in the beneficiary countries; A2.2. Organization of two capacity-building seminars for NTFBs on the development of national benchmark mechanisms to measure progress and achieve policy coherence in TF based on the UNICEF ACT TTFMM methodology; A2.3. Translation of learning materials and training courses into Russian;

35

A2.4. Update and maintenance of the UNECE Trade Facilitation Implementation Guide (TFIG) by incorporating project results and lessons learned and the new learning materials, including through the development of new itineraries providing a step-by-step approach to the implementation of TF measures in the project target areas, and the drafting of case stories; A2.5. Organize a concluding Conference (in the context of the UNICEF ACT Forum) on regulatory cooperation for TF to disseminate project’s outcomes and share lessons learned from project target countries. It was planned that if all activities are implemented as planned the project would achieve its two expected results:

EA.1. Enhanced capacities of policymakers and experts in National Trade Facilitation Committees (NTFCs), regulatory agencies and other relevant stakeholders, to develop and implement Trade Facilitation (TF) policies and measures, notably in the area of WTO Trade Facilitation Agreement (TFA) Articles 10.1; 10.3; 10.4; and 23.2 EA.2 Improved capacities of policymakers and experts to measure progress in the implementation of TF, and to achieve internal and cross-border policy coherence for the simplification and harmonization of trade procedures.

The following indicators of achievement were formulated in the E 293 concept note:

Indicator of achievement A1.1. - At least two countries established and/or maintained national trade facilitation bodies;

Indicator of achievement IA1.2. - At least four national policy documents (e.g. decree on establishing a National Trade Facilitation Committee, a strategy for the implementation of international trade facilitation standards or other document) promoting trade facilitation formulated;

Indicator of achievement IA2. - One pilot benchmarking system for measuring progress in trade facilitation developed. As written in the project final report the extent of implementation of the planned activities as of 31.12.2023 was the following:

A.1.1 – fully implemented – all planned training materials and modules were developed.

A.1.2 – partially implemented – against planned two capacity-building seminars for members of NTFBs, regulatory agencies and other relevant stakeholders on the development of national TF roadmaps, including for the implementation of the targeted TFA measures, only one such seminar was carried out. The gap was partially closed by online webinars.

A.1.3 – fully implemented

A.1.4 – fully implemented

A.2.1 – fully implemented

36

A.2.2 – partially implemented – from originally planned two capacity building seminars dedicated to the UN/CEFACT methodology, only one seminar was organized in Nur-Sultan in May 2019 (co- funded under the UNDA 11 Tranche project) together with GIZ, ESCAP and ADB.

A.2.3 – fully implemented

A.2.4 – fully implemented

A.2.5 – not implemented

Several activities that conditioned the fulfillment of expected results were not carried out for some subjective reasons. One of the reasons was a frequent change of the project manager in the course of project undermining consistency of original work plan implementation. It was also reported during the interviews that that a portion of the project financial resources was spent for activities with a limited relation to trade facilitation support in Central Asia. It is notable, from a sustainability and SDG perspective, that the project also contributed to enhancing the understanding on the relationship between trade, including trade facilitation, and sustainable development (e.g. in the context of the SPECA Principles for Sustainable Trade).

Interviews/electronic survey results indicated that the project expected results were achieved fully regarding EA.1 and, partially regarding EA.2. Several interviewed stakeholders thought that the progress achieved concerning internal and cross-border policy coherence for the simplification and harmonization of trade procedures was less tangible than expected. The progress made has been only partial, and there is still much work ahead of the Central Asian countries to achieve a workable harmonization and cross-border policy coherence for the simplification and harmonization of current trade procedures.

Rating: 3

2.3 Harmonization and coherence of activities with those of other partners operating within the same context

UNECE is one of several institutions/agencies supporting the Central Asia countries in the area of the trade facilitation. The evidence collected from interviews and electronic survey suggests that the project has maintained a dialogue and practical collaboration with a number of development partners such as UNCTAD, WTO, ITC, UNESCAP, GIZ, USAID.

Regular dialogue with majority of the “competing” agencies during the SPECA trade facilitation group meetings and various capacity building seminars facilitated harmonization of efforts in trade facilitation. As a matter of fact, several of these agencies were active in raising awareness about the trade facilitation and WTO TFA. However, considering the importance of the awareness raising about these issues in the region this situation can be classified as a “useful redundancy”. The view expressed by interviewees was that the relationship among the trade facilitation actors in Central Asia resulted in positive results and there had not been issues leading to a counterproductive competition.

37

Project achieved sound coordination in the support for the establishment of National Trade Facilitation Committees (NTFCs) and their regional network in partnership with all the partners mentioned above. Especially fruitful was the close collaboration with GIZ, which focused on support to the NTFCs. Project cooperated also with the WTO secretariat on several activities, especially on drafting the Guide for the implementation of Art 1.3 of the WTO Trade Facilitation Agreement (Trade facilitation Enquiry Points). WTO staff helped in drafting it, and accepted it as the very first Guide on Trade Facilitation Enquiry Points. Guide was published on the WTO web site, as a best practice example to be used by any country. Another positive example is cooperation in the context of GIZ Regional TF project for SPECA countries, in the context of which UNECE supported GIZ in providing assistance concerning the establishment of the national trade agreement facilitation committees (established in Kazakhstan, Kyrgyzstan, Tajikistan, Uzbekistan). Representatives of the donor agencies were invited to participate in meetings promoted by the project, and the contribution of the donors was highlighted at the meetings. The partnerships established under this project provide sound basis for continuing the work on trade facilitation in the SPECA region. Photo: Presentation of GIZ Trade Facilitation Project in Central Asia during SPECA meeting:

Rating: 4 2.4 How successfully did the project overcome the challenges/obstacles to achieving expected results?

Documentary evidence and discussion with stakeholders identified a few obstacles/challenges that prevented the project to achieve expected results in full. Three major factors considered as disruptions to the complete achievement of the project results were:

- The COVID 19 pandemic which impacted the implementation approach and strategy - several changes in project management responsibilities - temporal discontinuity of the project activities.

38

The Covid19 pandemic was definitely a huge problem for trade facilitation advancement in the SPECA region and the world as a whole. The pandemic resulted in a very low implementation rate in 2019 and 2020. Building on synergies with other projects and partner organizations colleagues, the project managed to do studies on the immediate effects of the pandemic on trade facilitation and organized one of the planned trainings, (originally planned in person), in 2020 in virtual mode. This on-line event had a strong focus on the effects of the pandemic in the region. Moreover, several meetings planned by the work plan had to be canceled. Several interviews showed that the effectiveness of the project was adversely affected by several factors, such as frequent changes in responsibility for project implementation and engagement of several actors on the UNECE side. While noting that the changes were done in compliance with UN rules and regulations and in consultation with the PMU at that time, it might have been more effective to ensure the continuity of the project responsibility with person who planned/formulated the project and negotiated it with the beneficiaries and the donor instead of removing this person temporarily from its responsibility. Moreover, such unexpected change raised several doubtful questions in the recipient countries. The project would have benefitted from appropriate communications with stakeholders in beneficiary countries to explain these changes. During the project implementation duration, the UNECE senior management changed several times the personal responsibility for the project management. There were significant gaps concerning effective planning for a project-related staff continuity. One interview partner noted that this situation resulted several times in decisions leading to diversion of the part of the project financial and human resources for purposes with limited relevance for trade facilitation strengthening in the Central Asia countries. Above mentioned obstacles resulted in considerable delays in implementation of planned activities with major disruptions in organizing meetings, field visits and trainings. Originally planned project duration was 2 years, however the project ended up lasting 6 years.

Rating: 2,5

Effectiveness overall rating: 3,38

Efficiency is to assess the extent a project intervention has converted its resources and inputs (funds, expertise, time, etc.) economically, efficiently organized and on time into expected results.

3.1 Adequacy of the project resources for achieving its planned results.

39

Financial efficiency of the project was quite good in terms of achievement of its planned results. As a matter of fact, the estimation of the ideal ratio of the financial, human and other resources, and the project results is not easy especially when dealing with trade facilitation capacity building in the Central Asia region. Nevertheless, it is appropriate to say that with a relatively modest budget the project activities contributed to strengthening of the capacity of Central Asian countries to implement trade facilitation measures and increase the window for a better integration of these countries into the international rules-based trading system. Only one of the interviewed/surveyed stakeholders felt that the project resources were not sufficiently robust for achieving its planned results.

Donor (Russian Federation) financial contribution for the project implementation was supplemented by human resources of UNECE office in Geneva, namely 4 months/year of one regular budget staff time at P-4 level and administrative support of one general staff on a part- time basis.

Table 4: Expenditure report for the UNECE project E293 (1 January 2017 to 30 June 2023)

Explanatory Note on Budget Classes: Staff and personnel costs: includes consultant contracts and travel of consultants Contractual Services: includes external commercial contracts such as for room rental, interpretation services and individual contractors for project support Budget Class Fiscal Year Expenditures (USD) 1. Staff and personnel costs 2018 9 000,00 2. Contractual Services 2018 6 090,00 3. Operating and Other Direct Costs 2018 0,00 4. Travel (staff travel and Meeting participants) 2018 0,00 5. Grants out 2018 49 610,92 6. Equipment Veh Furniture 2018 0,00 7. UN Programme Support Costs 2018 8 411,12

Total 2018 73 112,04

Budget Class Fiscal Year Expenditures (USD) 1. Staff and personnel costs 2019 3 240,00 2. Contractual Services 2019 4 002,40 3. Operating and Other Direct Costs 2019 418,40 4. Travel (staff travel and Meeting participants) 2019 12 458,79 5. Grants out 2019 0,00 6. Equipment Veh Furniture 2019 0,00 7. UN Programme Support Costs 2019 995,88

Total 2019 21 115,47

40

Budget Class Fiscal Year Expenditures (USD) 1. Staff and personnel costs 2020 56 990,26 2. Contractual Services 2020 9 704,40 3. Operating and Other Direct Costs 2020 1 494,53 4. Travel (staff travel and Meeting participants) 2020 0,00 5. Grants out 2020 0,00 6. Equipment Veh Furniture 2020 0,00 7. UN Programme Support Costs 2020 8 864,58

Total 2020 77 053,77

Budget Class Fiscal Year Expenditures (USD) 1. Staff and personnel costs 2021 61 967,20 2. Contractual Services 2021 11 965,40 3. Operating and Other Direct Costs 2021 656,20 4. Travel (staff travel and Meeting participants) 2021 10 606,51 5. Grant out 2021 0,00 6. Equipment Veh Furniture 2021 380,00 7. UN Programme Support Costs 2021 9 745,95

Total 2021 95 321,26

Budget Class Fiscal Year Expenditures (USD) 1. Staff and personnel costs 2022 38 219,60 2. Contractual Services 2022 345,00 3. Operating and Other Direct Costs 2022 329,70 4. Travel (staff travel and Meeting participants) 2022 76,09 5. Grant out 2022 0,00 6. Equipment Veh Furniture 2022 0,00 7. UN Programme Support Costs 2022 5 056,25

Total 2022 44 026,64

Budget Class Fiscal Year Expenditures (USD) 1. Staff and personnel costs 2023 25 000,00 2. Contractual Services 2023 0,00 3. Operating and Other Direct Costs 2023 212,00 4. Travel (staff travel and Meeting participants) 2023 3 449,75 5. Grant out 2023 0,00 6. Equipment Veh Furniture 2023 0,00 7. UN Programme Support Costs 2023 3 277,56

Total 2023 31 939,31

Budget Class Fiscal Year Expenditures (USD)

41

1. Staff and personnel costs 2018-2023 194 417,06 2. Contractual Services 2018-2023 32 107,20 3. Operating and Other Direct Costs 2018-2023 3 110,83 4. Travel (staff travel and Meeting participants) 2018-2023 26 591,14 5. Grant out 2018-2023 49 610,92 6. Equipment Veh Furniture 2018-2023 380,00 7. UN Programme Support Costs 2018-2023 36 351,34

Total 2017-2023 342 568,49

Rating: 4

3.2 Degree of achievement of the planned results on time an organization of all activities efficiently

The efficiency in achieving the planned results within the originally planned schedule was quite low. The evidence from Table 5 underneath indicates that the annual implementation rates during the entire duration of the project were very low. The achievement of the planned results took three times longer period than was assumed in the project document. At the same time, it has to be noted that project extensions were agreed upon with the donor, and that the project implementation coincided with important events, such as the COVID pandemic and related travel restrictions.

The Technical Cooperation Project Form established quite ambitious project implementation schedule for 24 months. This planned assumption turned out to be unrealistic for several reasons. The main reasons that led to a significant delay in meeting the planned goals on time were:

- Limited human resources for project implementation

- Frequent shifting of the main responsibility for project management/implementation from person to person

- The proliferation of small consultancy contracts with limited coordination and communication among these consultants

- Limitation imposed by COVID 19 pandemic

This complex situation could have been, at least partially, solved by hiring 1 or 2 highly qualified individual contractors for the extended period (12-24 months) to accelerate annual implementation rates. But instead, the project used plenty of its financial resources on short- term consulting contracts for pursuing small partial goals without sufficient coordination and shared vision among the consultants.

Rating: 2

42

3.3 Extent of efficiency in using the resources economically

The main challenge in evaluating this question was that the majority of the participants in the electronic survey and interviews did not have an overview of how the project financial resources were used. Their overall feeling was that they did not have sufficient information concerning the financial and other resources allocated to the project. Human resources consisted of a project manager at P level (part-time), his/her assistant at G level (part-time) and a long list of consultants.

Table 5: Consultancy services -Project E293

Consultant or IC contract dates duration amount AA

(Individual Contractor) 1.07-30.09.2018 3 months 3000 SB-008547

(Individual Contractor) 1.10 to 30.11.2018 2 months 3000 SB-008547

(Individual Contractor) 14.1.2019 to 14.2.2019 10 days 10 days 1000

SB-008547- E293

(consultant) 16 to 31 Dec 2019 11 days 3000 SB-008547

(Individual contractor- Editor) 16 to 31 Dec 2019 3000 SB-008548

(consultant) 16.7-16.10.2018 3 months 9000 SB-008547

(Individual Contractor)

2 months from 06.04.2020 to 05.06.2020 2 months 7000 SB-008547

( Consultant)

3 months between 15 May and 15 August 2020 3 months 5000 SB-008547

(Consultant) 60 days between 18 May and 30 Sep 2020 2 months 10000 SB-008547

(Consultant)

1 month between 12 Oct. And 6 Nov. 2020 1 month 6000

SB- 008547

(Individual Contractor)

2 months between 18.08 and 18.10.2020 2 months 2000

SB- 008547

43

(Individual contractor- Editor) 1/09-31/11 2020 3 months 3000 SB-008548

(Consultant)

3 months between 11 Nov 2020 and 11 Mar. 2021 3 months 15000

SB- 008547

(Consultant) 8.2.2021 to 5.03.2021 terminated on 5.3.2021

1 month

8000

SB- 008547

(Consultant) 05.07-05.10.2021

3 months 12000

SB- 008547

(Consultant) 02.08-19.10.2021 2,5 months

12000

SB- 008547 (E293)

(Consultant) 30.7 - 30.9.2021

2 months 20000

SB- 008547

(Consultant)

2 months between 11/10/2021 - 11/12/2021 2 months 9000

SB- 008547

(Consultant) 16/05/2022 - 16/09/2022 4 months 17000

SB- 008547

(Consultant) 23/05/2022 - 23/09/2022 4 months 10000

SB- 008547

(Consultant) 17/10/2023- 29/12/2023 56 days 14000

SB- 008547 (E293)

(Consultant) 22/12/2023- 30/04/2024 50 days 11000

SB- 008547 (E293)

183 000 total

(Olga Kharitonova) Staff GS, 2months Nov-Dec 2020 19 550 %

Staff & personnel Consultants and admin charges 174 000 % 194417

Contractual services Individual contractors 19 000 %

44

Contractual services

Contractual services (interpreters, translators, equipment hire during the events organised in the countries) 13 107 % 32107

The significant portion of financial resources was used for staff and personnel costs associated with the hiring of twenty-two consultants and associated travels. In most cases, the consulting contracts had a short duration from 10 days to 2 months. The consultants were usually focused on a narrow part of the project's activities, which did not allow them to gain a global view of the project. This problem could have been avoided if the project management entrusted responsibility to a smaller number of consultants with much more complex responsibilities who would ensure greater synergy between the main activities of the project.

The consultants’ focus was predominantly on the preparation of training manuals, capacity building events, preparation and realization of meetings within the framework of SPECA, preparation of country specific trade facilitation road maps and implementation of SPECA Trade Facilitation Strategy.

The project also relied on the expertise and knowledge of staff within the UNECE Secretariat to deliver and/or directly coordinate a number of activities, as for example:

(Activity 1.1) drafting training materials on the four targeted trade facilitation measures (national trade facilitation committees or NTFCs -TFA Art.23.2; streamlining documentary procedures – TFA Art. 10.1; using international standards – Art.10.3; and Single Window – Art.10.4) as well as PPt presentations as training tools were prepared during the normal project period. One additional guide on an additional measure (establishing a TF enquiry point – TFA Art.1.3, tailored for the SPECA countries) was requested by the countries and prepared later with support from the project. Training Material on the Implementation of a Single Window (ECE/TRADE/C/CEFACT/2021/INF.5)36 Training Material on the Use of International Standards (United Nations ECE/TRADE/C/CEFACT/2021/INF.6)37 Training Material on National Trade Facilitation Bodies (ECE/TRADE/C/CEFACT/2021/INF.7)38

36 https://unece.org/sites/default/files/2021-04/ECE_TRADE_C_CEFACT_2021_INF5E_TrainingMaterialSW.pdf 37 https://unece.org/sites/default/files/2021-04/ECE_TRADE_C_CEFACT_2021_INF6-StandardsTraining.pdf 38 https://unece.org/sites/default/files/2021-04/ECE_TRADE_C_CEFACT_2021_INF7-NTFB-Training.pdf

45

Guide for National Trade Facilitation Bodies on How to Use UN/CEFACT Standards and Tools (ECE/TRADE/C/CEFACT/2021/INF.8)39 (A1.2) technical cooperation Seminar on Implementing Specific Provisions of the WTO Trade Facilitation Agreement, was organized by UNECE in Nur-Sultan (Astana) May 201940 in collaboration with GIZ, ESCAP, and the Asian Development Bank as a model for a seminar under this project, which had to be replicated in other locations. (A1.2) online seminar on Trade Facilitation in Times of COVID-19, 6-7 July 2020 for the SPECA countries: hosted in Dushanbe, Tajikistan, online due to the pandemics and restrictions of travel41 training was based on materials prepared under the project for the SPECA countries and new analysis of the situation with trade facilitation under the Covid-19 pandemic. (A1.3) 18.11.2021- Tashkent – first physical regional meeting of the NTFCs after the pandemic, back-to-back with the 16th session of the SPECA WG on Trade, the annual SPECA Forum and session of the SPECA Governing Council. Organized in collaboration between the ECE Regional Adviser and GIZ42 The evaluator could not contact the project manager for the years 2018-2020 (retirement from UNECE) and therefore the evaluation of effectiveness is limited in scope for the period 2018- 2020. E293 provided generally good value for money, but economic efficiency of different intervention strategies differs significantly. UNECE did not achieve an ideal ratio of good value for money since some of its activities were diverted from the main project focus as planned in the project document. Other negative factors that reduced the overall effectiveness of the project were frequent changes in responsibility for project management.

Rating: 3

Efficiency overall rating: 3,0

4. Sustainability

Sustainability assesses the degree of the likelihood of the project long-term benefits for recipient countries. The project document does not discuss which aspects should be considered for assessment of sustainability of its achievements. In the absence of clear sustainability criteria at the beginning of the project, the evaluation focused on enhancing technical knowledge on trade facilitation

39 https://unece.org/sites/default/files/2021-04/ECE_TRADE_C_CEFACT_2021_INF8-Guide-NTFBs-UNCEFACT- Standards.pdf; https://unttc.org/documents/guide-national-trade-facilitation-bodies-how-use-uncefact-trade- facilitation-standards; https://unttc.org/documents/prakticheskoe-rukovodstvo-dlya-nacionalnykh-organov-po- uproscheniyu-procedur-torgovli-kak 40 https://unece.org/info/Trade/events/18240 41 https://unece.org/info/Trade/events/17768 42 (https://unece.org/sites/default/files/2021-11/SPECA_WG_on_Trade_2021_Agenda_EN.pdf)

46

standards in the recipient countries, empowering trade facilitation institutional structures and national legislative and strategic documents. 4.1 Measures adopted to ensure that project outcomes would continue after the project ended and extent of addressing existing risks of sustainability

It’s always difficult to measure at the end of the project focusing on trade facilitation strengthening its sustainability since by definition, sustainability can only be assessed later on. Stakeholders’ interviews identified several factors that could limit sustainability of the project: -Human resource limitation in terms of numbers and technical knowledge -Lack of internal coordination -Rotation of government staff -Lack of relevant capacity building activities -Not sufficiently robust institutional and regulation platforms The project contributed positively to reducing the risks of sustainability associated with two critical issues: 1. enhancing limited technical knowledge concerning trade facilitation measures and the international rules-based trading system by publishing a long series of relevant manuals (both in English and Russian), and conducting capacity building seminars and workshops, and 2. strengthening regional and national institutional trade facilitation platforms in the Central Asia countries. E293 used well for its trade facilitation capacity building activities SPECA working group on trade and collaboration with GIZ and other partners. In this context, the evaluation has noted several important contributions of the project, as follows: A1.1.1: A report was drafted on the results of activities in implementation of the SPECA TF Strategy and Principles of Sustainable Trade and was presented at the 16th session of the SPECA WG on Trade, the 2021 SPECA Economic Forum and 16th session of the SPECA Governing Council in November 2021.43 A1.2.1 Five meetings were organized with support from the project on how to strengthen operational capacity and subregional cooperation among the NTFBs in the SPECA countries (four of them in collaboration with GIZ). A1.3.1 GIZ and UNECE organized a sub-regional event (first in person seminar of the NTFCs in SPECA countries after the outbreak of the COVID-19 pandemic) on 16 November 2021 in Tashkent, Uzbekistan, back-to-back with the 16th session of the SPECA WG on Trade. A2.1.1 Two sessions of the WG on Trade in November 2021 and August 2021 reviewed the implementation of the SPECA Trade Facilitation Strategy and this project. The state of implementation of the recommendations of the SPECA working paper on harmonizing border- crossing procedures and the SPECA Trade Facilitation Strategy were discussed at the 16th session of the SPECA Working Group on Trade on 18 November 2021 and the 17th’s session of the SPECA

43 (https://unece.org/sites/default/files/2021- 11/Trade%2C%20Water%2C%20Food%2C%20Circular%20Economy_Analytical%20background%20paper_ENG. pdf)

47

WG on Trade on 31 Aug.2022, and the group decided to update the roadmap for the implementation of the SPECA TF Strategy for the next sessions of the WG. A2.4.1: A web facility on exchange of information on TF issues among regulatory agencies in the region was developed under the project. With a view to strengthening the sustainability dimension of the project, recognizing the relevance of sustainable trade facilitation, in consultation with PMU and in line with UNECE’s cross cutting priority theme of the circular economy transition, an additional area of activity to be managed by the Director of the ECTD was added: EA3: Increased awareness of Central Asian countries on better integration into the international rules-based trading system: - A webinar Central Asia: New Ways, Old Traditions. Trade Policy Perspectives at the Geneva

Trade Week in September 2021; recording https://www.youtube.com/watch?v=MJyWyHDr6tg,

- Peer review consultations on the draft SPECA paper on implementation of Principles of Sustainable Trade in October 2021,

- National consultations with governmental representatives from Tajikistan on the draft SPECA paper on implementation of Principles of Sustainable Trade in November 2021,

- Webinar on Trade in Circularity: Perspectives of Developing and Transition Economies as part of the Trade and Sustainability Hub on 2 December 2021 featuring Ilkhom Khaydarov, the Chairman of Uztextileprom Association, recording https://www.youtube.com/watch?v=2b4cE21uvPY,

- Regional Policy Dialogue: Harnessing trade and economic cooperation for the transition to a circular economy in the UNECE region (online) (28 Oct. 2021). Meeting featured high-level opening by H.E.Mr. Alibek Kuantyrov, vice Minister of National Economy, Republic of Kazakhstan, Roundtable: Circular Economy - Fostering Circularity in Food Trade (online) (16 Nov. 2021).

The results of several SPECA events held from 20-24 November 2023 in Baku confirmed that its member states are committed to continue strengthening regional and national institutional platforms in Central Asia countries :

a. 20 Nov. SPECA WG on Trade 18th session: https://unece.org/sites/default/files/2023- 12/Recommendations_WG%20on%20Trade%202023_EN.pdf

b. 21-22 Nov. the 2023 SPECA Economic Forum on Trans-Caspian trade, transport and energy connectivity: https://unece.org/speca/events/2023-speca-economic-forum

c. 23 Nov. 2023: 18th session of the SPECA Governing Council: https://unece.org/speca/events/eighteenth-session-speca-governing-council

d. 24 Nov.2024: the SPECA Presidential Summit, which was an intergovernmental and not a UN meeting, but we prepared much input. The Presidents adopted the Baku

48

Declaration https://unece.org/speca/baku-declaration; and the SPECA Roadmap for the Digitalization of Multimodal Data and Document Exchange along the Trans-Caspian Transport Corridor, using UNITED NATIONS legal instruments and standards (https://unece.org/speca/speca-digitalization-roadmap), to which we contributed the essential part.”

Regarding the institutional strengthening at national level, SPECA countries received project’s support directly or indirectly through SPECA WGT for the establishing and functioning of national trade facilitation committees (NTFCs). Four of them (Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan) actually launched such bodies during the duration of the project and also established a regional network of these bodies. UNECE provided much information on its experience with these bodies and technical advice on their functioning. This was reflected in the tangible improvement of the performance of the countries in the reports on the implementation of the WTO Trade Facilitation Agreement44 and the UN Survey on Trade Facilitation and Paperless Trade.45

E293 provided support in the preparation of Trade Facilitation Roadmap for Kyrgyzstan, Tajikistan and Uzbekistan. These Roadmaps provide a framework to overhaul the countries’ trade facilitation reforms. They represent a strategic vision, developed jointly, and shared by the numerous national stakeholders whose engagement is vital in making trade facilitation a success. There is a positive prospect that the results of the project will be further strengthened after the project’s termination as UNECE continues to implement the E379 project “Building cross-border cooperation for sustainable trade in the SPECA sub-region”.

Rating: 4

4.2 Extent to which the partners and beneficiary countries “own” the outcomes of the project and the likelihood of the stakeholders’ engagement continuation, scaling up, replication and institutional strengthening

The interviews demonstrated that the project stakeholders and beneficiary countries own the outcomes of its activities and are committed to continuation, scaling up, replication and institutional strengthening with respect to trade facilitation.

In the framework of the project, UNECE provided support to specific SPECA countries in their efforts to apply trade facilitation principles to expand trade, including through WTO accession.

UNECE has a long history of cooperation with Central Asian economies. This work is done, inter alia, as part of the United Nations Special Programme for Economies of Central Asia (SPECA), launched in 1998, which is serviced jointly by UNECE and UNESCAP.

44 https://www.tfadatabase.org/en/implementation 45 https://www.untfsurvey.org/

49

SPECA, including Afghanistan, Azerbaijan, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan, aims to strengthen subregional cooperation in Central Asia and its integration into the world economy. Three of the seven SPECA participating countries are in the process of WTO accession. Most recently, the SPECA Working Group on Trade in August 2022 and the high-level SPECA meeting in the framework of the WTO Ministerial Conference in June 2022 reiterated the pledge to support Azerbaijan, Turkmenistan, and Uzbekistan on their way to WTO accession. All these countries benefited from the capacity building and advisory activities of E293.

Uzbekistan, for example, recently reaffirmed its commitment to intensify accession efforts following successful meetings of its WTO Accession Working Party. In June 2022, the UNECE Steering Committee on Trade Capacity and Standards46 discussed progress in trade reforms in the country, based on an UNECE study Policy Options for Harnessing the Power of Trade and Economic Cooperation47. As recent reforms increase export opportunities for the country, particular attention needs to be given to ensuring that non-tariff measures during the pandemic do not become trade barriers. In addition, UNECE has provided capacity building regarding this country’s innovation ecosystem as the government seeks to implement recommendations from the Innovation for Sustainable Development Review of Uzbekistan (I4SDR)48 launched in 2022.

In the case of Turkmenistan, the country applied in November 2021 to become a member of the WTO. Subsequently, in March 2022, UNECE organised a national capacity-building seminar on WTO accession for Turkmenistan and contributed to the multi-stakeholder meeting organised by Turkmenistan on the occasion of the visit of the WTO Deputy Director General in July 202249.

Turkmenistan has already played a role in fostering sustainable trade in Central Asia. In 2019, under its SPECA chairmanship, countries adopted the SPECA Principles of Sustainable Trade in Ashgabat. These principles, with a focus on diversification and innovation of production and trade, contribute to shaping sustainable and circular economy-focused regional policies - an essential component of the overall trade reform relevant to WTO accession.

The main results from the project supporting its sustainability include: 1. Increased local capacity in the four areas in the focus of the project:

a. National Trade Facilitation Bodies: NTFCs were established in Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan, and they are sustainable. NTFCs are being established in Azerbaijan and Turkmenistan.

b. Increase of national capacity to streamline documentary procedures. Work continues on digitalization of document exchange using the UN standards, and this work will deepen in the years to come. In this area UNECE continues collaboration with ESCAP, ITC, GIZ, USAID.

46 https://unece.org/circular-economy/news/removing-barriers-trade-facilitates-economic-integration 47 https://unece.org/sites/default/files/2022-05/ECE_CTCS_2022_05_E.pdf 48 https://unece.org/economic-cooperation-and-integration/publications/unece-innovation-sustainable-development- review 49 https://www.wto.org/english/news_e/news22_e/acc_05jul22_e.htm

50

c. The same applies with the use of international standards. UNECE has prepared guidelines on how the SPECA countries and their NTFCs can use the UN standards for trade facilitation and electronic business, and they report increased capacity to implement the standards in the UN Survey on Trade Facilitation and Paperless Trade and the WTO Trade Facilitation Agreement web page.

d. Single Window for export and import clearance: practically all countries in Central Asia and Azerbaijan have Single Window projects or operating Single Window systems, which they are committed to ameliorate. Much of this work was impacted by our project and related activities.

In addition, E293 supported work on the following tools which are available for use by the SPECA participating States, their business communities and business and development partners:

e. Rating tool and the regular surveys on trade facilitation and paperless trade (currently, when the World Bank Doing Business Report was discontinued, the UN Survey remains a major tool for measuring progress in trade facilitation in the region and elsewhere), and

f. Guide for establishing trade facilitation enquiry points: Uzbekistan was the first country to develop and launch a Trade Facilitation Enquiry Point using the Guide.

The SPECA Trade Facilitation Strategy and the Roadmap for its implementation are major instruments for the sustainability of the results of the project. The meetings of the SPECA WG on Trade listen to periodic reports on the stage of implementation of the SPECA TF Roadmap, which includes all measures covered by the project. SPECA member states will continue reviewing the progress of implementation of the roadmap and envisage updating it according to evolving needs. Another significant guarantee that stakeholders' engagement will continue and scale up is the fact that GIZ has decided to invest another 6.5 million euros in strengthening trade facilitation in Central Asia in the years 2024-2026. As a confirmation of ongoing processes of strengthening the institutional framework for trade facilitation in Central Asia, the following short review from Tajikistan can serve as an example: - Export Agency under the Government of the Republic of Tajikistan, http://www.export.tj/ Created by Decree of the Government of the Republic of Tajikistan dated August 6, 2018 No. 396. - State Unitary Enterprise “TAJAGROPROMEXPORT” was created by government decree under the Export Agency of Tajikistan on May 27, 2020 and its scope of activity includes the purchase, storage, packaging and export of competitive domestic products abroad. According to its mandate, the Agency has formed a database of exporters of the main export products. Promotes products through participation in exhibitions and fairs, produces booklets and video materials with export products - the Coordination Committee for Trade Facilitation has been created in Tajikistan and is functioning. Government Decree No. 529 of December 24, 2016 - On April 26, 2019, the Trade Portal of Tajikistan was officially opened https://tajtrade.tj - On September 1, 2020, the State Unitary Enterprise “Single Window” began operating to process export-import and transit operations. https://www.swcustoms.tj/

51

Structure of Coordinating Committee on TF Procedures in Tajikistan

Principal trade facilitation laws and regulations of Tajikistan as of January 2019 are available in the UNECE publication “National TF Roadmap of Tajikistan 2019-2024.

Rating: 4

Sustainability overall rating: 4,0

III. Conclusions and recommendations

1. Conclusions

1. The evaluation is forward looking and aims to assist UNECE in moving the future trade facilitation activities in the right direction, especially because these activities have already taken solid roots in the UNECE overall portfolio.

2. The objective of the project was to strengthen national capacities of the beneficiary countries to implement trade facilitation measures and better integrate into the international rules-based trading system. The majority of activities implemented, outputs produced and outcomes attained supported this objective.

3. Both, the Technical Cooperation Form and the Project Concept Form missed the description of the overall trade facilitation context (for example main trade facilitation needs and national counterparts) in the recipient countries. The evaluation made a special effort to close this gap by a detailed documentary review and the stakeholders` interviews. These efforts confirmed that E293 was a demand/needs driven project.

4. Overall, the project responded to beneficiary countries needs as well as to the expectation of trade facilitation stakeholders in Central Asia. It was well aligned with development priorities of recipient countries ’governments and other stakeholders’ needs and helped

52

support these countries to support their international obligations in contributing to several SDGs.

5. The project was fully aligned with the program of work of UNECE. Its approach and main target groups were in line with the UNECE mandate, mission and goals.

6. Key aspects of the project activities’ consistency with the sustainable development goals were well explored, while gender, human rights and disability perspectives have barely been touched. The evaluation did not find any evidence concerning the integration of gender, human rights and disability perspectives in the project document.

7. The project explored the UNECE added value in trade facilitation by concentrating on the four measures (see detailed references in the evaluation report) in the WTO Trade Facilitation Agreement, in which UNECE has decades of experience, mandate and comparative advantage.

8. Interviews/electronic survey results indicated that the project expected results were achieved fully in enhancing capacities of policy makers and experts in National Trade Facilitation committees and regulatory agencies to develop and implement trade facilitation policies and measures, notable in the area of WTO TFA Articles 10.1, 10.3, 10.4, and 23.2. Examples of results directly impacted by the project include, but are not limited to: the establishment of NTFCs in Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan, and they are sustainable; the ongoing process of establishing NTFCs in Azerbaijan and Turkmenistan; the existence, in practically all countries in Central Asia and Azerbaijan of Single Window projects or operating Single Window systems, which they are committed to ameliorate.

9. E293 expected results defined under EA.2 were achieved partially. Several interviewed stakeholders thought that the progress achieved in internal and cross-border policy coherence for the simplification and harmonization of trade procedures was less tangible as expected.

10. Several shortcomings linked to the interrupted continuity of the management of the project somewhat affected the evaluation’s ability to fully assess the effectiveness and efficiency of the project. Another shortcoming was the absence of a framework enabling a structured analysis of progress with respect to timing of delivery of expected activities and outputs. This situation resulted in a low interest of the project’s stakeholders to participate in the electronic survey and on-line interviews.

11. E293 activities main focus was on knowledge development, capacity building including development and dissemination of training materials/tools, organization of training seminars and regular support to SPECA Working Group on Trade to support the achievement of the SDGs through sustainable and efficient trade lines among the Central Asia countries and beyond.

53

12. The project was consistent with the interventions of other actors in the same areas, ensuring complementarity and coordination of the activities implemented. Regular dialogue with majority of the agencies supporting trade facilitation enhancement in Central Asia during the SPECA trade facilitation group meetings and various capacity building seminars facilitated harmonization of efforts in trade facilitation with other partners operating in the same context.

13. The project had to face important challenges/obstacles (the COVID 19 pandemic which impacted the implementation approach and strategy; several changes in project management responsibilities; temporal discontinuity of the project activities), the negative impact of which was not mitigated sufficiently.

14. The limited strategy for overcoming the obstacles/challenges that the project had to face resulted in large delays in the implementation of the planned activities and an extension of the project’s duration from two to six years.

15. While E293 had adequate financial resources to achieve the planned results and was cost- efficient (to a great deal thanks to synergies operated by the management with SPECA activities as well as the activities of other international organizations), it was implemented with unacceptable delays.

16. The SPECA Trade Facilitation Strategy and the Roadmap for the implementation of this strategy are major instruments for the sustainability of the results of the project since their implementation is supported by the political leadership of Central Asian countries.

17. In practical terms, the confirmation of sustainability was manifested in the improvement of the legal basis for trade facilitation in several Central Asian countries in the last 6 years (see example from Tajikistan above), as well as in the commitment of these countries in the preparation of national trade facilitation road maps ( completed in Tajikistan and Kyrgyzstan and work in progress in Uzbekistan) aligned with SPECA regional trade facilitation road map and the creation of the national trade facilitation committees (Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan).

2. Recommendations

1. UNECE, through its three core functions – development of norms, standards and legal instruments; hosting of a convening platform; and technical cooperation across a number of relevant sectors to continue trade facilitation related assistance to SPECA participating States, in particular with regards to standards and best practice recommendations for trade facilitation and electronic businesses to support Governments efforts to fulfill their obligations stemming from the WTO Trade Facilitation Agreement and/or WTO accession negotiations.

2. In the future, it is necessary to gradually move focus of UNECE trade facilitation capacity building activities from SPECA Working Group on Trade centered activities to training

54

workshops and hands-on training at the national level in five Central Asia countries including all line ministries involved in export/import activities.

3. Future trade facilitation activities should include a more robust discussion/awareness raising concerning trade impacts on sustainable development, such as SDG 5 (gender equality) or SDG 13 (climate action), etc., to better integrate considerations of cross-cutting issues such as gender, human rights, disability perspectives, climate change. The COVID-19 crisis has made it more difficult for women to participate in economic activities and trade. Focusing on gender sensitive challenges should be further supported and type of assistance broadened.

4. In any scenario for future UNECE support to its member states in trade facilitation, UNECE internal management should be strengthened to ensure sound project management from planning through delivery to reporting. The future projects’ planning should therefore explicitly consider this vital function and specify how this is to be ensured.

ANNEX 1: Evaluation matrix

Evaluation matrix – Evaluation of E293 Project

Evaluation question Indicators of success Data/Information

sources Data collection

methods RELEVANCE

1/ To what extent did the project respond to the priorities and needs of participating countries?

• Extent to which the project design, objective and result are aligned with the needs and priorities of recipient countries.

• Extent to which the recipient governments and local stakeholders were consulted during project design phase.

• Degree of alignment between the justification of the Project in Project Document and the Project outputs/outcomes.

Technical Cooperation Project Form, Project documentary database, Project design and implementation team members, technical collaboration partners, participating governments, focal points, consultants, and other key stakeholders

Documentary review, Stakeholder interviews, Electronic survey

2/ To what extent were the project activities consistent with global and regional priorities and aligned with the SDGs?

• Degree of alignment of the Project activities with global and regional priorities and with the SDGs.

Technical Cooperation Project Form, Project documentary database, technical collaboration partners, government focal points, consultants

Documentary review, Interviews of stakeholders, Electronic Survey

3/ How relevant were the project activities vis-à-vis the program of work of the UNECE? What value have UNECE efforts added in this area?

• Degree of alignment of the Project activities with the program of work of the UNECE and the level of value added by UNECE in trade facilitation approach in recipient countries.

Outputs/outcomes generated, technical collaboration partners, government focal points, consultants, Project management team

Documentary review, Interviews of stakeholders, Electronic Survey

4/ To what extent were gender, human rights, disability perspectives and climate change integrated into the design and implementation of the project? How can these perspectives be better included in future projects design and implementation?

• Extent to which the project design and implementation integrated human rights, disability perspectives and climate change.

• Degree of alignment of project design and implementation with relevant UN frameworks

• Number and quality of measures in project design and implementation that address gender/disability inequalities and climate change perspective

Project documentary database, Project design and implementation team members, technical collaboration partners, participating governments focal points, consultants, and other key stakeholders

Documentary review, Interviews of stakeholders, Electronic Survey

EFFECTIVENESS

5/ To what extent were the project design and set-up effective for meeting the needs

• Extent to which the Project design and set-up addressed

Technical Cooperation Project Form,

Documentary review

Evaluation question Indicators of success Data/Information

sources Data collection

methods of the beneficiary countries? the needs of the beneficiary

countries Implementation team members, technical collaboration partners, participating government focal points, consultants, and other key stakeholders

Interviews of stakeholders, Electronic Survey

6/ To what extent were the project objectives and expected results achieved?

• Degree of achievements of the planned objectives

• Extent to which the planned activities have been implemented

Project design and implementation team members, technical collaboration partners, participating government focal points, consultants, and other key stakeholders. Project documentary database.

Documentary review Interviews of stakeholders, Electronic Survey

7) To what extent are the project activities coherent and harmonized with those of other partners operating within the same context, particularly those of other UN system entities?

• Degree of coherence and harmonization of the Project activities with those of other partners operating within the same context, particularly those of other UN system entities

Project design and implementation team members, technical collaboration partners, participating government focal points, consultants, and other stakeholders, Project documentary database

Documentary review Interviews of stakeholders, Electronic Survey

8/What were the challenges/obstacles (including COVID-19) to achieving the expected results? How successfully did the project overcome these?

• Evidence of the challenges/obstacles (including COVID-19) to achieving the expected results.

• Evidence of enabling conditions/activities to overcome challenges/obstacles.

Project design and implementation team members, technical collaboration partners, participating government focal points, consultants, and other stakeholders, Project documentary database

Documentary review Interviews of stakeholders, Electronic Survey

EFFICIENCY

9/ Were the project resources adequate for achieving the results?

• Degree to which the financial and human resources were adequate for achieving the planned results.

• Specific evidence on achieving the expected project results can be cited

Project financial report, Project design and implementation team members, technical collaboration partners, participating government focal points, Project documentary database

Documentary review Interviews of stakeholders, Electronic Survey

10/ Were the results achieved on time and were all activities organized efficiently?

• Extent to which activities have been implemented according to planned schedule and

Project financial report, Project design and implementation team

Documentary review

Evaluation question Indicators of success Data/Information

sources Data collection

methods within the anticipated budget

and allocation of resources. • Evidence of efficient use of

available resources

members, technical collaboration partners, participating government focal points, Project documentary database

Interviews of stakeholders, Electronic Survey

11/ To what extent were the resources used economically and how could the use of resources be improved?

• Degree to which human and financial resources have been used wisely to achieve expected results.

• Perceived efficiency of program and project governance, management structures and processes

Project documentary database, Project design and implementation team members, technical collaboration partners, participating governments focal points, consultants, and other key stakeholders

Documentary review Interviews of stakeholders, Electronic Survey

SUSTAINABILITY 12/ What measures were adopted to ensure that project outcomes would continue after the project ended and to what extent have these measures addressed the existing risks for sustainability?

• Extent to which sustainability risks were identified and measures adopted to address these risks

• Degree to which the trade facilitation tools developed, capacities of experts in NTFCs enhanced and measures integrated into beneficiary countries activities

Project design and implementation team members, technical collaboration partners, participating government focal points, consultants, and other key stakeholders, Project documentary database,

Documentary review Interviews of stakeholders, Electronic Survey

13/ To what extent do the partners and beneficiaries ‘own’ the outcomes of the project? How is the stakeholders’ engagement likely to continue, be scaled up, replicated, or institutionalized?

• Extent to which the partners and beneficiaries ‘own’ the outcomes of the work

• There are “focal points” for trade facilitation measures development and implementation and routine procedures in the beneficiary countries to ensure implementation of TF measures knowledge transfer

• Evidence of engagements mentioned above and of tangible outputs

Project design and implementation team members, technical collaboration partners, participating government focal points, consultants, and other key stakeholders, Project documentary database,

Documentary review Interviews of stakeholders, Electronic Survey

Annex 2: Interview guide – Evaluation of E293 Project INTERVIEW GUIDE

DATE

ORGANISATION FUNCTION EMAIL

Dusan Zupka UNECE consultant Evaluator [email protected]

Documents Requested:

Documents Received:

Web Links:

Action:

Final Evaluation: E293: Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system 1. Introduction

2. Follow-Up

Justification Criteria (JC) Indicators (IND)

Evaluation Criterion (√)

Relevance EQ1 To what extent did the project respond to the priorities and needs of participating countries?

1.1 Design and activities of the Project

are aligned with relevant ECE policy and mandates and beneficiary countries policy

1.1.1 Specific evidence on policy can be cited

1.2 Design and activities of the Project are aligned with needs and priorities of beneficiary countries

1.2.1 Specific examples of alignment with needs and priorities can be cited

EQ2 To what extent were the project activities consistent with global and regional priorities and aligned with the SDGs?

2.1 Activities of the Project show consistence with global and regional priorities and alignment with the SDGs?

2.1.1 Specific examples of consistency and alignment can be demonstrated

EQ3 How relevant were the project activities vis-à-vis the program of work of the UNECE? What value have UNECE efforts added in this area?

3.1 Activities and results of the Project show relevance vis-à-vis the program of work of the UNECE and value added by UNECE efforts in trade facilitation?

3.1.1 Specific examples of relevance and added value can be cited

Justification Criteria (JC) Indicators (IND)

EQ4 To what extent were gender, human rights, disability perspectives and climate change integrated into the design and implementation of the project? How can these perspectives be better included in future projects design and implementation?

4.1 Design of the project integrated gender, human rights, disability perspectives and climate change

5.1.1 Specific evidence on integration of gender, human rights, disability perspectives and climate change in the design of the project

4.2 Implementation of the project integrated gender, human rights, disability perspectives and climate change

5.2.1 Specific examples on integration of gender, human rights, disability perspectives and human rights during the implementation of the project

Effectiveness

EQ5 To what extent were the project design and set-up effective for meeting the needs of the beneficiary countries?

5.1 Project design addressed the needs of beneficiary countries

5.1.1 Survey and interviews confirmed the satisfaction of the beneficiary countries with project design

EQ6 To what extent were the project objectives and expected results achieved?

6.1 Planned objectives and expected results have been achieved

6.1.1 Monitoring results indicating that planned objectives and expected results have been achieved

EQ7 To what extent are the project activities coherent and harmonized with those of other partners operating within the same context, particularly those of other UN system entities?

7.1 Project activities are coherent and harmonized with those of other partners operating within the same context, particularly those of other UN system entities

7.1.1 Monitoring results indicating that project activities are coherent and harmonized with those of other partners operating within the same context, particularly those of other UN system entities

EQ8 What were the challenges/obstacles (including COVID-19) to achieving the expected results? How successfully did the project overcome these?

8.1 Challenges/obstacles identified by stakeholders

8.1.1 Evidence from survey and interviews that the challenges/obstacles were overcome or not

Efficiency

EQ9 Were the resources adequate for achieving the results?

9.1 Financial resources have been sufficient for achieving the project expected results

9.1.1

Specific evidence on achieving the expected project results can be cited

Justification Criteria (JC) Indicators (IND)

9.2 Human resources have been sufficient for achieving the project expected results

9.2.1 Specific evidence on achieving the expected project results can be cited

EQ10 Were the results achieved on time and were all activities organized efficiently?

10.1 The expected results were achieved within the planned implementation period

10.1.1 Monitoring results confirming this

EQ11 To what extent were the resources used economically and how could the use of resources be improved?

11.1 Management cost was reasonable and implementation cost produced expected results

11.1.1 Monitoring results indicating economic use of resources

Sustainability

EQ12 What measures were adopted to ensure that project outcomes would continue after the project ended and to what extent have these measures addressed the existing risks for sustainability?

12.1 The trade facilitation tools developed, capacities of experts in NTFCs enhanced and measures integrated into beneficiary countries activities

12.1.1

Evidence of tools, procedures and measures that have been adapted by the beneficiaries and the existing risks for their sustainability have been identified

EQ13 To what extent do the partners and beneficiaries ‘own’ the outcomes of the project? How is the stakeholders’ engagement likely to continue, be scaled up, replicated, or institutionalized?

13.1 There are “focal points” for trade facilitation measures development and implementation and routine procedures in the beneficiary countries to ensure implementation of TF measures knowledge transfer

13.1.1 No. of focal points” for trade facilitation procedures and lessons learnt disseminated in the partners agencies in beneficiary countries

What preliminary lessons learnt can be identified to inform the planning of future UNECE interventions in trade facilitation assistance? Please name at least 2 lessons learnt:

-

-

What recommendations you propose for the future UNECE efforts to support the promotion and implementation

Of trade facilitation assistance in the beneficiary countries in general? Please propose at least 2 recommendations:

-

-

-

Electronic Survey Questions

Evaluation of E293 project

Strengthening the capacity of Central Asian countries to implement trade facilitation measures and better integrate into the international rules-based trading system * Indicates required question

Email * Your email

Relevance

1) To what extent did the project respond to the priorities and needs of participating countries? Highly Partially Little Not at all Please explain reasons for your ranking:..................................................................................... Clear selection Relevance

2) To what extent were the project activities consistent with global and regional priorities and aligned with the SDGs? Highly Partially Little Not at all Please explain reason for your ranking:......................................................................................... Clear selection

Relevance

3) How relevant were the project activities vis-à-vis the programme of work of the UNECE? What value has UNECE’s efforts added in this area? Highly Partially

Little Not at all Please explain reason for your ranking:....................................................................................... Clear selection

Relevance

4) To what extent were gender, human rights, disability perspectives and climate change considerations integrated into the design and implementation of the project? How can these perspectives be better included in future projects design and implementation? Highly Partially Little Not at all Please explain reason for your ranking:...................................................................................... Clear selection Effectiveness 5) To what extent were the project design and set-up effective for meeting the needs of the beneficiary countries? Highly Partially Little Not at all Please explain reasons for your ranking:.................................................................................... Clear selection

Effectiveness

6) To what extent were the project objectives and expected results achieved? Highly Partially Little Not at all Please explain reasons for your ranking:.................................................................................. Clear selection

Effectiveness

7) To what extent are the project activities coherent and harmonized with those of other partners operating within the same context, particularly those of other UN system entities? Highly Partially Little Not at all Please explain reasons for your ranking:....................................................................................... Clear selection

Effectiveness

8) What were the challenges/obstacles (including COVID-19) to achieving the expected results? How successfully did the project overcome these? Your answer Efficiency

9) Were the project resources adequate for achieving the results? Highly Partially Little Not at all Please explain the reason for your ranking:................................................................................. Clear selection

Efficiency

10) Were the results achieved on time and were all activities organized efficiently? Highly Partially Little Not at all Please explain the reason for your ranking:............................................................................... Clear selection Efficiency 11) To what extent were the resources used economically and how could the use of resources be improved? Highly Partially Little Not atll

Please explain the reason for your ranking and propose ways for improvement:...................... Clear selection

Sustainability

12) What measures were adopted to ensure that project outcomes would continue after the project ended and to what extent have these measures addressed the existing risks for sustainability? Highly Partially Little Not at all Please explain the reason for your ranking and propose ways for improvement:.................... Clear selection

Sustainability

13) To what extent do the partners and beneficiaries ‘own’ the outcomes of the project? How is the stakeholders’ engagement likely to continue, be scaled up, replicated, or institutionalized? * Highly Partially Little Not at all Please explain reason for your ranking:......................................................................................... Submit

Annex 4: Documents reviewed:

1) Project E293 Concept Note, UNECE 2017 2) Project E293 Technical Cooperation Project Form approved by EXCOM, 26.09.2017 3) Project E293 Annual Implementation report 2018 4) Project E293 Annual Implementation report 2019 5) Project E293 Annual Implementation report 2020 6) Project E293 Annual Implementation report 2021 7) Project E293 Annual Implementation report 2022 8) Project E293 Final Implementation report 2023 9) Project E293 list of consultants 10) Project E293 list of project managers 11) L.KISLYAKOVA(CONSULTANT): Report: Harmonization of Border Crossing procedures

in the SPECA countries 12) UNECE Guide to the implementation of Art.1 para.3 of the WTO TFA: Trade

facilitation enquiry points 13) UNECE study on regulatory and procedural barriers to trade in Uzbekistan, 2021 14) UNECE study on regulatory and procedural barriers to trade in Kyrgyzstan,2023 15) Reports from the Seminar on implementing specific provision of the WTO TFA, Nur-

Sultan, 20-22 May 2019 16) Reports from Webinar “Trade facilitation in times of COVID-19, 6-7 July 2020 17) Program from the Seminar on Strengthening the private sector capacity in TF,

Dushanbe, 29 June 2022 18) Report form USAID Central Asia Forum on Paperless Trade, 12 October 2022 19) Multiple exchanges of letters UNECE-Permanent Mission of the Russian Federation

concerning several project extensions 20) Case study Tajikistan Coordinating Committee on TF Procedures 21) UNECE Training Manual on National TF Committees 22) UNECE Guide to drafting National TF Roadmap1 23) UNECE Single Window training materials 24) UNECE Training Material on the USE of International Standards 25) UNECE Training Manual on CEFACT 26) UNECE Training Material concerning CEFACT, ECE/TRADE/C/CEFACT/2019/20 27) UNECE Training Material on National Trade Facilitation Bodies 28) E293 Expenditure report, 23.01.2024 29) GIZ Kyrgyzstan: Trade Facilitation in Central Asia executive summary 30) Evaluation of UNECE subprogrammes 4 and 6, 30 May-30 June 2023 31) National TF Report of Kazakhstan 2023 32) UNECE SPECA Webpage2 33) UNECE: The national TF Roadmap of Tajikistan 2019-2024 34) UNECE: National TF Roadmap of Kyrgyzstan 2021-2025 35) Draft of UNECE National TF Roadmap of Uzbekistan 2023-2026 36) UNECE Assessment report of the readiness of Kazakhstan to implement future

commitments arising from WTO TFA, February 2016

1 https://unece.org/fileadmin/DAM/trade/Publications/ECE-TRADE-420E.pdf 2 https://unece.org/speca

37) UNECE Assessment report of the readiness of Uzbekistan to implement future commitments arising from WTO TFA, 2021

38) UN General Assembly Proposed programme budget for 2018-2019, 2020, section 20/Programme 17 – Economic development in Europe

39) USAID Fact Sheet: USAID’s Trade Central Asia activities 40) UNECE/ESCAP: Report on progress of the implementation of the programme of work

of the SPECA WG on Trade, 2020 41) UNECE/ESCAP: Report on progress of the implementation of the programme of work

of the SPECA WG on Trade, 2021 42) UNECE/ESCAP: Report on progress of the implementation of the programme of work

of the SPECA WG on Trade, 2022 43) UNECE/ESCAP: Report on progress of the implementation of the programme of work

of the SPECA WG on Trade, 2023 44) SPECA Trade Facilitation Strategy, Ashgabat, 19.11.2019 45) SPECA Roadmap for Implementation of the SPECA Trade Facilitation Strategy,

Ashgabat, 19.11.2019 46) SPECA High-Level Seminar on Trade in Central Asia, WTO, Geneva, 4 July 2019

Annex 5: List of individuals interviewed

Name Organization

Mr. MARIO APOSTOLOV UNECE Project E293 manager Ms. LARISA KISLYAKOVA Former Deputy-Minister of Trade, Tajikistan,

Chair of SPECA WGOT, Chief Tajikistan accession to WTO negotiator

Ms. POLINA TARSHIS UNECE focal point for E293 Project administrative issues

Ms. AIZHAN BEISEYEVA Expert on trade facilitation, Kazakhstan, USAID Kazakhstan Advisor

Mr. BOCARL LIBERT International Expert, Sweden/Former UNECE staff

Mr. Lance Thompson Advisor trade facilitation, UNECE Ms. Nadezhda SPORYSHEVA Secretary, Steering Committee on Trade

Capacity/Standards Ms. ASEL UZAGALIEVA Acting Head, GIZ Trade Facilitation Project

Report on preconditions for establishing a strategic environmental assessment (SEA) database model in the countries of Eastern Europe and the Caucasus – key elements and recommendations

The Report was prepared under the “European Union for Environment” (EU4Environment – Green Economy) Action by the consultants to the UNECE secretariat to the Convention on Environmental Impact Assessment in a Transboundary Context (Espoo Convention) and its Protocol on Strategic Environmental Assessment (Protocol on SEA): Dmytro Skrylnikov and Dmytro Averin. Valuable information was provided by several countries in Eastern Europe and the Caucasus.
The publication is issued in English and Russian.

Languages and translations
English

Report on preconditions for establishing a strategic environmental assessment (SEA) database model in the countries of Eastern Europe

and the Caucasus – key elements and recommendations

April 2024

2 │

The Report was prepared under the “European Union for Environment” (EU4Environment – Green Economy) Action by the consultants to the UNECE secretariat to the Convention on Environmental Impact Assessment in a Transboundary Context (Espoo Convention) and its Protocol on Strategic Environmental Assessment (Protocol on SEA): Dmytro Skrylnikov and Dmytro Averin. Valuable information was provided by several countries in Eastern Europe and the Caucasus.

The publication is issued in English and Russian. Disclaimer This publication was funded by the European Union. Its contents are the sole responsibility of the secretariat of the United Nations Economic Commission for Europe (UNECE) and do not necessarily reflect the views of the European Union.

│ 3

Contents

1. Digital transformation in selected countries of Eastern Europe and the Caucasus ............. 4 1.1. Introduction ......................................................................................................................... 4 1.2. Armenia ............................................................................................................................... 4 1.3. Azerbaijan............................................................................................................................ 5 1.4. Georgia ................................................................................................................................ 6 1.5. Republic of Moldova ........................................................................................................... 7 1.6. Ukraine ................................................................................................................................ 9 1.7. Conclusion ......................................................................................................................... 11

2. Key elements for developing a model electronic SEA register ............................................. 12 3. National SEA procedures and legal prerequisites for the launch of electronic registers ... 19

3.1. Armenia ............................................................................................................................. 19 SEA legislative framework and stakeholders’ interaction within the SEA procedure ................... 19 Prerequisites for launching an electronic SEA register .................................................................. 20

3.2. Azerbaijan.......................................................................................................................... 22 SEA legislative framework and stakeholders’ interaction within the SEA procedure ................... 22 Prerequisites for launching an electronic SEA register .................................................................. 25

3.3. Georgia .............................................................................................................................. 26 SEA legislative framework and stakeholders’ interaction within the SEA procedure ................... 26 Prerequisites for launching an electronic SEA register .................................................................. 30

3.4. Republic of Moldova ......................................................................................................... 31 SEA legislative framework and stakeholders’ interaction within the SEA procedure ................... 31 Legal preconditions for launching an electronic SEA register ....................................................... 36

3.5. Ukraine .............................................................................................................................. 37 SEA legislative framework and stakeholders’ interaction within the SEA procedure ................... 37 Unified SEA Register ..................................................................................................................... 38

3.6. Conclusions ....................................................................................................................... 41 4. Recommendations ..................................................................................................................... 41

4.1. Legal recommendations ..................................................................................................... 41 4.2. Technical recommendations .............................................................................................. 42

Annex. Process mapping: block diagrams of processes/functions of SEA registers ...................... 44

4 │

1. Digital transformation in selected countries of Eastern Europe and the Caucasus

1.1. Introduction The Eastern Partnership Policy 2025 sets out long-term policy goals aimed at responding to new priorities, strengthening resilience to common challenges, promoting sustainable development and delivering concrete results for citizens, and one of the key objectives of this policy is to support the sustainable digital transformation of partner countries. Over the past 10 years, Armenia, Azerbaijan, Georgia, the Republic of Moldova and Ukraine have actively sought to modernize and integrate information systems and registries to improve the efficiency of public administration, the business environment and the quality of life of the population. These countries are implementing projects to digitalize State and municipal services, introduce electronic document management and develop digital infrastructure. The main areas of application of information systems and registries include health care, education, social security, statistics and environmental protection. The countries have various key national strategies and legislative acts for developing e-government, improving access to information and increasing transparency in the activities of government agencies, including in relation to environmental protection and environmental safety. 1.2. Armenia Armenia continues to actively develop information systems and registers in public administration and environmental protection. The Armenian e-government portal integrates e-government tools and various registries (https://e- gov.am/). The portal implements mechanisms for contacting government authorities, submitting applications for licences and reporting from licensees. The portal also contains links to a large number of sectoral portals of electronic services that are not combined into a single system, for example, the electronic system of government payments (https://www.e-payments.am/), the electronic system of real estate cadastre (http://www. e-cadastre.am/), the single site for publishing draft legal acts (http://www.e- draft.am/), the single platform for electronic requests (http://www.e-request.am/), the electronic registration of organizations (http://www.e-register.am/), the judicial information system (http://www.datalex.am/) and the electronic tax reporting system (https://file-online.taxservice.am/). There is no open data portal in Armenia, and public information is distributed through the websites of various government bodies and is available to citizens upon request. Many government agencies publish documents in non-machine-readable formats, making such information difficult to find and use. The country has a large number of platforms for publishing data and information about the environment. The website of the Ministry of Ecology (http://www.mnp.am/en) is available in Armenian, English and Russian. The site contains environmental reports organized by topics: atmosphere, water, waste, climate change, bioresources, forest, land and environmental monitoring. Reports on each topic are published in PDF and DOCX formats. A register of organizations affecting air quality is available on the website. The website of the Center for Hydrometeorology and Monitoring (http://armmonitoring.am/#home) provides environmental data on atmospheric air, surface and ground water, waste and soil. The site contains statistical data and provides access to thematic maps, tables and graphs. Part of the data from environmental quality stations is available online.

│ 5

The Water Committee management website (https://www.scws.am/) provides access to activity reports, special reports on water resources, issued permits, and spatial data of the State Water Cadastral Information System. The Statistical Committee portal (https://www.armstat.am/) publishes official statistical information. Data is posted in the form of reports in DOCX format. For the environment, only the annual report, Environment and Natural Resources of the Republic of Armenia, is available. More statistical data are available using the data visualization platform of the Statistical Committee (https://statbank.armstat.am/). The portal contains a considerable amount of environmental information, including UNECE (United Nations Economic Commission for Europe) indicators, environmental accounts, and mining data in machine- readable format. It also allows for cross-sectoral and time series data analysis. Despite the large number of implemented projects in e-government, insufficient integration of systems and the lack of a uniform data quality standard and an open data portal mean limited access to information. Solving these problems will require additional efforts on the part of the State. 1.3. Azerbaijan Azerbaijan is a country with a rapidly developing economy and a dynamically growing public sector. The country has made great progress in developing information systems for public administration. To implement the Decree of the President of the Republic of Azerbaijan of 14 March 2018, No. 1885 “On measures related to the development of e-government and the transition to digital government,” the E- Government Development Center was created under the State Agency for Citizen Services and Social Innovation. The Center cooperates with other government bodies on the formation of electronic government as the coordinator of the work and carries out activities to create the necessary infrastructure. The open data portal (https://opendata.az/en) was developed in 2015 by the Ministry of Digital Development and Transport. A new version of the portal is now available, in which the presentation of data is expanded and the search capabilities are improved. The data are provided free of charge and can be used without restrictions. The portal is in Azerbaijani, English and Russian, and includes an application planning interface (API) for data exchange. In total, more than 500 data sets from five institutions have been published on the portal, including 30 data sets in the “Ecology” section. The most recent data sets are from 2022. The electronic government portal (www.e-gov.az) provides online services for citizens and businesses, including issuing passports, registering businesses and filing tax returns. The portal is connected to the electronic register of public services and the State register of information resources and systems. In total, the portal supports 451 services, including 20 electronic services in environmental protection. Together with the portal, an e-government gateway was created to ensure the exchange of information among more than 40 government agencies connected to this infrastructure. Through the “electronic government” portal, the Ministry of Ecology and Natural Resources offers electronic services for providing information about the environment and natural resources, issuing permits for emissions of pollutants into the air, using water bodies, forest lands, allocating land plots for waste placement, disposal and treatment, issuing permits for hunting and industrial fishing, mapping, and provides conclusions on objects of State environmental expertise. Electronic services for conducting environmental expertise are also available on the portal of the State Agency for Environmental Expertise (https://eis.eco.gov.az/)

6 │

The website of the Ministry of Ecology and Natural Resources (http://eco.gov.az) provides information on biodiversity, hydrometeorology, geology, forests and natural parks. Most information is available in text and graphic non-machine-readable format. In most cases, the information published by the Ministry is limited to annual environmental reports. The website of the National Hydrometeorological Service (http://meteo.az/) provides access to interactive data on the state of atmospheric air and surface waters and serves as a common platform for exchanging environmental information. Based on the site, with technical support from the European Environment Agency, within the framework of the ENI SEIS II East project, the EcoPortal Water Information System was developed, providing access to data on surface waters and UNECE environmental indicators. The website of the State Statistics Committee (https://www.stat.gov.az/) is the main platform for the dissemination of statistical data, including environmental data. All statistics are available for download. Statistical data are also available through the ASIS portal (https://www.azstat.org/portal/). The portal currently provides access to 11 environmental data sets, which can be visualized chronologically and exported in a variety of machine-readable formats (XLS, CSV and TXT). The portal provides statistical data on national parks, water and forest resources, air pollution, waste, etc. Data on air pollution and water use are available on the interactive statistical data map (https://www.azstat.org/webmap/). 1.4. Georgia Georgia has a sufficient legislative framework that promotes the development of information systems and registers and regulates the main aspects of their development and use. The country continues to improve its communications infrastructure and internet access, but there are differences between levels of access in urban and rural areas. The Open Data Portal (https://data.gov.ge/) was developed by the Georgian Data Exchange Agency and is a unified national platform for publishing open data owned by government agencies. The portal contains more than 250 data sets in machine-readable formats across various sectors, including 8 environmental data sets. However, environmental data sets contain mostly outdated information and outdated information on the implementation of national plans. Georgia is developing e-government projects to improve the quality of its public services and increase the transparency of public administration. One such project is the creation of a single portal of public services (https://my.gov.ge/). The portal provides the opportunity to receive various government services online, including environmental services, social, medical services and payment of fines and utility bills. Through the portal, public information from government agencies and statistical information from more than 50 government agencies can be requested. The portal provides access to the classifier of economic activities, the register of economic activities, the register of enterprises, the register of notarial actions, the electronic register of the Agency for the Development of Public Services, the electronic register of notaries, the mortgage register, the leasing register, the address register, etc. There is no access to environmental registers and services through the portal. The website of the Ministry of Environment and Agriculture (http://mepa.gov.ge/) contains public analytical information, information on the services of agencies within the system of the Ministry, reports, plans, strategic documents, environmental legislation, register of agricultural cooperatives, environmental

│ 7

impact assessment (EIA) notifications, information on public hearings and air pollution monitoring data. Documents are available in non-machine-readable format only. The Ministry’s open data portal (http://data.mepa.gov.ge/) covers data of different categories, including land use, vegetation cover, biodiversity, risk levels and protected areas. The data are provided by a variety of organizations, including the National Forestry Agency, the Protected Areas Agency, the Department of Biodiversity and Forestry, and non-governmental organizations. Data are available in non-machine- readable format only. In total, the portal contains 52 documents, mainly in PDF. The information on the portal has not been updated since 2019. The website of the Environmental Information and Education Center (http://eiec.gov.ge/) publishes national reports and annual reports on the state of the environment, action plans, newsletters and a register of environmental organizations. The air quality portal (http://air.gov.ge/) provides access to information on air pollution, including real- time air quality index at air quality monitoring stations. The portal collects information on air pollution and supports decision-making on improving air quality. The portal provides information on air quality, daily, monthly and annual reports, as well as recommendations on health protection and protection from air pollution. The portal of the National Statistics Office of Georgia (https://www.geostat.ge/en) provides annual and quarterly statistical information, including on environmental protection. It provides access to the register of economic entities, data from agricultural and household surveys, and sectoral registers. The portal http://pc-axis.geostat.ge/ provides access to statistical data on air pollution, forest resources, protected areas and violations of environmental legislation. Data on UNECE environmental accounts and environmental indicators are also available in machine-readable format. The website of the Agency of Protected Areas (http://apa.gov.ge/) provides general information about protected areas in Georgia, their number and categories, as well as information about the Agency itself. The Geographic Information System portal of the Ministry of Environment Protection and Natural Resources (https://gis.mepa.gov.ge/portal/) provides access to thematic layers and maps of protected areas, land use, forests, geological risks and land degradation and melioration. The Forestry and Land Use Atlas of Georgia (https://atlas.mepa.gov.ge/) is an online monitoring platform that allows users to view and analyse data at national and regional levels. The atlas is managed by the Ministry of Environment and Agriculture with support from the World Resources Institute (WRI) and includes data from government agencies and non-governmental organizations. Georgia is currently developing a public environmental assessment portal, which will combine EIA and SEA procedures on a single platform. 1.5. Republic of Moldova The Republic of Moldova is actively implementing information systems and registers in various fields of activity, such as public administration, healthcare, education, statistics and environmental protection. The country has a number of key national strategies and legislation aimed at developing e-government, improving access to information and increasing transparency in the activities of government agencies. The Republic of Moldova is actively developing the infrastructure of e-government and information technology. It is implementing projects aimed at modernizing and optimizing public administration using modern technologies.

8 │

The open data portal (https://date.gov.md/) was launched by the Electronic Government Agency with the support of the World Bank in 2011 and updated in 2019. The portal provides access to 1,176 data sets, registers of legal entities and legislation. Through the portal, data on water use in the country, data from the national geospatial data fund, an electronic cadastre information portal, a public procurement register, a statistical data bank, a State register of legal acts and others are available. The Ministry of Environment publishes 35 data sets on the portal, including data on the quality of atmospheric air and surface water, emissions of pollutants and greenhouse gases, a register of water bodies, registers of organizations that have received permits to carry out activities causing environmental degradation and statistical data sets. The e-government portal (https://servicii.gov.md/en) was launched in 2012. Today, it provides information on 715 government services, 275 of which are available electronically. It gives access to all government services related to environmental protection, but there is no separate “environment” tab on the portal. Services are available when selecting “Ministry of the Environment”. The portal provides services for obtaining permits for emissions of pollutants from stationary sources, special use of water, import, export or re-export of ozone-depleting substances, permits for waste management, logging, fishing and hunting, services for conducting EIA procedures, and obtaining the conclusion of State environmental expertise. The Republic of Moldova has developed several thematic portals on which environmental data and information are published. The website of the Ministry of Environment (https://mediu.gov.md/en) provides information about the activities of the Ministry, its programme and regulatory documents. The automated waste management information system ( https://siamd.gov.md/portal/ ) was developed by the Ministry of Environment with the support of the Slovak Development and Cooperation Agency. The system provides access to the register of waste producers who are subject to the extended producer responsibility regime, the register of institutions and enterprises exempt from compliance with permitting requirements for waste disposal and waste placement activities, registers of manufacturers of packaging, batteries and accumulators, electrical and electronic equipment, oils, vehicles, etc. The National Bureau of Statistics (https://statistica.gov.md/) publishes thematic statistical reporting information, including aggregated information on environmental protection. Data of "Statbank" (http://statbank.statistica.md/) contains detailed tables of statistical information with time series. This tool allows selections in a variety of file formats to be created and saved, including machine-readable format. The service also provides access to data via an API for application developers. Environmental data are available in a separate section. The Bureau also publishes data on current expenditures on environmental protection and environmental payments, data on waste management, land and forest funds, meteorology, air protection and water use. The National Register of Emissions and Transfer of Pollutants (https://retp.gov.md/) provides systematic data on emissions of pollutants into the atmospheric air, water, soil from stationary and diffuse sources, as well as on emissions outside the facility and transfer of pollutants with wastewater. The automated information system “Register of chemical products placed on the market of the Republic of Moldova” allows for managing the processes of registration of chemical products, receiving and processing annual reports from manufacturers and importers of chemical products, as well as issuing permits for placing chemical products on the market. The system includes information on chemical products on the market, health risks, urgent measures in case of emergencies, physical and chemical

│ 9

properties of products, their biological activity, hygienic and environmental standards, including safety measures at the stage of production, import, export, transportation and use of chemical products. The Automated Information System for Management and Permitting (SIA GEAP) is used to automate environmental permitting at the Environmental Protection Agency. The Agency has also developed an automated information system for issuing permits “E-fishing” (https://e-pescuit.mediu.gov.md/), which is intended for obtaining permits for sport, recreational and contract fishing. The Republic of Moldova has also developed automated information systems, State Water Cadastre (https://csa.gov.md ) and State Cadastre of Fauna (https://am.gov.md/ro/node/590). The State Water Cadastre processes and stores data on water resources, hydrographic network, water bodies, hydraulic structures, protected areas, security zones and coastal water protection strips, as well as data on water intakes and water discharge sites, water balance data and information on river basin management. The State cadastre of fauna contains data on the number of livestock, data on game species and hunting resources, endangered, vulnerable, rare species of animals and vulnerable species listed in the “Red Book”. It is planned to create an electronic register of information on SEA, a register of Emerald Network facilities and a register of facilities subject to the requirements of Directive 2012/18/EU on the control of major- accident hazards involving dangerous substances. 1.6. Ukraine In recent years, Ukraine has been striving to modernize and integrate its information systems and registers. It is implementing projects for digitalizing State and municipal services, introducing electronic document management and developing digital infrastructure. Digital transformation is among the country’s priority goals, and the Ministry of Digital Transformation planned to ensure full access of the population and businesses to all government services online by 2024, to provide high-speed internet access for 95 per cent of public facilities and transport infrastructure, as well as increasing the share of digital products to 10 per cent of gross domestic product. In 2014, Ukraine launched the national open data portal (https://data.gov.ua) to provide access to publicly available information in the form of open data and access to information held by government agencies. Today, the portal has been modernized and contains more than 29,000 data sets, but on the topic “Environment” the portal contains only 7. Many data sets have not been updated since February 2022. The unified electronic services portal, “Diya”, (https://diia.gov.ua) provides online services for legal entities and individuals, which are grouped into categories: security and law and order, transport, land, construction, real estate, certificates, environment, health, documents and citizenship, and entrepreneurship. The national online platform “EcoSystema” (https://eco.gov.ua) is part of the unified portal of electronic services “Diya” and contains up-to-date information on the state of the environment, as well as providing access to environmental services and registers. A number of projects on environmental protection are being implemented on the basis of this platform. The project for the digital transformation of waste management “e-Vidhody” involves the creation of an electronic waste inventory and monitoring system. The service provides access to administrative services

10 │

and registers, including the State register of business entities that receive and/or dismantle scrapped vehicles, the register of licences for carrying out economic activities for the management of hazardous waste, the register of conclusions on transboundary transportation of waste, the register of declarations on waste, and the register of reports on transboundary movement of waste. The “e-Povitria” electronic service ensures the operation of an electronic system for recording pollutant emissions, as well as the introduction of electronic services for reporting and issuing permits for pollutant emissions. The service provides access to administrative services for State registration of facilities that have or may have an adverse effect on human health and the state of atmospheric air and the issuance of reference documents on background concentration values, access to a register of business entities that have approved emission limit values (specific emissions) of pollutant substances into the atmospheric air, a register of issued reference documents on background concentration values, a register of environmental auditors and legal entities entitled to carry out environmental audits. The “e-Lis” digital transformation project is aimed at creating an electronic system of State and public forest control. Today, the service provides access to administrative services to the population for issuing a special permit for the use of forest resources and for purchasing firewood. The digital transformation project for handling pesticides and agrochemicals “e-Pesticydy” involves the creation of an electronic inventory system for pesticides and agrochemicals. The service provides access to the register of pesticides and agrochemicals approved for use in the country. The “e-OVD” electronic service ensures digitalization of the issuance of conclusions on environmental impact assessment and automation of post-project monitoring. The service provides access to the unified State register for EIA (http://eia.menr.gov.ua ), a list of types of discussions and fees for holding public discussions during the EIA process, a list of authorized territorial bodies for conducting public discussions and a register of agreements for holding public discussions during the implementation of the EIA. The service also provides access to services for issuing conclusions on EIAs and registering contracts for providing services for conducting public discussions. The “e-Dovkillia” service involves providing automation of environmental monitoring by integrating information from various sources into a single analytical module. Today, the service ( https://ecozagroza.gov.ua ) has implemented functions for processing citizens’ requests and providing information about the impact of military operations on the environment. Interactive information about the background radiation at radiation monitoring stations is available. On the basis of the platform, it is also planned to implement projects on the digital transformation of biological and landscape diversity “e-PZF”, State enforcement in environmental protection “e- Ecocontrol”, strategic environmental assessment “e-SEO”, water management “e-Voda”, and use of subsoil "e-Nadrokorystuvannia". The Ministry of Environment Protection and Natural Resources is also implementing projects to create automated reporting systems under the Protocol to the 1979 Convention on Long-Range Transboundary Air Pollution Concerning the Control of Emissions of Nitrogen Oxides or their Transboundary Fluxes, concerning long-term financing of the cooperative programme for monitoring and evaluating the long- range transmission of air pollutants in Europe (EMEP) and the Protocol on Pollutant Release and Transfer Registers to the 1998 Convention on Access to Information, Public Participation in Decision-Making and Access to Justice in Environmental Matters (Aarhus Convention), which will become part of the common platform.

│ 11

Information on the state of surface waters is available on the “clean water” portal (http://texty.org.ua/water/). The portal is an interactive map of river pollution in Ukraine based on data from the State Water Agency. On the interactive map of river pollution indicators one can access information from more than 400 surface water quality stations on rivers including the Dnipro, the Danube and the Pivdennyi Bug. The website of the State Statistics Service (http://ukrstat.gov.ua) provides statistical data on the generation and management of waste, emissions of pollutants into the air from stationary sources of pollution, environmental protection costs, as well as environmental indicators recommended by UNECE. The website provides access to environmental accounts of air emissions and expense accounts for environmental protection. Civil society projects aimed at providing information about the state of the environment are active in Ukraine. One example of successful implementation is the SaveEcoBot System (https://www.saveecobot.com/), which aggregates open government data and data from public monitoring networks of air quality and background radiation in Ukraine. The Ecodozor portal (https://ecodozor.org/) – developed with the support of the Zoe Environmental Network (Switzerland), the Organization for Security and Co-operation in Europe, the REACH Humanitarian Initiative and the United Nations Environment Programme – provides information on the environmental effects and risks of military operations in Ukraine, including pollution risks due to disruption of industrial enterprises and critical infrastructure, soil damage, forest and grass fires. The Russian military invasion of Ukraine in February 2022 led to a change in information policy. Tasks related to data protection and cybersecurity took priority: modernizing personal data protection systems, ensuring the security of digital services, registries and information systems. At the same time, access to many information systems and registers, including those relating to environmental protection, was limited. For example, the website of the Ministry of Environmental Protection and Natural Resources (https://mepr.gov.ua/ ), which publishes information on the environment and the results of the Ministry’s work, is working only intermittently. The SEIS-Ukraine system is temporarily disconnected from the network (http://seis.menr.gov.ua/ ) to publish information on the dissemination of environmental information in Ukraine, including data from CORINE Land Cover, a “DEIS” system providing information on the environmental effects and risks of military operations in eastern Ukraine from 2014 to 2022. 1.7. Conclusions Despite the progress made, all countries face similar problems and challenges in the use of information systems and registries. One of the main problems is the lack of funding for the development and maintenance of information systems and registries. Budget constraints slow the pace of adoption of new technologies and reduce the quality of services provided. Some countries still have problems with access to the internet and the development of IT infrastructure in the regions, which make it difficult for the population to access information systems and registries, especially in remote rural areas. The shortage of qualified personnel in information technology and data management is another challenge for development. Countries need to take measures to improve their education systems and attract highly

12 │

qualified specialists into public administration structures and attract third-party developers to implement digitalization projects. Existing information systems and registries are often fragmented and do not provide effective interaction among different government agencies or local and central authorities. This is especially true for the integration of regional data and systems. In some cases, imperfections in the legislative framework and standards for working with data may hinder the development of information systems and registers in public administration. To ensure compatibility with international standards, countries need to constantly improve legislation, and review data quality standards, methods of processing and presenting information. The COVID-19 pandemic has had a mixed impact on the development of information systems and registries. On the one hand, countries were interested in information technologies to provide remote access to data; on the other hand, the pandemic in some cases led to a slowdown in economic growth and a decrease in budget revenues, which negatively affected the financing of digitalization projects. Fighting in the territories of some countries has led to limited access to data and a slowdown in progress in the digitalization of government and municipal services and the development of digital infrastructure. For security reasons, in some cases the work of information systems and registers was suspended, and access to data, including environmental information, was limited. Despite challenges and limitations, countries continue to develop and integrate information systems and registries for public administration and environmental protection. To improve the effectiveness of their information systems, countries need to overcome existing problems and ensure sufficient funding, coordination between government agencies and strengthened data protection measures. In general, all countries have a sufficient level of infrastructure development and human resources to implement projects in digitalizing the strategic environmental assessment process.

2. Key elements for developing a model electronic SEA register

The UNECE Protocol on Strategic Environmental Assessment to the Convention on Environmental Impact Assessment in a Transboundary Context (the Espoo Convention) was adopted in 2003 and entered into force in 2010. The Protocol defines the obligations of the Parties to evaluate likely environmental, including health, effects of certain plans and programmes, and, where possible, policies and legislation, at an early stage of their development. Article 10 of the Protocol also requires Parties to notify and consult with each other about plans and programmes that are likely to have significant transboundary environmental effects. The Protocol provides for a broad public participation in governmental decision- making, as well as early, timely and effective consultation with environmental and health authorities. The definition of strategic environmental assessment in the Protocol reflects the key elements of the strategic environmental assessment procedure. According to the Protocol, SEA means the evaluation of the likely environmental, including health, effects, which comprises the determination of the scope of an environmental report and its preparation, the carrying out of public participation and consultations, and the taking into account of the environmental report and the results of the public participation and consultations in a plan or programme (art. 2, para. 6).

│ 13

Articles 5 to 12 of the Protocol describe the process for carrying out an SEA of plans and programmes:

• Preliminary assessment (screening) (art. 5)

• Determining the scope of the SEA (scoping) to determine the content of the environmental report (art. 6)

• Environmental report (art. 7)

• Public participation (art. 8)

• Consultations with environmental and health authorities (art. 9)

• Transboundary consultations (art. 10)

• Decision to approve a plan or programme (art. 11)

• Monitoring (art. 12)

According to the SEA Protocol, at different stages of the SEA procedure an interaction occurs between various stakeholders, including authorities responsible for preparing and adopting relevant plans and programmes, authorities to be consulted with for reasons of their specific environmental or health responsibilities, the public, and, in the case of potential transboundary impact, other Parties to the Protocol.

The interaction may include notification, the exchange of documents or their publication, the provision of written comments, consultations, public hearings, various activities that ensure the monitoring of significant environmental, including health, effects of the implementation of plans and programmes and the communication of their results to the relevant authorities and the public (see figure 1).

14 │

Figure 1. SEA stakeholders’ interaction within the framework of the requirements of the SEA Protocol

│ 15

16 │

The Protocol explicitly recommends electronic media as a means of ensuring timely public access to documents (e.g. arts. 5.4 and 8.2). However, given the need to provide effective opportunities for public participation, using electronic media as a sole means of distributing documents may not be sufficient. This approach may exclude from participation important groups of the public who might not use the internet, including low-income individuals, isolated population groups (for example, in rural areas) and the elderly.1

Thus, providing access to information through electronic media does not replace or exclude the need to use other methods of informing; rather the use of various methods of disseminating information and documents, combined with informing through electronic media and ensuring the possibility of constant open access to documents on a publicly accessible electronic resource, could provide timely and most effective access to information in the public participation process within SEA.

The Meeting of the Parties to the Convention on Access to Information, Public Participation in Decision- making and Access to Justice in Environmental Matters (Aarhus Convention) at its seventh session (18– 20 October 2021, Geneva) adopted Updated recommendations on the more effective use of electronic information tools2 aiming to assist Parties, signatories and other interested States in supporting the implementation of the Aarhus Convention through the promotion of the development, maintenance, upgrade and use of electronic information tools by applying common approaches and standards. In addition, they will support efforts to implement other relevant international commitments.

In the Updated recommendations it is recommended to ensure where necessary through the introduction of appropriate legislative or regulatory measures, that subject to article 5 (10) of the Aarhus Convention:

(a) Public access to environmental information is provided in searchable electronic form and made available through the Internet, so that information required to be publicly available under the Aarhus Convention is to be provided in searchable electronic form where so requested and where the information exists in that form or can be readily converted to that form at reasonable cost;

(b) Documentation which is required to be drawn up and/or submitted in the context of decision-making procedures in environmental matters that are subject to the provisions of articles 6, 7 and 8 of the Aarhus Convention is required to be provided in electronic form and is progressively made accessible to the public through the Internet;

(c) A nationwide digital environmental information system supports public access to real-time and other dynamic and historical, up-to-date, accurate and quality-controlled, comprehensive, standardized and functional environmental information and this information is made discoverable and accessible through the Internet in machine-readable open forms and formats meeting the needs of different users;

(d) The following types of information are made publicly accessible, in a timely manner, through the Internet, preferably through a one-stop web access point:

(vii) Documentation related to strategic environmental assessment or other processes of preparing plans, programmes or policies relating to the environment subject to the provisions of article 7 of the Convention (for example, public notices, all other relevant documentation, including risk assessment and other studies, economic analysis and assumptions, comments of third parties, drafts and final decisions) where it is held in or can be readily converted to electronic form. Where it is not available in electronic form and cannot

1Information and Reference Guide for the Application of the UNECE Protocol on Strategic Environmental Assessment, ECE/MP.EIA/17, paragraph 81. 2 Updated recommendations on the more effective use of electronic information tools. ECE/MP.PP/2021/2/Add.2, https://unece.org/environment/documents/2022/02/updated- recommendations-more-effective-use-electronic-information

│ 17

be converted to electronic form at reasonable cost, a reference to where such documentation can be accessed (arts. 5 (3) (d) and 7 of the Aarhus Convention).3

One of the effective integrated solutions of using electronic media is the creation and operation of a centralized electronic SEA register. Such a register could be used to streamline SEA procedures and support access to information requirements in decision-making. It could be either a self-standing system or be integrated with other registers (e.g. EIA register) or be part of other registers or e-government systems.

The availability of documents and information online, in turn, simplifies the exchange of these documents between authorities at different stages of the SEA procedure, as well as providing access to them not only directly during the SEA procedure, but also in the future, when implementing plans and programmes, for example, for the purposes of monitoring significant environmental, including health, effects of the implementation of plans and programmes.

The use of a centralized SEA register helps to systematize SEA procedures at all levels, improve the exchange of information at the local and central levels, especially in those countries where there is a division of powers for conducting SEA at the local and national levels. Furthermore, a centralized SEA register generally contributes to the transparency of the planning system and strategic environmental decision-making processes at the national level, improves the quality of SEA, and contributes to more effective implementation of the reporting obligations under Article 14.7 of the SEA Protocol4.

Thus, as a model electronic register, it is proposed to consider a centralized electronic register, which is an electronic information or information and communication system that ensures interaction between SEA stakeholders, collection, storage, systematization of information and documents that originate in the SEA process, and during the monitoring of significant environmental, including health, effects of the implementation of plans and programmes, as well as publication and/or open access to information and documents (subject to relevant requirements and restrictions provided for by law).

The key functions of a model electronic register are the following:

• arrangement of information and documents on SEA into a single electronic structured database on SEA at all levels (local/regional and national) searchable by certain parameters

• ensuring the exchange of information and documents between the SEA stakeholders (including administering the comments and suggestions from the public and authorities)

• ensuring timely public notification

• ensuring the publication of information and documents

• constant open access to information and documents on all SEA procedures, as well as information on monitoring of significant environmental, including health, effects of the implementation of plans and programmes

• posting information within the framework of transboundary consultations (if carried out).

3 Ibid, paragraph 23. 4Among the findings of the Fourth Review of the implementation of the Protocol on Strategic Environmental Assessment (2019–2021), which is to be confirmed by the Meeting of the Parties to the Protocol at its fifth session (Geneva, 12–15 December 2023), it is acknowledged that the lack of a central registry or database of national strategic environmental assessment procedures in many Parties made reporting on the number of procedures during the survey period complicated and imprecise. See https :// unece . org / environment / documents /2023/10/ session - documents / fourth - review - implementation - protocol - strategic

18 │

Depending on technical solutions and capacity, in addition to the basic functions, the electronic register could have more advanced features, including additional capabilities for searching and analysing information, supporting public discussions/hearings, integration with a cartographic framework, as well as contain links to legislation and other databases and sources of information and so on. For example, new amendments to the EIA legislation in Ukraine provide for the introduction of a direct mailing service into the Unified EIA Register. It is assumed that individuals and legal entities will be able to register in the Unified EIA Register for the purpose of targeted notification on the publication of information and documents therein.

│ 19

3. National SEA procedures and legal prerequisites for launching electronic registers

This part examines the legislative framework for SEA in Armenia, Azerbaijan, Georgia, the Republic of Moldova and Ukraine, as well as national SEA procedures in the context of the interaction of various stakeholders, the exchange and publication of information through electronic means of communication.

This report does not aim to assess the compliance of SEA procedures in the countries covered with the SEA Protocol but is intended to identify the needs and prerequisites for the launch of electronic SEA registers in the countries concerned. In relation to Ukraine, the recently introduced Unified SEA register is being considered. The figures presented in this section are intended to illustrate the interaction of stakeholders at various stages of SEA and the publication of documents and information through electronic means of communication. The figures were developed based on the analysis of legislation and information provided by countries5, and are demonstrative in nature and may not fully describe all aspects of interaction, as well as the established practices of conducting individual procedures in the countries.

3.1. Armenia

SEA legislative framework and stakeholders’ interaction within the SEA procedure

The legislative framework for conducting SEA in Armenia is the 2014 Law on Environmental Impact Assessment and Expertise (hereinafter referred to as the Law on EIA and Expertise).6

According to this Law, SEA means the process of determining and evaluating the effects on the environment of the implementation of provisions of a draft fundamental document, which must include determining the scope and provisions of the SEA report, preparing a SEA report, ensuring participation of the public concerned and holding professional consultations, taking into account the provisions of the SEA report, the results of public participation and consultations in the fundamental document.

The fundamental document, in accordance with that Law, is a document that has potential effects on the environment (strategy, concept paper, scheme for the use of natural resources, project, plan, layout, urban development programme document) or any amendment to the document approved by national laws or other legal acts of State bodies or local self-government bodies.

The SEA and expertise of a draft fundamental document are carried out before the adoption of the fundamental document.

According to article 2.2 of the Law, the stages of SEA are:

• determination of the scope of SEA

• preparation of the SEA report

• incorporation of the results of SEA in the fundamental document.

At these stages, a State body or a local self-government body submitting a draft fundamental document (hereinafter referred to as “the initiator”) is obliged to:

5 National focal points for the Espoo Convention and its SEA Protocol. 6 This report examined the Law as amended on 3 May 2023.

20 │

(a) contact the authorized body7 and the authorized body in the field of healthcare for consultations, as well as the authorized State bodies in the areas in which the fundamental document is being developed, or local self-government bodies;

(b) involve the public concerned, and, if necessary, carry out transboundary consultations.

The expertise of the SEA report is carried out within 80 working days from the date of submission, with a cover letter to the authorized body (except for the documents with transboundary effect, for which the timeframe for providing a conclusion may be extended). On this basis, the authorized body provides the initiator with a positive or negative expert conclusion. Approval of a fundamental document is prohibited without a positive conclusion of the State expertise.

The authorized body, local State bodies, local self-government bodies and the initiator, to ensure the participation of the public concerned in the SEA and expertise procedure:

(a) disseminate information about the right to initiate and participate in the SEA and expertise procedure;

(b) provide access to documents on SEA, expertise and other information;

(c) facilitate participation of the public concerned in hearings, inform the public concerned about a draft fundamental document, the progress of the expertise through official websites and other means of disseminating information (press, other media, notice boards of local self-government bodies, public buildings) (See figure 2).

Prerequisites for launching an electronic SEA register On 11 February 2021, the Government approved resolution №183-L on the Digitalization Strategy of Armenia, the Strategy Action Programme and Results Indicators, which acknowledges the need for the development of digitalization in various areas and outlines the main directions and approaches to digitalization for the period until 2025.

In terms of SEA legislation, the establishment of a centralized electronic register on SEA or a unified electronic database/portal of documents on SEA is not provided for by the Law on EIA and Expertise. That Law provides for the exchange of information and the submission of comments both on paper and in electronic format. Informing the public and providing access to documentation is provided through the websites of the authorized body, local State bodies, local self-government bodies and the initiator.

7 Authorized body - a public authority developing and implementing the Government policy in the field of ensuring the State environmental expertise.

│ 21

Figure 2. Stakeholders’ interactions in the SEA and expertise procedure

22 │

3.2. Azerbaijan

SEA legislative framework and stakeholders’ interaction within the SEA procedure

The legislative basis for conducting SEA in Azerbaijan is the 2018 Law on Environmental Impact Assessment as well as by-laws adopted for the purpose of its implementation, including the Regulation on conducting the State and public environmental assessments, approved by a resolution of the Cabinet of Ministers No. 184 dated 21 May 2020, as well as the Procedure for conducting the strategic environmental assessment, approved by a resolution of the Cabinet of Ministers No. 354 dated 15 September 2022, which directly regulates the SEA procedure.

The Law on Environmental Impact Assessment defines SEA as a systematic assessment of strategic documents, territorial planning documents from the point of view of environmental safety to identify and prevent potential effects on the environment and human health and ensure the effective use of the environment. An SEA is carried out by government authorities, government agencies and municipalities that prepare strategic documents (hereinafter referred to as “planning authorities”) at the stage of preparation of strategic documents. The Law considers plans, programmes, strategies and concepts in the areas of agriculture, forestry, fisheries, energy, industry, transport, waste and water management, land use, tourism and telecommunications to be strategic documents.

The SEA process commences when one contacts the State Agency for Environmental Expertise under the Ministry of Environment and Natural Resources for preliminary consultations and concludes with the issuance of a conclusion of environmental expertise.

The legislation of Azerbaijan establishes the following stages of a SEA:

• identification of the scope of the SEA and preliminary consultations with the Agency

• analysis and assessment of possible effects in accordance with the requirements of normative legal acts

• specification of environmental protection measures according to the collected data

• development of the SEA report

• organization of public consultations

• preparation of appropriate final justification for the findings of the planning authorities

• approval of the strategic document by the Ministry of Environment and Natural Resources

• monitoring of SEA results.

At the stage of preliminary consultations on SEA and determining the scope of the SEA, the planning authority carries out preliminary consultations with the Agency in order to determine the content, scope and methods of the SEA and to ensure the completeness and accuracy of the information to be included in the SEA report. Preliminary consultations are initiated by the planning authority at the stage of preparation of the strategic document by submitting an application to the Agency for notification of planning intentions and defining the scope of the SEA. Once the Agency approves the SEA scoping document, the planning authority posts the document on its website along with the draft strategic document and informs the public (see figure 3).

Figure 3. Preliminary consultations on SEA and determination of the scope of SEA

│ 23

At the next stages, the planning authority prepares the SEA report and submits it to the Agency for environmental expertise. When a planning authority submits a draft SEA report to the Agency, it must also advise of the date, time and location of the public consultations.

An Agency representative must be formally notified by the planning authority of the public hearing at least seven working days in advance to ensure that an Agency representative is present at the consultations. Also, the planning authority, at least seven working days before the consultations, publishes on its website, as well as in the media, an announcement on public consultations and the draft SEA report, the purpose, date, place and time of the meeting, its format, as well as information on the venue where the relevant documents can be reviewed.

Public hearings on strategic documents of regional importance are organized in several territorial units (districts and cities); individuals and legal entities within the relevant administrative-territorial units, as well as property owners, are involved in and informed of the consultations.

The conclusion of the environmental expertise issued for the SEA report is posted on the website of the Agency (see figure 4).

24 │

Figure 4. Review of the SEA report: public consultations and environmental expertise

The strategic document is drawn up taking into account consultations with relevant governmental authorities and public consultations on SEA, as well as opinions and proposals submitted after the conclusion of the State environmental expertise is issued.

The planning authority prepares a final justification document on the issues considered and not considered in relation to the proposals on the strategic document and submits the strategic document to the Ministry for approval, which must be accompanied by an SEA report or a brief overview of its final findings, the conclusion of the State environmental expertise and the final justification document of the planning authority.

After approval of the strategic document, the planning authority summarizes the final results received, including the SEA findings and the conclusion of the environmental expertise, posts them on the website and informs the Agency about this in writing (see figure 5).

│ 25

Figure 5. Final decision on SEA

Prerequisites for launching an electronic SEA register The legal framework for the introduction of electronic government in Azerbaijan was established through the Presidential Decree “On approval of the State programme (E-Azerbaijan) for the development of communications and information technologies in the Azerbaijan Republic in 2010–2012”, as well as the and the Presidential Decree “On approval of some measures for the provision of electronic services by government agencies” dated 23 May 2011.8 Subsequently, several Presidential decrees and other regulatory legal acts were adopted, which determined the strategies for developing e-government and digitalization in the country, as well as their individual aspects. For example, the Presidential Decree dated 14 March 2018 “On measures related to the development of electronic government and the transition to digital government”.

In Azerbaijan, there is currently no centralized electronic database of SEA procedures and relevant documents within the framework of such procedures. Moreover, from the interaction patterns discussed above, it appears that even within the framework of one SEA procedure some documents are posted on the planning authority’s website and some on the Agency’s website.9 At the same time, it should also be taken into account that planning authorities encompass various government bodies, State agencies and municipalities that prepare strategic documents, and accordingly, only information within their purview is posted on their websites.

8 https://www.e-gov.az/en/content/read/2 9According to the information provided by Azerbaijan as part of the preparation of this report, the Agency’s website is currently unavailable.

26 │

3.3. Georgia

SEA legislative framework and stakeholders’ interaction within the SEA procedure

The legislative basis for conducting an SEA in Georgia is the 2017 Environmental Assessment Code.10 According to the Code, an SEA includes scoping, the preparation and review of an SEA report, public participation and consultations, taking account of the information included in the SEA report in the decision-making on a strategic document and taking account of recommendations issued by the National Environmental Agency11 and the Ministry of Internally Displaced Persons from the Occupied Territories, Labour, Health and Social Affairs of Georgia (hereinafter - the Ministry of Health) in relation to the draft strategic document and the SEA report, taking account of the results of public participation during the adoption/approval of the strategic document, as well as the provision of information on the decision made to the public and concerned agencies.

The stages of SEA, according to the Code, include:

• submission of an application by the planning authority12 to the Agency and the Ministry of Health

• scoping procedure

• preparation by the planning authority and/or consultant of a SEA report

• the planning authority's assessment of the information obtained from the SEA report, public participation and consultation

• if necessary, performance of the transboundary environmental impact assessment procedure

• public participation

• issuance of recommendations by the Agency and the Ministry of Health in relation to the draft strategic document and the SEA report.

To determine the need for an SEA (minor changes in the strategic document, as well as the strategic document relating to the territory of the self-governing community), a preliminary assessment (screening) procedure is carried out (see figure 6). In such cases, the planning authority applies to the Agency and the Ministry of Health with an application for screening and submits a concept or draft strategic document (it should contain brief information on the aims, objectives and activities provided for in the strategic document) as early as possible in its development both in paper and electronic format.

The Agency sends the application and concept or draft strategic document to the Centre for Environmental Information and Education.13 The Centre ensures their posting on the environmental information portal. Also, the Ministry of Health and the planning authority post the application and concept or draft strategic

10This Report considered the edition of the Code as of 29 June 2023 - https://matsne.gov.ge/ru/document/view/3691981?publication=12 . 11A legal entity of public law, part of the system of the Ministry of Environment Protection and Agriculture of Georgia. 12Planning authority – an administrative body or other authorized organization that, according to the relevant regulatory act, is responsible for preparing a strategic document. 13A legal entity of public law, part of the system of the Ministry of Environment Protection and Agriculture of Georgia.

│ 27

document on their websites, and the Centre also ensures that the screening application is posted on the information board of the executive body and/or representative body of the respective municipality.

Once the screening statement and concept or draft strategic document are posted on the websites and information board, the public has the right to submit opinions and comments about these documents within a specified period. The Agency and the Ministry of Health consider the opinions and comments submitted by the public and, if there are appropriate grounds, take them into account in the decision-making process.

After making a screening decision, the Agency forwards the decision, as well as opinions and comments submitted by the public, to the Centre, which ensures that they are posted on the environmental information portal. Also, after making screening decisions, the Ministry of Health and the planning authority post these decisions, as well as opinions and comments submitted by the public, on their websites, and the Centre also ensures that these documents are posted on the information board of the executive body and/or representative body of the respective municipality.

Figure 6. Screening

A similar algorithm for the interaction of the SEA stakeholders and the publication of information using electronic resources is provided at the scoping stage (see figure 7). The exception being the provisions on the posting by the Centre, the Ministry of Health and the planning authority of opinions and comments provided by the public during the scoping process. The Code does not specify in the scoping procedure, in contrast to the screening procedure, the posting of public opinions and comments on the relevant electronic resources; however, it stipulated that the Agency and the Ministry of Health consider the opinions and comments submitted by the public and, if there are appropriate grounds, take them into account in the decision-making process.

The Code provides for the possibility to combine the stages of screening and scoping and submit simultaneously an application for screening and an application for scoping.

Also, in the case of a transboundary environmental impact assessment procedure, the Agency determines the obligation for the planning authority to reflect in the scoping report/scoping application the results of consultations related to the transboundary impact assessment procedure.

28 │

Figure 7. Scoping

After the planning authority prepares an SEA report, the procedure begins for the Agency and the Ministry of Health to review the report and issue recommendations (see figure 6). The planning authority applies to the Agency and the Ministry of Health and submits the SEA report and the draft strategic document in both paper and electronic format. The application also contains information on the proposed time, place and procedure for holding a public discussion.

The Agency sends this application and attached documents to the Centre. The Centre ensures that these documents are posted on the environmental information portal. The Ministry of Health and the planning authority post the application and the attached documents on their official websites, and the Centre ensures that it is posted on the information board of the executive body and/or representative body of the relevant municipality.

Also, in order to review the SEA report and the draft strategic document, the Agency creates an Expert Commission, which submits an expert opinion to the Agency.

The public has the right to submit suggestions and comments to the Agency and the Ministry of Health in relation to the SEA report and the draft strategic document. The Agency and the Ministry of Health review public submissions and, if there are appropriate grounds, take them into account in the decision-making process.

The planning authority also holds public consultations on the SEA report with the participation of the Agency and the Ministry of Health. After a public discussion on the SEA report, the planning authority ensures that a protocol is drawn up on the results of the public consultations on the SEA report and submits the protocol on the results of the public consultations on the SEA report to the Agency and the Ministry of Health.

Within their purview the Agency and the Ministry of Health issue appropriate recommendations in relation to the SEA report and the draft strategic document and forward these recommendations to the planning authority.

│ 29

The Agency also forwards recommendations issued in relation to the SEA report and the draft strategic document to the Centre. The Centre ensures that they are posted on the environmental information portal. The Ministry of Health and the planning authority publish these recommendations on their websites, and the Centre also ensures that these recommendations are posted on the information board of the executive body and/or representative body of the relevant municipality.

In the case of a transboundary environmental impact assessment procedure, the Agency ensures that the results of transboundary consultations are taken into account in the process of issuing recommendations.

Figure 8. Review of the SEA report and a draft strategic document

After the adoption/approval of the strategic document, the planning authority ensures that the decision on the adoption/approval of the strategic document is posted on its website and sends the decision to the Agency and the Ministry of Health.

The Agency forwards the decision to the Centre. The Ministry of Health is obliged to post the decision on the adoption/approval of the strategic document on its website, and the Centre ensures that information on the decision is posted on the environmental information portal and information board of the executive body and/or representative body of the relevant municipality.

30 │

If there is a need and possibility, the Agency, based on the data available to it, ensures the performance of an independent analysis of significant effects on the environment resulting from the implementation of the strategic document and provides the Centre with the results of this analysis. In order to inform the public, the Centre ensures systematic posting of the results of further analysis of the results of the implementation of the strategic document on the environmental information portal.

Prerequisites for launching an electronic SEA register In accordance with paragraph 2(d) of article 4 of the Code, the purview of the Ministry of Environmental Protection and Agriculture of Georgia includes the creation of a unified database on issues within the purview of the Ministry, and ensuring the security, openness and accessibility of the information stored within, in order to ensure access to information, government effectiveness and public participation.

Currently, SEA documents are submitted both digitally and in paper format. Digital materials are posted on the websites of the authorities and the Agency’s website and are sent to departments/various organizations for review. Thus, all administrative procedures are carried out digitally. However, special software/IT tools are not used to ensure greater digitalization of the SEA procedure. Also, currently there is no centralized electronic database of SEA procedures and relevant documents within the framework of such procedures. Moreover, from the interaction patterns discussed above, it is concluded that even within the framework of one SEA procedure some of the documents are posted on the website of the planning authority or the portal, when the part related to health is posted on the website of the planning authority and the Ministry of Health. It should be taken into account that planning authorities encompass various administrative bodies or other authorized organizations responsible for preparing strategic documents, and accordingly, information related only to their purview is posted on their websites.

From 1 July 2023, the Centre for Environmental Information and Education (www.eiec.gov.ge) is responsible for involving the public in the decision-making process in the procedures provided for by the Code, ensuring access to information and organizing public hearings.

According to the information provided by Georgia as part of the preparation of this report, Georgia is developing a public environmental assessment portal, which will combine EIA/SEA documents on a single platform and provide the public with systematic and simplified access to information on EIA and SEA procedures.

│ 31

3.4. Republic of Moldova

SEA legislative framework and stakeholders’ interaction within the SEA procedure

The SEA procedure in the Republic of Moldova is regulated by the 2017 Law on Strategic Environmental Assessment with appropriate amendments.14

According to the SEA Law, an SEA is an assessment of effects that policy and planning documents may have on the environment, including public health, which involves determining the scope of the SEA report and its preparation, carrying out consultations with the public authorities concerned and the public concerned, as well as carrying out transboundary consultations where appropriate and taking into account the conclusions of the SEA report and the results of public participation and consultation in a policy and planning document

The SEA Law refers to policy and planning documents as public policy documents and planning documents, which: (a) are to be prepared according to the provisions of normative or administrative acts; (b) are to be prepared by a public authority and approved by the Government or, by means of a legislative procedure, by the Parliament; and (c) are to be prepared and approved by a local public administration authority.

The preliminary assessment (determining the need for an SEA) and an SEA are carried out at the initial stage of the development of a concept of the policy and planning document or of its modification.

The SEA process includes the following stages:

• defining the scope of the SEA report

• preparing the SEA report

• analysing the quality of the SEA report

• analysing the integration of environmental aspects into the draft policy and planning document

• issuing an environmental opinion15

• approving the policy and planning documents

• monitoring the significant impact of the policy and planning document on the environment, including on public health.

Among the obligations of the initiator and the competent authority the SEA Law provides for the posting of information on the SEA procedure on their websites.

In accordance with the SEA Law, the initiator is the central or local public administration authority that initiates policy or planning documents and is responsible for developing them. The competent authorities, according to the SEA Law, are the Ministry of the Environment (MoE) - for national-level policy and planning documents, including sectoral and cross-sectoral documents approved by the Government or

14This report examined the law, including the amendments dated 30 September 2022, which came into force on 21 October 2023. 15Environmental opinion is a technical-legal document issued by the competent authority confirming that the SEA report complies with the requirements of the SEA Law, and the aspects related to environmental protection are integrated into the policy or planning documents submitted for approval and do not have adverse effects on the environment, including public health.

32 │

Parliament and the Environment Agency (EA) - for local level policy and planning documents, approved by the local public administration.

The initiator, under the guidance of the competent authority, informs (through public announcements in local and national newspapers and posting information on official websites) and organizes the participation of the public concerned at all stages of the SEA. At all stages, too, the initiator posts and stores information on the SEA procedure on its official website in the section dedicated to SEA.

For policy and planning documents developed in the areas of agriculture, forestry, fisheries, energy, industry, transport, waste management, water management, electronic communications, tourism, land use, urban and rural planning (urban planning and landscaping documentation, including plans for urban planning and development of the territory and programmes for urban planning and development of the territory at the national, regional, district, local levels, as well as for general and zonal urban plans), which establish the framework for the implementation of planned activities subject to environmental impact assessment, the SEA procedure begins with defining the scope of the SEA report.

Where these policy and planning documents relate to the use of small areas at the local level or provide for minor changes to the policy and planning documents, and where policy and planning documents other than those above establish the framework for the implementation of any type of planned activity for which a permit is required, a preliminary assessment is carried out (determining the need for an SEA).

To carry out a preliminary assessment, the initiator, at the stage of developing policy or planning documents, submits to the competent authority the information necessary for the preliminary assessment. The competent authority publishes this information on its official website and initiates consultations with central and local public authorities identified as public authorities concerned (PAC) and with the Ministry of Health (MoH).

The initiator informs the public concerned. The public concerned may submit their comments and suggestions to the competent authority within 10 days from the date of notification.

The competent authority, taking into account the opinion of the public authorities concerned, as well as comments and suggestions received from the public concerned, makes a decision on preliminary assessment, sends it to the initiator and posts it on its official website, as well as at its headquarters.

The initiator, after receiving a decision on preliminary assessment, posts it on its official website and informs the public concerned regarding the decision on preliminary assessment, including through public announcements in local and national newspapers (see figure 9).

For policy and planning documents in relation to which a decision to carry out an SEA was taken, the competent authority determines the scope of the SEA report and issues the decision on the scope of the report in accordance with the scoping requirements.

│ 33

Figure 9. Preliminary assessment

A similar algorithm for the interaction of SEA stakeholders and the publication of information using electronic resources is also provided at the stage of determining the scope of the SEA report (see figure 10). The exception is that at the stage of determining the scope of the SEA report, the SEA Law does not provide for posting the decision of the competent authority on its website, but states that the competent authority forwards in writing the decision on determining the scope of the report to the initiator, and the latter publishes it on its website and informs the public concerned.

Also, at the stage of determining the scope of the SEA report, the results of the consultations are recorded in a protocol, which contains a list of mutually agreed information necessary for preparing the SEA report and which must be an integral part thereof.

34 │

Figure 10. Defining the scope of the SEA report

At the following stages, the initiator prepares an SEA report, submits to the competent authority a draft policy and planning document and an SEA report (see figure 11), publishes them on its official website, informs the public concerned regarding these documents and the deadlines for submitting comments and proposals on them to the competent authority, as well as regarding the conducting of public consultations.

The competent authority, having received the SEA report and the draft policy and planning document, transmits them to the Panel of Experts,16 publishes the report and the draft document on its official website, and forwards them to non-governmental organizations promoting environmental protection.

Depending on the nature and complexity of the policy and planning document, the competent authority may involve representatives of the public authorities concerned, as well as other independent experts, in analysing the quality of the SEA report.

Comments and suggestions from the public concerned on the SEA report and the draft policy and planning document may be submitted to the competent authority within 30 days from the date of their publication on the website of the competent authority. The competent authority forwards copies of these comments and suggestions to the Panel of Experts.

The Panel of Experts provides the competent authority with a detailed and reasoned opinion on the quality of the SEA report in writing.

16The Panel of Experts is set up by order of the Minister of Environment and made up of experts appointed by the Ministry of Health, the Ministry of Environment, the Ministry of Infrastructure and Regional Development, the “Moldsilva” Agency, the National Archaeological Agency, the Academy of Sciences of Moldova, the Agency for Geology and Mineral Resources, and the “Apele Moldovei” Agency, the State Hydrometeorological Service.

│ 35

If the quality of the SEA report is inadequate, the competent authority having received the opinion of the Panel of Experts draws up the conclusion on the quality of the SEA report and indicates the aspects of the report that it considers inadequate with reference to comments and suggestions received from the public concerned and to the opinion of the Panel of Experts, as well as the recommended measures for finalizing the SEA report. If the SEA report is returned for revision, the initiator revises and resubmits the report.

Having received a conclusion on the quality of the SEA report from the competent authority, the initiator posts a copy of the report on its website and informs the public concerned.

Simultaneously with assessing the quality of the SEA report, the Panel of Experts evaluates the draft policy and planning document. Once the SEA report is found to meet the required quality standards, the Panel of Experts provides the competent authority with a reasoned opinion on the draft policy and planning document.

Taking into account the opinion of the Panel of Experts on the draft policy and planning document, as well as the results of public consultations and, as the case may be, transboundary consultations, the competent authority issues an environmental opinion, notifies the initiator in writing about the environmental opinion and publishes it on its official website. The initiator also posts the environmental opinion on its official website and informs thereof the public concerned.

Figure 11. Review of the quality of an SEA report and a draft policy or planning document

After the approval of a policy or planning document according to the procedure established by the legislation, the initiator places on its official website and informs the public concerned, the public authorities concerned and makes available to them, the environmental opinion, as well as the following:

36 │

(a) a policy or planning document in the approved form;

(b) information on how environmental concerns were integrated into the policy or planning document and the effects of the policy or planning document on the environment, including public health;

(c) information on consultations with the public and the public authorities concerned, the results of consultations in a transboundary context.

The initiator, within its purview and in accordance with the established programme, monitors the significant environmental, including public health, effects of the implementation of the policy or planning document in order to identify unforeseen adverse effects at an early stage and to be able to take appropriate remedy measures. The initiator submits the monitoring results to the competent authority in accordance with the schedule and form provided in the approved policy or planning document and publishes them on its official website.

Legal preconditions for launching an electronic SEA register The general legal framework for the development and operation of electronic registers in the Republic of Moldova consists of the 2003 Law on Informatization and State Information Resources, the 2007 Law on Registers, Government Decree No. 562 of 22 May 2006 on the creation of State automated information systems and resources, Government Decree no. 153 of 25 August 2021 on Approval of the Concept of the Information System “Register of State Information Resources and Systems” and the Regulations on the procedure for maintaining the Register.

The SEA Law also stipulates that the competent authority creates, develops and maintains an electronic register of information on SEA and makes it available to the public by posting it on the official website in the section dedicated to SEA (art. 84)17.

Currently, in the Republic of Moldova there is no centralized electronic database of SEA procedures and relevant documents within such procedures. From the interaction patterns discussed above, it is concluded that information and documents within the framework of the SEA procedure are posted on the websites of the initiator and the competent authority. Initiators encompass various central or local public authorities that initiate policy or planning documents and are responsible for their development. Accordingly, information related only to their purview is posted on their websites.

Information is also posted on various websites depending on the status of the document and the type of competent authority involved: for national-level policy and planning documents, including sectoral and cross-sectoral documents approved by the Government or Parliament – on the website of the Ministry of Environment; and for local-level policy and planning documents approved by local public administration – on the website of the Environmental Agency.

17This provision came into force on 21 October 2023.

│ 37

3.5. Ukraine

SEA legislative framework and stakeholders’ interaction within the SEA procedure

The SEA procedure in Ukraine is regulated by the 2018 Law of Ukraine on Strategic Environmental Assessment.18 Also, a number of by-laws were adopted to regulate certain aspects of the SEA implementation, including the Procedure for monitoring the effects of the implementation of the State planning documents for the environment, including for public health, approved by the resolution of the Cabinet of Ministers of Ukraine dated 16 December 2020, No. 1272, the procedure for maintaining the unified register on strategic environmental assessment, approved by a resolution of the Cabinet of Ministers of Ukraine dated 2 May 2023, No. 430 and others. The legal framework and general requirements for the development and operation of State electronic registers in Ukraine are prescribed by the 2021 Law of Ukraine on Public Electronic Registers.

According to the Law of Ukraine on SEA, SEA is a procedure of identifying, describing and evaluating effects of implementation of a public planning document on the environment, including human health, reasonable alternatives, developing measures to prevent, reduce and mitigate possible adverse effects, which includes the scoping of SEA, the preparation of a SEA report, the carrying out of public consultations and consultations (where necessary – transboundary consultations), the taking into account of the SEA report and the results of public consultations and consultations in the public planning document, the provision of information on the adoption of the public planning document.

Public planning documents in accordance with the Law on SEA are strategies, plans, schemes, urban planning documentation, national programmes, State target programmes and other programmes and programme documents, including amendments to them, developed and/or subject to adoption by a public authority or local self-governance authority.

The stages of an SEA include:

1) defining the scope of the SEA

2) drawing up an SEA report

3) conducting public discussion and consultations with authorized bodies in the field of environmental protection and health, transboundary consultations

4) taking into account the SEA report, the results of public discussion and consultations

5) informing about the adoption of the public planning document

6) monitoring the effects of the implementation of the public planning document on the environment, including public health.

Until May 2023, the exchange of information and documents within the framework of the SEA procedure was carried out by sending relevant documents in electronic or paper form between SEA stakeholders via e-mail or regular mail. For the purpose of consultation, planners19 submitted relevant documents to authorized bodies at the national level (Ministry of Environmental Protection and Natural Resources and

18This report examined the version of the Law as of 9 July 2023 - https://zakon.rada.gov.ua/laws/show/2354-19#Text 19Planner – a State executive authority or a local self-governance authority responsible for the preparation of public planning documents and exercising general leadership and control over the implementation thereof, or other defined by the legislation planner of the public planning documents.

38 │

Ministry of Health) or authorized territorial bodies (regional level). The documents were made public on the websites of the planners and relevant authorities consulted. Also, planners additionally informed the public through print media.

Thus, SEA documents were posted on the websites of numerous and varied authorities (depending on the planner and the status of the state planning document). Moreover, within the framework of one SEA procedure, some documents were posted only on the planner’s website, while other were posted exclusively on the websites of the authorities consulted.

In May 2023, amendments to the Law of Ukraine on Strategic Environmental Assessment came into force, introducing provisions on the Unified SEA register. As a result of such amendments, the SEA procedure and the exchange of documents at all stages is carried out through an electronic register - the Unified SEA register (hereinafter referred to as the Register). Since May of this year, all documents on SEA created by relevant authorities at all levels are submitted in electronic form20 and are freely available in the Register21. Disclosure of documents on SEA on the Internet is carried out through the Register (authorized access), as well as on the official websites of planners (unauthorized access). At the local level in rural areas, information is also provided through notice boards.

Unified SEA Register According to the legislation of Ukraine, the Register is a unified information and communication system that ensures the creation, viewing, collection, entry, accumulation, processing, use, review, storage, protection, recording and provision of information in the field of SEA. The Register was created using software that ensures its compatibility and electronic information interaction in real time with other electronic information systems and networks that make up the State information resources, including the Unified Register on Environmental Impact Assessment, the urban planning cadastre, the State Land cadastre, etc.

The Register is an electronic database of information and documents created and loaded by electronic user accounts that facilitate electronic interaction between SEA stakeholders.

The Register is also a functional module of the Unified Environmental Platform ‘EcoSystem’, intended for the official publication of documentation on SEA, ensuring that SEA stakeholders and other parties concerned are informed about the progress and results of SEA implementation.

The Register is maintained to ensure:

• access of the SEA stakeholders to up-to-date information on the implementation of SEA (regulatory and legal acts, guidelines, reference information);

• electronic interaction between SEA stakeholders, submission and processing of information by planners of public planning documents and authorities consulted through electronic user accounts;

• carrying out of consultations with authorized bodies and transboundary consultations;

• free access of all parties concerned to the information on SEA of public planning documents;

20Except for the graphic materials of urban planning documentation, which are submitted in paper form. At the State level, there are prerequisites for the transition to a digital format in terms of graphic materials of urban planning documentation. 21Currently, access to information in the Register requires user authorization, which may not be fully consistent with the “free public access” approaches to registers provided for by the Aarhus Convention.

│ 39

• holding public discussions during the SEA process and providing suggestions and comments from the public.

The Ministry of Environmental Protection and Natural Resources is the holder and administrator of the Register, ensuring the maintenance of the Register, as well as being responsible for the technical, technological and software support of the Register, and the storage and protection of the data contained in the Register.

Planners and authorities consulted have access to the Register with the right to enter information in accordance with their powers through electronic user accounts. When a planner or an authority consulted enters information into the Register, the Register’s software automatically creates messages sent through electronic user accounts to the authorities consulted or to the planner. Such messages include the date and time of entering the information in the Register and a link to view them in the Register.

The public has access to information entered into the Register via the internet (authorized access). The public is also informed through other means (planners websites, notice boards). Suggestions and comments from the public during the SEA process are submitted in electronic or paper form to the planner, who enters them into the Register (see also figure 12).

The following are entered into the Register:

1) a statement defining the scope of the SEA and a notice of its publication;

2) a draft public planning document, an SEA report and a notice of the publication of these documents;

3) comments and suggestions of the public; a statement of public consultations;

4) comments and suggestions of the authorities consulted; a statement of consultations;

5) an approved public planning document (except for information that, according to the law, constitutes a State secret or relates to information with limited access) and the decision on its approval;

6) information on transboundary consultations (if any);

7) results of monitoring the effects of the implementation of public planning documents on the environment, including public health, information on the adverse effects of the implementation of the public planning document on the environment, including public health, unforeseen by the SEA report, and appropriate remedial actions undertaken;

8) recommendations on the need to carry out SEA of minor changes to the public planning document for which SEA was previously carried out, and a decision not to carry out SEA of such changes.

Information entered into the Unified SEA Register is public and open. Access to it is provided via the internet; however, as mentioned above, it requires authorization.

40 │

Figure 12. SEA procedure stakeholders’ interactions after the launch of the Register

│ 41

3.6. Conclusions In all countries under consideration, except Ukraine, there are currently no centralized electronic SEA registers. Access to information and documents in the SEA process in the countries under review is carried out, for example, by posting them on the websites of the planning authorities, the environmental authorities, the health authorities and various other bodies. At the same time, planning authorities may be various administrative bodies or other authorized organizations responsible for preparing plans and programmes, and accordingly, only information within their purview is posted on their websites.

In terms of posting information within the framework of an SEA on the websites of environmental and health authorities, in a number of countries there is a division according to the status of the plan or programme and the level of the authority consulted (national, regional/local), and the information is also posted on various websites of the relevant authorities.

The dissemination of information on the websites of various authorities does not in itself contradict approaches of providing broad public information, but the need to search for information on the websites of various authorities (in some cases even within the same SEA procedure) may significantly reduce the effectiveness of information and opportunities for public participation. Launching a centralized database of information on SEA could help solve this problem.

In all the countries under consideration, digitalization processes are taking place. Many countries are switching to electronic document management and creating e-government systems. Thus, in the countries there are prerequisites for launching SEA registers.

4. Recommendations

4.1. Legal recommendations When developing and launching electronic SEA registers, it is advisable to take into account the following recommendations:

(a) Develop a system that can provide not only the accumulation of and access to information on an SEA, but also more effective interaction between SEA stakeholders, consultations and exchange of documents, as well as facilitate the analysis and collection of information in order to fulfil reporting obligations under the SEA Protocol;

(b) Take into account the national specifics and features of the SEA procedure in specific countries, levels of decision-making, the purview of the authorized bodies, as well as the key functions and recommendations provided in section 1 of this report;

(c) The SEA electronic register can be either an independently functioning system or combined with other registers (for example, an EIA register) or integrated into existing electronic e-government systems or other databases. For example, in Georgia, a public environmental assessment portal is being developed that will combine EIA/SEA documents on a single platform and provide the public with systematic and simplified access to information related to EIA and SEA procedures;

(d) In addition to securing technical solutions and financial resources, it is necessary to amend or supplement the legislation on SEA and/or develop and adopt relevant by-laws;

(e) When preparing amendments to legislation and/or developing new acts, legislative norms should provide a legal basis for the timely entry of information and documents into the register, compliance with the deadlines stipulated by the SEA procedure, and storage and protection of information. For example, in Ukraine, the SEA procedure is directly linked to the entry of information and documents into the Unified SEA Register, and the deadlines provided for by the procedure start from the date of placement/publication of information in the Unified SEA register;

42 │

(f) The register should, at a minimum, ensure the availability of information as it becomes available throughout the SEA process, as well as access to monitoring information on significant environmental, including health, effects of the implementation of plans and programmes;

(g) In the legislation, determine the periods of storage and availability in the electronic SEA register of information on completed SEA procedures, as well as information on monitoring of the effects of the implementation of plans and programmes after the expiration of their validity periods and / or completion of monitoring activities.

For some countries, the following should be additionally noted. In Georgia and the Republic of Moldova, in 2023, the legislation was supplemented by separate provisions on registers (for more details, see the chapters on Georgia and the Republic of Moldova). However, these provisions require further detailing in the relevant laws and/or regulations, in particular regarding the procedure for creating and operating such registers, and the interaction of stakeholders within the framework of the SEA procedure.

In Ukraine, changes to the legislation and the corresponding by-law regulating the operations of the Unified SEA register were adopted, and a centralized electronic SEA register was created and began to operate (for more details, see the chapter on Ukraine).

In the future, based on the results of the practical application of the register, it may be necessary to refine both the by-laws and the functionality of the register, including the issue of exclusively authorized public access to the register, which may not fully correspond to the approaches of “free access” of the public to the registers provided for by the Aarhus Convention.

4.2. Technical recommendations Creating software to automate the SEA process requires an integrated approach that includes the development not only of the software itself but also of the associated data registries. Automation of the process can greatly simplify and speed up the exchange of information, as well as increase the transparency of procedures in all the participating countries. At the same time, it is important that the automated system be maximally integrated and harmonized with data registers and the requirements of the Protocol, and also that it should ensure the possibility of effective data exchange between countries and the secretariat of the Convention. The following are some general recommendations for planning to automate the SEA process and create common data registries:

(a) A digital platform should be central, and accessible both to authorities and to the public. This platform may include modules for initiators to submit applications, conduct public consultations and discussions, and monitor the results of the SEA. Thanks to data centralization, process participants will be able to monitor the status of applications and documents in real time, as well as to receive notifications about the need for their revision or about new legal requirements;

(b) To ensure consistency with the Protocol, it is important to implement common data exchange

standards that comply with international requirements and regulations. This will ensure comparability and compatibility of information at the international level and facilitate cross-border data exchange. Existing national systems and data registries should also be taken into account when automating the SEA process. For example, SEA registers should be integrated with the institution's automated document management system, if available, and registers of national legislation and environmental registers available in the country;

(c) User feedback and support mechanisms should be in place to ensure prompt responses to emerging

issues and problems and to facilitate the continuous improvement of software and data registries and increase their effectiveness;

│ 43

(d) An important element of automation is also a module for conducting expert assessments and issuing conclusions. Experts should be able to interact with the system to upload their reports and conclusions, which will increase the efficiency of the document approval process, and have access to other national environmental registers, including registers related to public health and environmental monitoring data;

(e) Registers can be integrated with geographic information systems to visualize the affected areas of

planned projects and programmes, allowing for a better assessment of their potential environmental effects;

(f) In the process of creating SEA registers and automating them, data protection is a key aspect. Data

protection not only ensures the security and confidentiality of information, but also maintains the confidence of all stakeholders in the SEA process. Given this, security recommendations should include a number of measures aimed at protecting information at all stages of its life cycle;

(g) As data confidentiality can be guaranteed by encrypting both stored and transmitted data,

information from SEA registers must be securely encrypted to prevent leakage or unauthorized access. It is also important to differentiate access by providing different levels of access to different groups of users by establishing strict authentication and authorization procedures based on the use of passwords, biometrics or electronic keys;

(h) To maintain data integrity, use systems that can track and verify any changes to the data, including

regular audits and reviews. This will ensure that the information in the registers is accurate and up to date, and that any changes are documented and can be traced back to the source;

(i) As availability of data is essential for the efficiency of the SEA process, data must be protected

from attacks and failures that could disrupt access to it. Backup and disaster recovery strategies must be integrated into the registry infrastructure so that in the event of any disaster, data can be quickly recovered;

(j) Technological solutions such as firewalls, antivirus software, as well as virtualization and cloud

technologies, can serve to strengthen the overall system security. These tools will help protect the registry infrastructure from external threats and ensure its stable operation;

(k) Compliance with international data protection standards such as the European Union General Data

Protection Regulation and national legislation is mandatory. This requires regularly reviewing policies and procedures to ensure that SEA registries comply with current regulatory requirements and data protection best practices;

(l) Regular training should be provided for software users to interact effectively with data registries

and take full advantage of their capabilities. Personnel training may include training, seminars and information sessions aimed at increasing users’ security competencies – for example, training in the correct handling of confidential information, rules for creating and using passwords, rules for searching for information, and using geographic information systems.

In conclusion, to maintain confidence in the SEA process and its results, it is important not only to create a reliable data protection system but also to ensure the transparency and accessibility of the information for all stakeholders: international organizations, government bodies, scientific and expert organizations, and the public – whose participation and involvement is key in the environmental assessment process.

44 │

Annex. Process mapping: block diagrams of processes/functions of SEA registers

SEA Protocol

│ 45

Armenia

46 │

Azerbaijan

│ 47

Georgia

48 │

The Republic of Moldova

  • 1. Digital transformation in selected countries of Eastern Europe and the Caucasus
    • 1.1. Introduction
    • 1.2. Armenia
    • 1.3. Azerbaijan
    • 1.4. Georgia
    • 1.5. Republic of Moldova
    • 1.6. Ukraine
    • 1.7. Conclusions
  • 2. Key elements for developing a model electronic SEA register
  • 3. National SEA procedures and legal prerequisites for launching electronic registers
    • 3.1. Armenia
      • SEA legislative framework and stakeholders’ interaction within the SEA procedure
      • Prerequisites for launching an electronic SEA register
    • 3.2. Azerbaijan
      • SEA legislative framework and stakeholders’ interaction within the SEA procedure
      • Prerequisites for launching an electronic SEA register
    • 3.3. Georgia
      • SEA legislative framework and stakeholders’ interaction within the SEA procedure
      • Prerequisites for launching an electronic SEA register
    • 3.4. Republic of Moldova
      • SEA legislative framework and stakeholders’ interaction within the SEA procedure
      • Legal preconditions for launching an electronic SEA register
    • 3.5. Ukraine
      • SEA legislative framework and stakeholders’ interaction within the SEA procedure
      • Unified SEA Register
    • 3.6. Conclusions
  • 4. Recommendations
    • 4.1. Legal recommendations
    • 4.2. Technical recommendations
  • Annex. Process mapping: block diagrams of processes/functions of SEA registers
Russian

Отчет о предпосылках для создания модели базы данных СЭО в странах Восточной Европы и Кавказа, ключевые элементы и

рекомендации

Апрель 2024

2 │

Отчет подготовлен в рамках программы "Европейский Союз за окружающую среду" (EU4Environment – Зеленая экономика) консультантами секретариата ЕЭК ООН по Конвенции об оценке воздействия на окружающую среду в трансграничном контексте (Конвенция Эспо) и Протокола по стратегической экологической оценке (Протокол по СЭО): Дмитрием Скрыльниковым и Дмитрием Авериным. Ценная информация была предоставлена несколькими странами Восточной Европы и Кавказа.

Публикация выпущена на английском и русском языках.

Отказ от ответственности

Данная публикация была профинансирована Европейским Союзом. Ответственность за ее содержание несет исключительно секретариат Европейской экономической комиссии ООН (ЕЭК ООН), и он не обязательно отражает точку зрения Европейского Cоюза.

│ 3

Содержание

1. Цифровая трансформация в отдельных странах Восточной Европы и Кавказа ............ 4

1.1. Введение ............................................................................................................................. 4 1.2. Армения .............................................................................................................................. 4 1.3. Азербайджан ...................................................................................................................... 5 1.4. Грузия ................................................................................................................................. 7 1.5. Республика Молдова ......................................................................................................... 8 1.6. Украина ............................................................................................................................ 10 1.7. Заключение....................................................................................................................... 13

2. Ключевые элементы для создания модельного электронного регистра СЭО ............... 14

3. Национальные процедуры СЭО и предпосылки для создания электронных регистров ............................................................................................................................................. 20

3.1. Армения ............................................................................................................................ 20 Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО ....... 20 Предпосылки для создания электронного регистра СЭО ........................................................ 21

3.2. Азербайджан .................................................................................................................... 23 Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО ....... 23 Предпосылки для создания электронного регистра СЭО ........................................................ 26

3.3. Грузия ............................................................................................................................... 27 Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО ....... 27 Предпосылки для создания электронного регистра СЭО ........................................................ 32

3.4. Республика Молдова ....................................................................................................... 33 Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО ....... 33 Предпосылки для создания электронного регистра СЭО ........................................................ 39

3.5. Украина ............................................................................................................................ 40 Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО ....... 40 Единый регистр СЭО ................................................................................................................... 41

3.6. Заключение....................................................................................................................... 43

4. Рекомендации .............................................................................................................................. 44

4.1. Юридические рекомендации .......................................................................................... 44 4.2. Технические рекомендации ............................................................................................ 45

Приложение 1. Картирование процессов: блок-схемы процессов/функций регистров СЭО ...................................................................................................................................................... 48

4 │

1. Цифровая трансформация в отдельных странах Восточной Европы и Кавказа

1.1. Введение

Новая политика Восточного партнерства до 2025 года определяет долгосрочные цели политики, направленные на реагирование на новые приоритеты, укрепление устойчивости к решению общих проблем, содействие устойчивому развитию и обеспечение конкретных результатов для граждан и одной из ключевых целей этой политики является поддержка устойчивой цифровой трансформации стран-партнеров. Последние 10 лет Азербайджан, Армения, Грузия, Молдова и Украина активно стремились к модернизации и интеграции информационных систем и реестров, с целью повышения эффективности государственного управления, улучшения бизнес-среды и повышения качества жизни населения. В странах реализуются проекты по цифровизации государственных и муниципальных услуг, внедрению электронного документооборота и развитию цифровой инфраструктуры. Основные области применения информационных систем и реестров включают здравоохранение, образование, социальное обеспечение, статистику, охрану окружающей среды и другие направления. В странах существует ряд ключевых национальных стратегий и законодательных актов, направленных на развитие электронного правительства, улучшение доступа к информации и повышение прозрачности деятельности государственных органов, в том числе в области охраны окружающей среды и экологической безопасности.

1.2. Армения

Армения продолжает активно развивать информационные системы и реестры в области государственного управления и охраны окружающей среды. Армянский портал электронного правительства объединяет инструменты электронного управления и различные реестры (https://e-gov.am/). На портале реализованы механизмы обращения к правительственным органам, подачи заявок на получение лицензий и отчетности лицензиатов. Кроме этого портал содержит ссылки на большое количество секторальных порталов электронных услуг не объединенных в единую систему, например, Электронная система государственных платежей (https://www.e-payments.am/), Электронная система кадастра недвижимости(http://www.e- cadastre.am/), Единый сайт для публикации проектов правовых актов (http://www.e-draft.am/), Единая площадка для электронных запросов (http://www.e-request.am/), Электронная регистрация организаций (http://www.e-register.am/), Судебная информационная система (http://www.datalex.am/), Электронная система налоговой отчетности (https://file- online.taxservice.am/) и другие. В Армении отсутствует портал открытых данных, а публичная информация распространяется через сайты различных государственных органов и доступна по запросам граждан. Многие государственные органы публикуют документы в немашиночитаемых форматах, что усложняет поиск и использование такой информации.

│ 5

В стране реализовано большое количество платформ для публикации данных и информации об окружающей среде. Сайт Министерства экологии (http://www.mnp.am/) доступен на армянском, английском и русском языках. Сайт содержит отчеты об окружающей среде, организованные по тематикам «атмосфера», «вода», «отходы», «изменение климата», «биоресурсы», «лес», «земля», «экологический мониторинг». Отчеты по каждой теме публикуются в форматах PDF и DOCX. На сайте доступен реестр организаций, оказывающих влияние на качество атмосферного воздуха. Сайт Центра гидрометеорологии и мониторинга (http://armmonitoring.am/) предоставляет данные об окружающей среде по атмосферному воздуху, поверхностным и подземным водам, отходам, почве и грунтам. Сайт содержит статистические данные и представляет доступ к тематическим картам, таблицам и графикам. Часть данных по постам качества окружающей среды доступна интерактивно. Сайт управления Водного комитета (https://www.scws.am/) предоставляет доступ к отчетам о деятельности, специальным отчетам по водным ресурсам, выданным разрешениям, пространственным данным Государственной водной кадастровой информационной системы. Портал Статистического комитета (https://www.armstat.am/) публикует официальную статистическую информацию. Данные размещаются в виде отчетов в формате DOCX. По окружающей среде доступен только ежегодный отчет «Окружающая среда и природные ресурсы Республики Армения». Больше статистических данных доступно с помощью платформы визуализации данных Статистического комитета (https://statbank.armstat.am/). Портал содержит значительное количество информации по окружающей среде, включая показатели ЕЭК ООН, экологические счета, данные по добыче полезных ископаемых в машиночитаемых форматах. Портал позволяет осуществлять межсекторальный и временной анализ данных. Несмотря на большое количество реализованных проектов в области электронного управления, недостаточная интеграция систем, отсутствие единого стандарта качества данных и портала открытых данных приводит к ограничению доступа к информации. Решение этих проблем потребует дополнительных усилий со стороны государства. 1.3. Азербайджан

Азербайджан является страной с быстро развивающейся экономикой и динамично растущим государственным сектором. Страна значительно продвинулась в развитии информационных систем для государственного управления. На исполнение указа Президента Азербайджанской Республики от 14 марта 2018 года № 1885 «О мерах, связанных с развитием электронного правительства и переходом на цифровое правительство» при Государственном агентстве обслуживания граждан и социальных инноваций был создан «Центр развития электронного правительства». Центр сотрудничает с другими государственными органами по формированию «Электронного правительства» в качестве координатора работ и осуществляет деятельность по созданию соответствующей инфраструктуры. Портал открытых данных (https://opendata.az/) был разработан в 2015 году Министерством цифрового развития и транспорта. Сейчас доступна новая версия портала, в которой расширено представление данных и реализованы улучшенные возможности для поиска информации. Данные предоставляются бесплатно и могут быть использованы без ограничений. Портал доступен на национальном, русском и английском языках, реализован интерфейс API для обмена данными. В

6 │

общей сложности на портале опубликовано более 500 наборов данных от 5 учреждений, в том числе 30 наборов данных в разделе «Экология». Самые свежие наборы данных датируются концом 2022 года. Портал «Электронное правительство» (www.e-gov.az) предоставляет онлайн-услуги для граждан и бизнеса, включая выпуск паспортов, регистрацию бизнеса и подачу налоговых деклараций. Портал подключен к Электронному реестру государственных услуг и Государственному реестру информационных ресурсов и систем. В целом портал поддерживает 451 услугу, в том числе 20 электронных услуг в области охраны окружающей среды. Вместе с порталом был создан шлюз электронного правительства для обеспечения обмена информацией между более чем 40 государственными органами, подключенными к этой инфраструктуре. Через портал «Электронное правительство» Министерство экологии и природных ресурсов оказывает электронные услуги по предоставлению информации об окружающей среде и природных ресурсах, выдаче разрешений на выбросы загрязняющих веществ в атмосферный воздух, использованию водных объектов, земель лесного фонда, отводу земельных участков для размещения, захоронения и переработки отходов, выдаче разрешений на охоту и промышленный лов рыбы, картографирование, предоставляет заключения по объектам государственной экологической экспертизы и т.д. Электронные услуги по проведению экологической экспертизы доступны также на портале Государственного агентства экологической экспертизы (https://eis.eco.gov.az/) Сайт Министерства экологии и природных ресурсов (http://eco.gov.az) предоставляет информацию по биоразнообразию, гидрометеорологии, геологии, лесам и природным паркам. Большая часть информации доступна в текстовых и графических немашиночитаемых форматах. В большинстве случаев информация, опубликованная Министерством, ограничивается ежегодными экологическими отчетами. Сайт национальной гидрометеорологической службы (http://meteo.az/) предоставляет доступ к интерактивным данным о состоянии атмосферного воздуха и поверхностных вод и служит общей платформой для обмена экологической информацией. На базе сайта при технической поддержке Европейского агентства по окружающей среде в рамках проекта ENI SEIS II East была разработана «Водная информационная система «ЭкоПортал», обеспечивающая доступ к данным о поверхностных водах и экологических показателях ЕЭК ООН. Сайт Государственного статистического комитета (https://www.stat.gov.az/) является основной платформой для распространения статистических данных, включая данные по окружающей среде. Все статистические данные доступны для скачивания в формате XLSX. Статистические данные доступны также через портал ASIS (https://www.azstat.org/portal/). В настоящее время портал предоставляет доступ к 11 наборам данных об окружающей среде, которые могут быть визуализированы хронологически и экспортированы во многих машиночитаемых форматах (XLS, CSV и TXT). На портале доступны статистические данные по национальным паркам, водным и лесным ресурсам, загрязнению атмосферного воздуха, отходам и т.д. Данные по загрязнению атмосферного воздуха и использованию водных ресурсов доступны на интерактивной карте статистических данных (https://www.azstat.org/webmap/).

│ 7

1.4. Грузия

Грузия имеет достаточную законодательную базу, которая способствует развитию информационных систем и реестров и регулирует основные аспекты их развития и использования. В стране продолжает улучшаться инфраструктура связи и доступ к сети Интернет, однако существуют различия между уровнем доступа в городах и сельских районах. Портал открытых данных (https://data.gov.ge/) разработан Агентством по обмену данными Грузии и представляет собой единую национальную платформу для публикации открытых данных, принадлежащих государственным учреждениям. Портал содержит более 250 наборов данных в машиночитаемых форматах по различным секторам, в том числе 8 наборов данных по окружающей среде. Вместе с тем, наборы данных по окружающей среде содержат в основном устаревшую информацию и информацию по реализации национальных планов. Грузия развивает проекты электронного правительства с целью улучшения качества предоставляемых государственных услуг и повышения прозрачности государственного управления. Одним из таких проектов является создание единого портала государственных услуг (https://my.gov.ge/). Портал предоставляет возможность получения различных государственных услуг онлайн, включая услуги в сфере окружающей среды, социальные, медицинские услуги, оплату штрафов и коммунальных платежей и т.д. Через портал может быть запрошена публичная информация у государственных органов и статистическая информация из более чем 50 государственных органов. Через портал организован доступ к Классификатору экономической деятельности, Реестру хозяйственной деятельности, Реестру предприятий, Реестру нотариальных действий, Электронному реестру агентства по развитию госуслуг, Электронному реестру нотариусов, Ипотечному реестру, Реестру лизинга, Адресному реестру и т.д. Доступ к реестрам и услугам в сфере охраны окружающей среды через портал отсутствует. На сайте Министерство охраны окружающей среды и сельского хозяйства Грузии (http://mepa.gov.ge/) размещена публичная аналитическая информация, информация об услугах агентств, входящих в систему Министерства окружающей среды и сельского хозяйства, отчеты, планы, стратегические документы, экологическое законодательство, реестр сельскохозяйственных кооперативов, объявления по ОВОС, информация о публичных слушаниях и данные мониторинга загрязнения воздуха. Документы доступны в немашиночитаемых форматах. Портал открытых данных Министерства окружающей среды и сельского хозяйства (http://data.mepa.gov.ge/) охватывает данные разных категорий, включая землепользование, растительный покров, биоразнообразие, уровни рисков и охраняемые территории. Данные представлены различными организациями, в том числе Национальным агентством лесного хозяйства, Агентством по охраняемым территориям, Департаментом биоразнообразия и лесного хозяйства, а также неправительственными организациями. Данные доступны в немашиночитаемых форматах. Всего на портале размещено 52 документа в основном в формате PDF. Данные на портале не обновлялись с 2019 года. Сайт Экологического информационно-образовательного центра (http://eiec.gov.ge/) публикует национальные доклады и ежегодные отчеты о состоянии окружающей среды, планы действий, информационные бюллетени и реестр экологических организаций. Портал качества воздуха (http://air.gov.ge/) предоставляет доступ к информации о загрязнении атмосферного воздуха, включая индекс качества воздуха в реальном времени на постах контроля качества атмосферного воздуха. Целью портала является сбор информации о загрязнении воздуха

8 │

и поддержка процесса принятия решений в сфере улучшения качества воздуха. Портал предоставляет населению информацию о качестве атмосферного воздуха, ежедневные, ежемесячные и годовые отчеты, а также рекомендации по охране здоровья и защите от загрязнения атмосферного воздуха. Портал Национальной статистической службы Грузии (https://www.geostat.ge/) Предоставляет ежегодную и ежеквартальную статистическую информацию, в том числе в сфере охраны окружающей среды. На сайте организован доступ к реестру экономических субъектов, данным сельскохозяйственных обследований и обследований домохозяйств, секторальным реестрам. Через портал (http://pc-axis.geostat.ge/) организован доступ к статистическим данным загрязнения атмосферного воздуха, лесных ресурсов, охраняемых территорий, нарушений экологического законодательства. Также доступны в машиночитаемых форматах данные по экологическим счетам и экологическим показателям ЕЭК ООН. Сайт Агентства охраняемых территорий (http://apa.gov.ge/) предоставляет общую информацию об охраняемых территориях в Грузии, их количестве и категориях, а также информацию об Агентстве. ГИС-портал Министерства охраны окружающей среды и природных ресурсов (https://gis.mepa.gov.ge/portal/) предоставляет доступ к тематическим слоям и картам охраняемых территорий, землепользования, лесов, геологическим рискам, деградации земель, мелиорации и т.д. Атлас лесопользования и землепользования Грузии (https://atlas.mepa.gov.ge/) представляет собой платформу онлайн-мониторинга, которая позволяет пользователям просматривать и анализировать данные на национальном и региональном уровнях. Атлас находится в ведении Министерства окружающей среды и сельского хозяйства Грузии при поддержке Института мировых ресурсов (WRI) и включает данные государственных учреждений, а также информацию неправительственных организаций. В настоящее время Грузия разрабатывает Общественный портал экологической оценки, который объединит процедуры ОВОС и СЭО на единой платформе.

1.5. Республика Молдова

Республика Молдова активно внедряет информационные системы и реестры в различных сферах деятельности, таких как государственное управление, здравоохранение, образование, статистика, охрана окружающей среды. В стране существует ряд ключевых национальных стратегий и законодательных актов, направленных на развитие электронного правительства, улучшение доступа к информации и повышение прозрачности деятельности государственных органов. Молдова активно развивает инфраструктуру электронного правительства и информационных технологий. В стране реализуются проекты, направленные на модернизацию и оптимизацию государственного управления с использованием современных технологий. Открытый портал данных (https://date.gov.md/) был запущен Агентством электронного правительства при поддержке Всемирного банка в 2011 году и обновлен в 2019 году. Портал предоставляет доступ к 1176 наборам данных, реестрам юридических лиц, законодательству. Через портал доступны данные по использованию воды в Республике Молдова, данные национального фонда геопространственных данных, информационный портал электронного кадастра, реестр государственных закупок, банк статистических данных, государственный реестр правовых актов и другие. Министерство окружающей среды публикует на портале 35 наборов данных, включая данные о качестве атмосферного воздуха и поверхностных вод, выбросах загрязняющих веществ и

│ 9

парниковых газов, реестр водных объектов, реестры организаций, получивших разрешения на осуществление деятельности, связанной с ухудшением качества окружающей среды, наборы статистических данных и т.д. Портал электронного правительства (https://servicii.gov.md/) был запущен в 2012 году. На сегодня портал предоставляет информацию о 715 государственных услугах, из которых 275 услуг доступны в электронном виде. Портал государственных услуг обеспечивает доступ ко всем государственным услугам, связанным с охраной окружающей среды, однако отдельной вкладки «окружающая среда» на портале нет. Услуги доступны при выборе организации «Министерство окружающей среды». На портале доступны услуги по получению разрешений на выбросы загрязняющих веществ из стационарных источников, специальное использование воды, ввоз, вывоз или реэкспорт веществ, разрушающих озоновый слой, разрешений на обращение с отходами, рубки, рыбалку и охоту, услуги по проведению процедуры оценки воздействия на окружающую среду, получения заключения государственной экологической экспертизы и т.д. В Республике Молдова разработано несколько тематических порталов, на которых публикуются экологические данные и информация. На сайте Министерства окружающей среды (https://mediu.gov.md/) представлена информация о деятельности Министерства, программные и нормативные документы. Автоматизированная информационная система управления отходами (https://siamd.gov.md/portal/) была разработана Министерством окружающей среды при поддержке Словацкого агентства по развитию и сотрудничеству. Система предоставляет доступ к Реестру производителей отходов, на которых распространяется режим расширенной ответственности производителя, Реестру учреждений и предприятий, освобожденных от выполнения разрешительных требований к деятельности по утилизации и размещению отходов, реестрам производителей упаковки, батареек и аккумуляторов, электрического и электронного оборудования, масел, транспортных средств и т.д. Национальное бюро статистики (https://statistica.gov.md/) публикует тематическую статистическую отчетную информацию в том числе и агрегированную информацию по охране окружающей среды. Данные «Статбанк» (http://statbank.statistica.md/) содержит подробные таблицы статистической информации с временными рядами. С помощью этого инструмента можно создавать и сохранять выборки в различных форматах файлов, в том числе в машиночитаемых форматах. Сервис также предоставляет доступ к данным через API для разработчиков приложений. Данные по окружающей среде доступны в отдельном разделе. Публикуются данные по текущим расходам на охрану окружающей среды и экологическим платежам, данные по обращению с отходами, земельному и лесному фондам, метеорологии, охране атмосферного воздуха, водопользованию. Национальный реестр выбросов и переноса загрязняющих веществ (https://retp.gov.md/) предоставляет систематизированные данные о выбросах загрязняющих веществ в атмосферный воздух, воду, почву от стационарных и диффузных источников, а также о выбросах за пределы объекта и переносе загрязняющих веществ со сточными водами. Автоматизированная информационная система «Регистр химических продуктов, размещенных на рынке Республики Молдова» позволяет управлять процессами регистрации химических продуктов, получения и обработки ежегодных отчетов производителей и импортеров химических продуктов, а также выдавать разрешения на размещение химических продуктов на рынке. Система включает информацию о химических продуктах, представленных на рынке, риске для здоровья, экстренных мерах в случае чрезвычайных ситуаций, физико-химических свойствах продуктов, их

10 │

биологической активности, гигиенических и экологических нормах, включая меры безопасности на этапе производства, импорта, экспорта, перевозки и использования химических продуктов. Автоматизированная информационная система «Управление и выдача разрешительных актов» (SIA GEAP) используется для автоматизации получения экологических разрешений в Агентстве по охране окружающей среды. Также Агентством разработана автоматизированная информационная система выдачи разрешений «Электронная рыбная ловля» (https://e-pescuit.mediu.gov.md/), которая предназначена для получения разрешений на спортивное, любительское и договорное рыболовство. В Республике Молдова разработаны также автоматизированные информационные системы Государственный водный кадастр (https://csa.gov.md) и Государственный кадастр животного мира (https://am.gov.md/ro/node/590). Государственный водный кадастр обрабатывает и хранит данные о водных ресурсах, гидрографической сети, водных объектах, гидротехнических сооружениях, охраняемых территориях, охранных зонах и прибрежных водоохранных полосах, а также данные о водозаборах и местах отведения воды, данные водного баланса и информацию об управлении речными бассейнами. Государственный кадастр животного мира содержит данные о поголовье скота, данные о видах промысловых животных и ресурсах охотничьего фонда, исчезающих, уязвимых, редких видах животных и уязвимых видах животных, занесенных в Красную книгу Республики Молдова. В Республике Молдова планируется создание электронного реестра информации о стратегической экологической оценке, реестра объектов Изумрудной сети и реестра объектов, подпадающих под требования Директивы 2012/18/ЕС о контроле угроз возникновения значительных аварий, связанных с использованием опасных веществ.

1.6. Украина

Украина в последние годы активно стремится к модернизации и интеграции информационных систем и реестров. В стране реализуются проекты по цифровизации государственных и муниципальных услуг, внедрению электронного документооборота и развитию цифровой инфраструктуры. Цифровая трансформация в Украине определена как одна из приоритетных целей, а Министерство цифровой трансформации Украины планировало до 2024 года обеспечить полный доступ населения и бизнеса ко всем государственным услугам онлайн, обеспечить высокоскоростной доступ к сети Интернет для 95% социальных объектов и объектов транспортной инфраструктуры, а также увеличить долю цифрового продукта до 10% ВВП. В 2014 году в Украине был запущен национальный Портал открытых данных (https://data.gov.ua), который предназначен для предоставления доступа к общедоступной информации в виде открытых данных и предоставления доступа к информации, находящейся у государственных учреждений. На сегодня портал модернизирован и содержит более 29 тыс. наборов данных, однако по тематике «Окружающая среда» портал содержит всего 7 наборов. Многие наборы данных не обновляются с февраля 2022 года. Единый портал электронных услуг «Дія» (https://diia.gov.ua) предоставляет онлайн-услуги для юридических и физических лиц, которые сгруппированы по категориям: безопасность и правопорядок, транспорт, земля, строительство, недвижимость, справки, окружающая среда, здоровье, документы и гражданство, предпринимательство.

│ 11

Национальная онлайн-платформа «ЭкоСистема» (https://eco.gov.ua) является частью единого портала электронных услуг «Дія» и содержит актуальную информацию о состоянии окружающей среды, а также обеспечивает доступ к экологическим сервисам и реестрам. На основе этой платформы реализуется ряд проектов в области охраны окружающей среды. Проект по цифровой трансформации обращения с отходами «е-Отходы» предполагает создание электронной системы учета и мониторинга отходов. Сервис обеспечивает доступ к административным услугам и реестрам, в том числе к государственному реестру субъектов хозяйствования, осуществляющих прием и/или разборку утилизируемых транспортных средств, реестру лицензий на осуществление хозяйственной деятельности по обращению с опасными отходами, реестру заключений о трансграничной перевозке отходов, реестру деклараций об отходах, реестру сообщений о трансграничной перевозке отходов. Электронный сервис «е-Воздух» обеспечивает работу электронной системы учета выбросов загрязняющих веществ, а также внедрение электронных услуг по отчетности и выдаче разрешений на выбросы загрязняющих веществ. Сервис обеспечивает доступ к административным услугам по взятию на государственный учет объектов, которые оказывают или могут оказать вредное влияние на здоровье людей и состояние атмосферного воздуха и выдаче справки о величинах фоновых концентраций, доступ к реестру субъектов хозяйствования, которым утверждены показатели эмиссии (удельные выбросы) загрязняющих веществ в атмосферный воздух, реестру выданных справок о величинах фоновых концентраций, реестру экологических аудиторов и юридических лиц, имеющих право на осуществление экологического аудита. Проект цифровой трансформации лесного хозяйства «е-Лес» направлен на создание электронной системы государственного и общественного лесного контроля. На сегодня сервис обеспечивает доступ к административным услугам по выдаче специального разрешения на использование лесных ресурсов и по приобретению дров населением. Проект цифровой трансформации обращения с пестицидами и агрохимикатами «е-Пестициды» предполагает создание электронной системы учета пестицидов и агрохимикатов. На сегодня сервис обеспечивает доступ к реестру пестицидов и агрохимикатов, разрешенных для использования в Украине. Электронный сервис «е-ОВД» обеспечивает цифровизацию выдачи заключений по оценке влияния на окружающую среду и автоматизацию послепроектного мониторинга. Через сервис организован доступ к единому государственному реестру по оценке влияния на окружающую среду (http://eia.menr.gov.ua), перечню видов обсуждений и размерам платы за проведение общественного обсуждения в процессе осуществления ОВОС, перечню уполномоченных территориальных органов для проведения общественных обсуждений и реестру договоров на проведение общественных обсуждений в процессе осуществления ОВОС. Сервис также обеспечивает доступ к услугам по выдаче заключений по оценке влияния на окружающую среду и регистрации договоров о предоставлении услуг по проведению общественных обсуждений. Сервис «е-Окружающая среда» предполагает обеспечение автоматизации мониторинга окружающей среды путем интеграции информации из различных источников в единый аналитический модуль. На сегодняшний день в сервисе (https://ecozagroza.gov.ua) реализованы функции обработки обращений граждан и предоставления информации о влиянии военных действий на окружающую среду, доступна интерактивная информация о радиационном фоне на постах контроля радиационного состояния.

12 │

На базе платформы также планируется реализация проектов по цифровой трансформации биологического и ландшафтного разнообразия «е-ПЗФ», государственного надзора в сфере охраны окружающей природной среды «е-Экоконтроль», стратегической экологической оценки «е-СЕО», водного хозяйства «е-Вода», использования недр «е-Недропользование». В Министерстве защиты окружающей среды и природных ресурсов Украины также реализуются проекты по созданию систем автоматизации отчетности по Протоколу к конвенции о трансграничном загрязнении воздуха на большие расстояния 1979 года, касающиеся долгосрочного финансирования совместной программы наблюдения и оценки распространения загрязнителей воздуха на большие расстояния в Европе (ЕМЕП) и Протоколу о регистрах выбросов и переноса загрязнителей к Конвенции о доступе к информации, участии общественности в процессе принятия решений и доступе к правосудию по вопросам, касающимся окружающей среды 1998 года (Орхусская конвенция), которые станут частью общей платформы. Информация о состоянии поверхностных вод доступна на портале «Чистая вода» (http://texty.org.ua/water/). Портал представляет собой интерактивную карту загрязнения рек Украины на основе данных Государственного водного агентства. Интерактивная карта показателей загрязненности рек позволяет получить доступ к информации от более чем 400 постов качества поверхностных вод на реках Днепр, Днестр, Южный Буг и т. д. На сайте Государственной службы статистики (http://ukrstat.gov.ua) доступны статистические данные по образованию и обращению с отходами, выбросам загрязняющих веществ в атмосферный воздух от стационарных источников загрязнения, расходам на охрану окружающей среды, а также экологические показатели, рекомендованные ЕЭК ООН. Через сайт организован доступ к экологическим счетам выбросов в атмосферный воздух и счетам расходов на охрану окружающей природной среды. В Украине активно развиваются общественные проекты, направленные на предоставление информации о состоянии окружающей среды. Одним из примеров успешной реализации может служить Система «SaveEcoBot» (https://www.saveecobot.com/), агрегирующая открытые государственные данные и данные по сетям общественного мониторинга качества воздуха и радиационного фона в Украине. Портал «Ecodozor» (https://ecodozor.org/), разработанный при поддержке Экологической сети «Зой» (Швейцария), ОБСЕ, Гуманитарной инициативы REACH и Программы ООН по окружающей среде предоставляет информацию об экологических последствиях и рисках боевых действий в Украине, включая риски загрязнения при нарушении работы промышленных предприятий и объектов критической инфраструктуры, повреждения почв, лесные и травяные пожары. Военное вторжение РФ в Украину в феврале 2022 года привело к изменению информационной политики. На первое место вышли задачи, связанные с защитой данных и кибербезопасностью: модернизация систем защиты персональных данных, обеспечение безопасности цифровых сервисов, реестров и информационных систем. В то же время, в целях безопасности, был ограничен доступ ко многим информационным системам и реестрам, в том числе в области охраны окружающей среды. Например, с перебоями работает сайт Министерства защиты окружающей среды и природных ресурсов Украины (https://mepr.gov.ua/), на котором публикуется информация об окружающей среде и результаты работы Министерства, временно отключены от сети система «SEIS-Ukraine» (http://seis.menr.gov.ua/) для публикации информации о распространении экологической

│ 13

информации в Украине, включая данные CORINE Land Cover, система «DEIS», предоставляющая информацию об экологических последствиях и рисках боевых действий на востоке Украины с 2014 по 2022 гг.

1.7. Заключение

Несмотря на достигнутые успехи, все страны сталкиваются с похожими проблемами и вызовами в области использования информационных систем и реестров. Одной из основных проблем является недостаток финансирования для разработки и поддержания информационных систем и реестров. Бюджетные ограничения замедляют темпы внедрения новых технологий и снижают качество предоставляемых услуг. В некоторых странах все еще наблюдаются проблемы с доступом к сети Интернет и развитием ИТ- инфраструктуры в регионах, что затрудняет доступ населения к информационным системам и реестрам, особенно в удаленных сельских районах. Дефицит квалифицированных кадров в области информационных технологий и управления данными является еще одним вызовом для развития. Странам необходимо принимать меры по улучшению систем образования и привлечению высококвалифицированных специалистов в структуры государственного управления, привлечению сторонних разработчиков для реализации проектов в области цифровизации. Существующие информационные системы и реестры зачастую являются разрозненными и не обеспечивают эффективного взаимодействия между различными государственными органами или местными и центральными властями. Особенно это актуально для интеграции региональных данных и систем. В некоторых случаях несовершенство законодательной базы и стандартов по работе с данными может препятствовать развитию информационных систем и реестров в государственном управлении. Для обеспечения совместимости с международными стандартами странам необходимо постоянно совершенствовать законодательство, пересматривать стандарты качества данных, методы обработки и представления информации. Пандемия COVID-19 неоднозначно повлияла на развитие информационных систем и реестров. С одной стороны, страны были заинтересованы в развитии информационных технологий для обеспечения удаленного доступа к данным, с другой, пандемия в некоторых случаях привела к замедлению экономического роста и уменьшению бюджетных поступлений, что негативным образом сказывалось на финансировании проектов в области цифровизации. Боевые действия на территориях некоторых стран привели к ограничению доступа к данным и замедлению прогресса в цифровизации государственных и муниципальных услуг и развитии цифровой инфраструктуры. Из соображений безопасности в некоторых случаях была приостановлена работа информационных систем и реестров, ограничен доступ к данным, в том числе и к экологической информации. Несмотря на вызовы и ограничения, страны продолжают развивать и интегрировать информационные системы и реестры в области государственного управления и охраны окружающей среды.

14 │

Для повышения эффективности их использования, странам необходимо преодолеть существующие проблемы и обеспечить достаточное финансирование, координацию между государственными органами и усиление мер по защите данных. В целом, все страны имеют достаточный уровень развития инфраструктуры и кадровый потенциал для реализации проектов в области цифровизации процесса проведения стратегической экологической оценки.

2. Ключевые элементы для создания модельного электронного регистра СЭО

Протокол Европейской экономической комиссии ООН (ЕЭК ООН) по стратегической экологической оценке (Протокол по СЭО) к Конвенции «Об оценке воздействия на окружающую среду в трансграничном контексте» (Конвенции Эспо) был принят в 2003 году и вступил в силу в 2010 году. Протокол по СЭО определяет обязанности Сторон в отношении оценки вероятных аспектов воздействия на окружающую среду, включая воздействие на здоровье человека, определенных планов и программ, а также, по возможности, политики и законодательства, на раннем этапе их разработки. Статья 10 Протокола также требует от Сторон уведомлять и консультироваться друг с другом по поводу планов и программ, которые могут оказать значительное трансграничное экологическое воздействие. Протокол предусматривает участие широких кругов общественности в процессе принятия решений на государственном уровне, а также предварительные, своевременные и эффективные консультации с природоохранными и здравоохранительными ведомствами.

Определение стратегической экологической оценки (СЭО) в Протоколе по СЭО отражает ключевые элементы процедуры стратегической экологической оценки. Согласно Протоколу по СЭО определяется как: «Оценка возможных экологических последствий, включая воздействие на здоровье, которая предполагает определение сферы охвата оценки и подготовку экологического отчета, вовлечение общественности и проведение консультаций, принятие во внимание экологического отчета и результатов участия общественности и консультаций при разработке плана или программы» (статья 2, абзац 6).

В статьях 5 – 12 Протокола описывается процесс проведения СЭО планов и программ:

• Предварительная оценка (скрининг) (Ст. 5) • Определение сферы охвата процедуры СЭО (скопинг) для определения содержания

экологического отчёта (Ст. 6) • Экологический отчёт (Ст. 7) • Участие общественности (Ст. 8) • Консультации с природоохранными органами и органами здравоохранения (Ст. 9) • Трансграничные консультации (Ст. 10) • Решение об утверждении плана или программы (Ст. 11) • Мониторинг (Ст. 12)

Согласно Протоколу по СЭО на разных этапах происходит взаимодействие различных участников процедуры СЭО в том числе органов, которые отвечают за разработку соответствующих планов или программ, органов власти с которыми проводятся консультации исходя из их компетенции в сфере охраны окружающей среды или здоровья, общественности, а в случае потенциального трансграничного воздействия, и с другими сторонами Протокола по СЭО. Такое взаимодействие

│ 15

может включать информирование, обмен документами или их обнародование, предоставление письменных замечаний, проведение консультаций или общественных слушаний, различные мероприятия, обеспечивающие мониторинг существенных экологических, в том числе связанных со здоровьем населения, последствий осуществления планов и программ и доведение его результатов до сведения соответствующих органов и общественности (См. Схему 1).

16 │

Схема 1. Взаимодействие субъектов СЭО в рамках требований Протокола по СЭО

│ 17

Протокол по СЭО прямо рекомендует электронные средства информации в качестве средства обеспечения своевременного доступа общественности к документам (например, статьи 5.4 и 8.2). Однако, исходя из необходимости обеспечить эффективные возможности для участия общественности, использование электронных средств в качестве единственного способа распространения документов может оказаться недостаточным. Такой подход может исключить из процесса участия важные группы общественности, которые могут не пользоваться Интернетом, включая лиц с низкими доходами, изолированно проживающие группы населения (например, в сельской местности) и пожилых людей1. Таким образом информирование посредством электронных средств информации не подменяет и не исключает необходимости использования других способов информирования, но различные способы распространения информации и документов в совокупности с информированием посредством электронных средств информации и обеспечением возможности постоянного открытого доступа к документам на общедоступном электронном ресурсе могут обеспечить своевременный и наиболее эффективный доступ к информации в процессе участия общественности в рамках СЭО. Совещание Сторон Конвенции о доступе к информации, участии общественности в принятии решений и доступе к правосудию по вопросам окружающей среды (Орхусская конвенция) на своей седьмой сессии (18–20 октября 2021 г., Женева) приняло Обновленные рекомендации по повышению эффективности использования электронных информационных средств2 направленные на оказание помощи Сторонам Конвенции, подписантам и другим заинтересованным государствам в поддержке осуществления Конвенции путем поощрения разработки, обслуживания, модернизации и использования электронных информационных средств на основе применения общих подходов и стандартов. Кроме того, они будут поддерживать усилия по выполнению других соответствующих международных обязательств. В Обновленных рекомендациях рекомендуется обеспечить, при необходимости, путем принятия соответствующих законодательных или нормативных мер, с учетом статьи 5 (10) Орхусской конвенции: a) доступ общественности к экологической информации обеспечивался в допускающей поиск электронной форме и предоставлялся через Интернет, с тем чтобы информация, требуемая для публичного доступа в соответствии с Конвенцией, предоставлялась в допускающей поиск электронной форме, когда это требуется и когда информация существует в этой форме или может быть легко преобразована в эту форму по разумной цене; b) документация, которая должна быть подготовлена и/или представлена в контексте процедур принятия решений по экологическим вопросам, подпадающим под действие положений статей 6, 7 и 8 Конвенции, представлялась в электронной форме и постепенно становилась доступной для общественности через Интернет;

1 Информационно-справочное руководство по применению Протокола ЕЭК ООН по стратегической экологической оценке, ECE/MP.EIA/17, пункт 81. 2 Обновленные рекомендации по повышению эффективности использования электронных информационных средств. ECE/MP.PP/2021/2/Add.2.- https://unece.org/sites/default/files/2022- 05/ECE_MP.PP_2021_2_Add.2_R.pdf

18 │

c) общенациональная цифровая экологическая информационная система поддерживала общественный доступ к динамичной, в том числе в режиме реального времени, и исторической, актуальной, точной и контролируемого качества, всеобъемлющей, стандартизированной и функциональной экологической информации; чтобы данная информация могла быть обнаружена и доступна через Интернет в поддающихся машинной считке формах и форматах, отвечающих потребностям различных пользователей; d) следующие виды информации являются общедоступными, своевременно, через Интернет, предпочтительно через универсальную сетевую точку доступа: … vii) документация, относящаяся к стратегической экологической оценке или другим процессам подготовки планов, программ или политики в области окружающей среды с учетом положений статьи 7 Конвенции (например, публичные уведомления, вся другая соответствующая документация, включая оценку рисков и другие исследования, экономический анализ и допущения, комментарии третьих сторон, проекты и окончательные решения), если она хранится в электронной форме или может быть легко преобразована в электронную форму. Если она не доступна в электронном виде и не может быть преобразована в электронную форму с разумными затратами, ссылка на то, где можно получить доступ к такой документации (статьи 5 (3) (d) и 7 Орхусской конвенции)3. Одним из эффективных комплексных решений с использованием электронных средств информации может быть создание и внедрение централизованного электронного регистра СЭО. Такой электронный регистр может быть использован для оптимизации процедур СЭО и обеспечения требований по обеспечению доступа к информации в процессе принятия решений. Электронный регистр СЕО может быть как самостоятельно функционирующей системой, так и интегрироваться с другими регистрами (например, по ОВОС) или быть частью других регистров или систем электронного правительства.

Наличие документов и информации в электронном доступе, в свою очередь, может упростить обмен этими документами между органами власти на разных этапах процедуры СЭО, а также обеспечить доступ к ним не только непосредственно в течении процедуры СЭО, но и в дальнейшем, при осуществлении планов и программ, например, для целей мониторинга существенных экологических, в том числе связанных со здоровьем населения, последствий осуществления планов и программ.

Использование централизованного регистра СЭО может способствовать систематизации процедур СЭО на всех уровнях, улучшению обмена информацией на местном и центральном уровне, особенно в тех странах, в которых существует разделение полномочий по проведению СЭО на местном и национальном уровнях. Также, централизованный регистр СЭО может в целом способствовать прозрачности системы планирования и процессов принятия экологически значимых решений на национальном уровне, повышению качества СЭО, а также содействовать более эффективному выполнению обязательств по отчётности в рамках статьи 14.7 Протокола по СЭО4.

3 Там же, параграф 23 4 Среди выводов Четвёртого обзора реализации Протокола по стратегической экологической оценке (2019–2021 годы), которые должны быть подтверждены Совещанием Сторон Протокола на его пятой сессии (Женева, 12–15 декабря 2023 г.) указано, что «отсутствие

│ 19

Таким образом, в качестве модельного электронного регистра предлагается рассматривать централизованный электронный регистр который представляет собой электронную информационную или информационно-коммуникационную систему, обеспечивающую взаимодействие между субъектами СЭО, сбор, хранение, систематизацию информации и документов, которые создаются в процессе СЭО, а также в процессе мониторинга существенных экологических, в том числе связанных со здоровьем населения, последствий осуществления планов и программ, а также обнародование и/или открытый доступ к информации и документам (с учётом соответствующих требований и ограничений, предусмотренных законодательством).

Ключевыми функциями модельного электронного регистра можно выделить следующие:

• Систематизация информации и документов по СЭО в единую электронную структурированную базу данных по СЭО всех уровней (местный/региональный и общегосударственный) с возможностью поиска по определённым параметрам;

• Обеспечение обмена информацией и документами между субъектами СЭО (в т.ч. администрирование замечаний и предложений общественности и органов власти);

• Обеспечение своевременного информирования общественности;

• Обеспечение обнародования информации и документов;

• Постоянный открытый доступ к информации и документам по всем процедурам СЭО, а также к информации по мониторингу существенных экологических, в том числе связанных со здоровьем населения, последствий осуществления планов и программ;

• Размещение информации в рамках трансграничных консультаций (с случае проведения).

Исходя из технических решений и возможностей, кроме основных функций электронный регистр может иметь более расширенный функционал, включая дополнительные возможности для поиска и анализа информации, сопровождения проведения общественных обсуждений/слушаний, интеграции с картографической основой, а также содержать ссылки на законодательство и другие базы данных и источники информации и прочее. Например, новыми изменениями к законодательству про ОВОС в Украине предусматривается внедрение в Единый электронный регистр ОВОС адресной рассылки. Предполагается возможность физических и юридических лиц зарегистрироваться в Едином реестре ОВОС с целью адресного информирования об обнародовании в нем информации и документов.

центрального реестра или базы данных национальных процедур стратегической экологической оценки у многих Сторон осложнило отчетность о количестве процедур за отчетный период и привело к неточности информации». См. https://unece.org/environment/documents/2023/10/session-documents/fourth-review- implementation-protocol-strategic.

20 │

3. Национальные процедуры СЭО и предпосылки для создания электронных регистров

В данной части рассматриваются законодательные основы СЭО в Армении, Азербайджане, Грузии, Республике Молдова и Украине, а также национальные процедуры СЭО в контексте взаимодействия различных субъектов процедуры, обмена и обнародования информации посредством электронных средств коммуникации. Данный отчёт не ставит целью оценку соответствия процедур СЭО в охваченных странах Протоколу по СЭО, а призван выявить потребности и предпосылки для создания электронного регистра СЭО в соответствующих странах. В отношении Украины рассматривается недавно созданный Единый регистр СЭО. Схемы, приведённые в этом разделе, призваны проиллюстрировать взаимодействие субъектов на различных этапах СЭО и обнародование документов и информации посредством электронных средств коммуникации. Схемы созданы на основе анализа законодательства и информации, предоставленной странами5, носят демонстративный характер и могут не в полной мере описывать все аспекты взаимодействия, а также сформировавшуюся практику проведения отдельных процедур в странах.

3.1. Армения

Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО

Законодательной основой для проведения СЭО в Республике Армения выступает Закон «Об оценке воздействия на окружающую среду и экспертизе» (2014) (далее - Закон об ОВОС и экспертизе)6.

Согласно Закону об ОВОС и экспертизе СЭО это процесс определения и оценки воздействия на окружающую среду вследствие применения положений проекта основополагающего документа, который должен включать определение сферы охвата и положений отчета о СЭО, подготовку отчета о СЭО, обеспечение участия заинтересованной общественности и проведение профессиональных консультаций, учет положений отчета о СЭО, результатов участия общественности и консультаций в основополагающем документе. Основополагающим документом в соответствии с Законом об ОВОС и экспертизе является документ, имеющий потенциальное воздействие на окружающую среду (стратегия, концепция, схема использования природных ресурсов, проект, план, планировка, градостроительный программный документ) или любая поправка к документу, утвержденная законами Республики Армении или иными правовыми актами государственных органов или органов местного самоуправления.

СЭО и экспертиза проекта основополагающего документа проводятся до принятия основополагающего документа.

Согласно части 2 статьи 2 Закона об ОВОС и экспертизе этапами проведения СЭО являются:

• определение сферы охвата СЭО;

• составление отчета о СЭО;

5 Национальными координаторы по вопросам Конвенции Эспо и ее Протокола по СЭО 6 В данном отчёте рассматривался Закон в редакции от 3 мая 2023 года.

│ 21

• учет результатов СЭО в основополагающем документе.

На этих этапах государственный орган или орган местного самоуправления, внесший проект основополагающего документа (далее - инициатор) обязан:

1) обратиться в уполномоченный орган7 и уполномоченный орган в области управления здравоохранением за консультацией, а также обратиться в уполномоченные государственные органы в сферах, в которых разрабатывается основополагающий документ, и (или) органы местного самоуправления;

2) привлекать заинтересованную общественность, и при необходимости осуществлять трансграничные консультации.

Экспертиза отчета о СЭО проводится в срок до 80 рабочих дней со дня представления отчета о СЭО с сопроводительным письмом в уполномоченный орган (за исключением документов, имеющих трансграничное воздействие, по которым сроки предоставления заключения могут быть продлены), на основании чего уполномоченный орган предоставляет инициатору положительное или отрицательное экспертное заключение. Без положительного заключения государственной экспертизы запрещается утверждение основополагающего документа.

Уполномоченный орган, органы территориального управления, органы местного самоуправления и инициатор в целях обеспечения участия заинтересованной общественности в процедуре СЭО и экспертизы:

1) распространяют информацию о праве инициировать и участвовать в процедуре СЭО и экспертизы;

2) обеспечивают доступ к документам по СЭО, экспертизам и другой информации;

3) создают условия для участия заинтересованной общественности в слушаниях, информируют заинтересованную общественность о проекте основополагающего документа, ходе экспертизы через официальные сайты и другие средства распространения информации (пресса, другие средства массовой информации, доски объявлений органов местного самоуправления, зданий общественного значения). (См. Схему 2)

Предпосылки для создания электронного регистра СЭО Правительством Армении 11 февраля 2021 года принято Постановление N 183-Л «О стратегии цифровизации Армении, программе действий стратегии и показателях результатов» в котором определена необходимость развития цифровизации во различных сферах и обозначены основные направления и подходы к цифровизации на период до 2025 года.

В части законодательства в сфере СЭО, создание централизованного электронного регистра по СЭО или единой электронной базы данных/портала документов по СЭО Законом об ОВОС и экспертизе не предусмотрено. Закон об ОВОС и экспертизе предусматривает обмен информацией и подачу замечаний как на бумажных носителях, так и в электронной форме. Информирование общественности и обеспечение доступа к документации обеспечивается, в том числе, через веб-

7 Уполномоченный орган – орган государственного управления, разрабатывающий и реализующий политику Правительства в области обеспечения государственной экспертизы воздействия на окружающую среду;

22 │

страницы уполномоченного органа, органов территориального управления, органов местного самоуправления и инициатора.

Схема 2. Взаимодействие субъектов процедуры СЭО и экспертизы

│ 23

3.2. Азербайджан

Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО

Законодательной основой для проведения СЭО в Азербайджане выступает Закон Азербайджанской Республики «Об оценке воздействия на окружающую среду» (2018), а также принятые с целью его применения подзаконные акты, в том числе, Положение о проведении государственной и общественной экологической экспертизы, утвержденное постановлением Кабинета Министров Республики Азербайджан №184 от 21 мая 2020 года, Порядок проведения стратегической экологической оценки, утверждённый постановлением Кабинета Министров Азербайджанской Республики №354 от 15 сентября 2022 года, который непосредственно регулирует процедуру СЭО.

Закон Азербайджанской Республики «Об оценке воздействия на окружающую среду» определяет, что CЭО – это систематическая оценка стратегических документов, документов территориального планирования с точки зрения экологической безопасности в целях выявления и предотвращения потенциального воздействия на окружающую среду и здоровье человека и обеспечения эффективного использования окружающей среды. СЭО должна проводиться государственными органами, государственными агентствами и муниципалитетами, которые готовят стратегические документы (далее – органами планирования) на этапе подготовки стратегических документов. К стратегическим документам Закон относит планы, программы, стратегии и концепции в сферах сельского хозяйства, лесного хозяйства, рыболовства, энергетики, промышленности, транспорта, управления отходами и водными ресурсами, использования земельных ресурсов, туризма и телекоммуникаций.

Процесс СЭО начинается с момента обращения в Государственное агентство экологической экспертизы при Министерстве окружающей среды и природных ресурсов (далее – Агентство) за предварительной консультацией и заканчивается выдачей заключения экологической экспертизы.

Законодательство Азербайджана определят следующие этапы проведения СЭО:

• определение сферы охвата СЭО и предварительные консультации с Агентством; • анализ и оценка возможных последствий в соответствии с требованиями нормативных

правовых актов; • уточнение мер по охране окружающей среды на основе полученных данных; • разработка отчёта о СЭО; • организация общественных консультаций; • подготовка соответствующего окончательного обоснования выводов органами

планирования; • утверждение стратегического документа Министерством окружающей среды и

природных ресурсов (далее – Министерство); • мониторинг результатов СЭО.

На этапе предварительных консультаций по СЭО и определения сферы охвата СЭО орган планирования проводит предварительные консультации с Агентством с целью предварительного определения содержания, сферы охвата и методов СЭО, а также обеспечения полноты и точности информации, которая должна быть включена в отчёт о СЭО. Предварительные консультации инициируются органом планирования на этапе подготовки стратегического документа путем подачи заявления в Агентство об уведомлении о намерениях планирования и определении сферы охвата СЭО. После утверждения Агентством документа по определению сферы охвата СЭО орган

24 │

планирования должен разместить этот документ на своей веб-странице вместе с проектом стратегического документа и проинформировать общественность (См. Схему 3).

Схема 3. Предварительные консультации по СЭО и определение сферы охвата СЭО

На следующих этапах орган планирования готовит отчёт о СЭО и представляет его в Агентство для проведения экологической экспертизы. Когда орган планирования представляет проект документа СЭО в Агентство, он также должен сообщить о дате, времени и месте проведения общественных консультаций. Представитель Агентства должен быть официально уведомлен органом планирования об общественных слушаниях не менее чем за 7 рабочих дней до начала мероприятия для обеспечения присутствия представителя Агентства на консультациях. Также, орган планирования не менее чем за 7 рабочих дней до консультации размещает на своем официальном сайте, а также в СМИ объявление о проведении общественных консультаций и проекте документа СЭО, цели, дате, месте и времени проведения собрания, его формате, а также информацию о месте, где можно ознакомиться с соответствующими документами. Общественные слушания по стратегическим документам регионального значения организовываются в нескольких территориальных единицах (районах и городах), к обсуждению привлекаются и информируются физические и юридические лица, находящиеся в соответствующих административно-территориальных единицах, а также собственники недвижимости.

│ 25

Заключение экологической экспертизы, выданное на отчет о СЭО, размещается на официальном сайте Агентства. (См. Схему 4) Схема 4. Рассмотрение отчёта о СЭО: консультации с общественностью и экологическая экспертиза

Стратегический документ составляется с учетом консультаций с соответствующими государственными органами (ведомствами) и общественных консультаций по СЭО, а также мнений и предложений, высказанных после получения заключения государственной экологической экспертизы. Орган планирования должен подготовить окончательный документ с обоснованием по вопросам, рассмотренным и не рассмотренным в отношении предложений по стратегическому документу и представить на согласование Министерству стратегический документ, который должен сопровождаться отчётом о СЭО или кратким обзором её окончательных результатов, заключением государственной экологической экспертизы и окончательным обосновывающим документом органа планирования. После утверждения стратегического документа орган планирования обобщает полученные окончательные результаты, включая выводы СЭО и заключение экологической экспертизы, размещает их на официальном сайте и информирует об этом Агентство в письменной форме (См. Схему 5).

26 │

Схема 5. Окончательное решение по СЭО

Предпосылки для создания электронного регистра СЭО Правовая база для формирование электронного правительства в Азербайджане была основана Указом Президента Азербайджанской Республики «Об утверждении Государственной программы (Е-Азербайджан) по развитию связи и информационных технологий в Азербайджанской Республике в 2010-2012 годах», а также Указом Президента Азербайджанской Республики «Об утверждении «Некоторые меры по предоставлению электронных услуг государственными органами» от 23 мая 2011 года8. В дальнейшем было принято ряд Указов Президента и других нормативных правовых актов, которыми определялись стратегии развития электронного правительства и цифровизации в стране, а также их отдельные аспекты. Например, Указ Президента Азербайджанской Республики от 14 марта 2018 года № «О мерах, связанных с развитием электронного правительства и переходом на цифровое правительство» и другие.

В Азербайджане в настоящее время отсутствует централизованная электронная база данных процедур СЭО и соответствующих документов в рамках таких процедур. Более того, из вышерассмотренных схем взаимодействия можно сделать вывод, что даже в рамках одной процедуры СЭО часть документов должна размещается на веб-странице органа планирования, а часть – на веб-странице Агентства9. При этом, также следует учитывать, что органами планирования могут быть различные государственные органы, государственные агентства и муниципалитеты, которые готовят стратегические документы, и соответственно, на их веб- страницах размещается информация, касающаяся только их сферы компетенции.

8 https://www.e-gov.az/en/content/read/2 9 Согласно предоставленной Азербайджаном в рамках подготовки данного отчёта информации, в настоящее время веб- страница Агентства отсутствует.

│ 27

3.3. Грузия

Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО

Законодательной основой для проведения СЭО в Грузии выступает Кодекс об экологической оценке (2017)10 (далее- Кодекс). Согласно Кодексу СЭО включает в себя определение сферы охвата (скопинг), подготовку и рассмотрение отчета о СЭО, участие общественности и консультации с ней, учет информации, отраженной в отчете о СЭО в процессе принятия решения по стратегическому документу, а также рекомендаций, выданных Национальным агентством окружающей среды11 (далее- Агентство) и Министерством лиц, вынужденно перемещенных с оккупированных территорий, труда, здравоохранения и социальной защиты Грузии (далее- Министерство здравоохранения) в связи с проектом стратегического документа и отчетом о СЭО, и результатов участия общественности при принятии/утверждении стратегического документа, и предоставление общественности и заинтересованным ведомствам информации о принятом решении. К этапам СЭО, согласно Кодексу, относятся:

• подача органом планирования12 заявления в Агентство и Министерство здравоохранения; • процедура скопинга; • подготовка органом планирования или (и) консультантом отчета о СЭО; • оценка органом планирования информации, полученной в результате подготовки отчета о

СЭО, участия общественности и проведения консультаций; • в случае необходимости, осуществление процедуры оценки трансграничного воздействия

на окружающую среду; • участие общественности; • выдача рекомендаций Агентством и Министерство здравоохранения в связи с проектом

стратегического документа и отчетом о СЭО.

В целях определения необходимости проведения СЭО (незначительных изменений в стратегическом документе, а также стратегического документа, касающегося территории самоуправляющейся общины), осуществляется процедура предварительной оценки (скрининга) (См. Схему 6). В таких случаях, орган планирования обращается в Агентство и Министерство здравоохранения с заявлением о скрининге и представляет концепцию или проект стратегического документа (в нем должна содержаться краткая информация о целях, задачах и мероприятиях, предусмотренных стратегическим документом) по возможности на раннем этапе его разработки как в материальной, так и в электронной форме.

10 В данном Отчёте рассматривалась редакция Кодекса по состоянию на 29.06.2023 - https://matsne.gov.ge/ru/document/view/3691981?publication=12 11 Юридическое лицо публичного права, входящее в систему Министерства охраны окружающей среды и сельского хозяйства Грузии. 12 Орган планирования – административный орган или другая уполномоченная организация, которые согласно соответствующему нормативному акту ответственны за подготовку стратегического документа.

28 │

Агентство направляет указанное заявление и концепцию или проект стратегического документа в Центр природоохранной информации и образования13 (далее- Центр). Центр обеспечивает их размещение на портале экологической информации (далее- Портал). Также, Министерство здравоохранения и орган планирования размещают указанное заявление и концепцию или проект стратегического документа на своих официальных веб-страницах, а Центр также обеспечивает размещение заявления о скрининге на информационной доске исполнительного органа или (и) представительного органа соответствующего муниципалитета. После размещения заявления о скрининге и концепции или проекта стратегического документа на указанных веб-страницах и информационной доске общественность в течении установленного строка вправе представлять соображения и замечания в связи с указанными документами. Агентство и Министерство здравоохранения рассматривают соображения и замечания, представленные общественностью, и при наличии соответствующего основания учитывают их в процессе принятия решения. Агентство после принятия решений о скрининге направляет указанные решения, а также представленные общественностью соображения и замечания в Центр, который обеспечивает их размещение на природоохранном информационном портале. Также, после принятия решений о скрининге Министерство здравоохранения и орган планирования размещают указанные решения, а также представленные общественностью соображения и замечания на своих официальных веб- страницах, а Центр также обеспечивает размещение указанных документов на информационной доске исполнительного органа или (и) представительного органа соответствующего муниципалитета. Схема 6. Скрининг

13 Юридическое лицо публичного права, входящее в систему Министерства охраны окружающей среды и сельского хозяйства Грузии.

│ 29

Подобный алгоритм взаимодействия субъектов СЭО и обнародования информации с использованием электронных ресурсов предусмотрен и на стадии скопинга (См. Схему 7). Исключение составляют положения про размещение Центром, Министерством здравоохранения и органом планирования предоставленных общественностью в процессе скопинга соображений и замечаний. В Кодексе в процедуре скопинга, в отличии от процедуры скрининга, про размещение соображений и замечаний общественности на соответствующих электронных ресурсах не указано, однако предусмотрено, что Агентство и Министерство здравоохранения рассматривают соображения и замечания, представленные общественностью, и при наличии соответствующего основания учитывают их в процессе принятия решения. Кодексом предусмотрена возможность объединить стадии скрининга и скопинга и подать одновременно заявление о скрининге и заявление о скопинге. Также, в случае проведения процедуры оценки трансграничного воздействия на окружающую среду, Агентство определяет для органа планирования обязательство по отражению в отчете о скопинге/заявлении о скопинге результатов консультаций, связанных с процедурой оценки трансграничного воздействия. Схема 7. Скопинг

30 │

После подготовки органом планирования отчёта о СЭО начинается процедура рассмотрения Агентством и Министерством здравоохранения отчёта и выдачи рекомендаций (См. Схему 6). Орган планирования обращается в Агентство и Министерство здравоохранения с заявлением и представляет отчет о СЭО и проект стратегического документа как в материальной, так и электронной форме. Заявление должно содержать также информацию о предполагаемом времени, месте и порядке проведения публичного обсуждения. Агентство направляет данное заявление и прилагаемые документы в Центр. Центр обеспечивает размещение указанных документов на природоохранном информационном портале. Министерство здравоохранения и орган планирования размещают данное заявление и прилагаемые документы на своих официальных веб-страницах, а Центр также обеспечивает размещение этого заявления на информационной доске исполнительного органа или (и) представительного органа соответствующего муниципалитета. Также, Агентство в целях рассмотрения отчета о СЭО и проекта стратегического документа создает экспертную комиссию, которая представляет заключение об экспертизе в Агентство. Общественность вправе представить в Агентство и Министерство здравоохранения соображения и замечания в связи с отчетом о СЭО и проектом стратегического документа. Агентство и Министерство здравоохранения рассматривают представленные общественностью соображения и замечания и при наличии соответствующего основания учитывают их в процессе принятия решения. Орган планирования также проводит публичное обсуждение отчета о СЭО с участием Агентства и Министерства здравоохранения. После проведения публичного обсуждения отчета о СЭО орган планирования обеспечивает составление протокола о результатах публичного обсуждения отчета о СЭО представляет протокол о результатах публичного обсуждения отчета о СЭО в Агентство и Министерство здравоохранения. Агентство и Министерство здравоохранения в пределах своей компетенции выдают соответствующие рекомендации в связи с отчетом о СЭО и проектом стратегического документа и направляют указанные рекомендации органу планирования. Агентство также направляет выданные рекомендации в связи с отчетом о СЭО и проектом стратегического документа в Центр. Центр обеспечивает их размещение на природоохранном информационном портале. Министерство здравоохранения и орган планирования размещают указанные рекомендации на своих официальных веб-страницах, а Центр также обеспечивает размещение этих рекомендаций на информационной доске исполнительного органа или (и) представительного органа соответствующего муниципалитета. В случае проведения процедуры оценки трансграничного воздействия на окружающую среду Агентство обеспечивает учет результатов трансграничных консультаций в процессе выдачи рекомендаций.

│ 31

Схема 8. Рассмотрение отчёта о СЭО и проекта стратегического документа

После принятия/утверждения стратегического документа орган планирования обеспечивает размещение решения о принятии/утверждении стратегического документа на своей официальной веб-странице и направляет указанное решение в Агентство и Министерство здравоохранения. Агентство направляет решение о принятии/утверждении стратегического документа в Центр. Министерство здравоохранения обязано размещать решение о принятии/утверждении стратегического документа на своей официальной веб-странице, а Центр обеспечивает размещение информации о вышеуказанном решении на природоохранном информационном портале и информационной доске исполнительного органа или (и) представительного органа соответствующего муниципалитета. При наличии соответствующей возможности и в случае необходимости Агентство, опираясь на имеющиеся у него данные, обеспечивает проведение самостоятельного анализа значительного воздействия на окружающую среду, вызванного осуществлением стратегического документа, и предоставление Центру результатов указанного анализа. Центр в целях информирования

32 │

общественности обеспечивает систематическое размещение итогов последующего анализа результатов осуществления стратегического документа на природоохранном информационном портале.

Предпосылки для создания электронного регистра СЭО В соответствии с пунктом 2(г) статьи 4 Кодекса к компетенции Министерства охраны окружающей среды и сельского хозяйства Грузии отнесено создание единой базы данных по вопросам, входящим в компетенцию Министерства, и обеспечение безопасности, публичности и доступности хранящейся в ней информации, в целях обеспечения доступа к информации, эффективности государственного управления и участия общественности. В настоящее время документы по СЭО подаются как в цифровой форме, так и в бумажном виде. Цифровой материал размещается на официальных веб-страницах органов и веб-странице Агентства и рассылается в ведомства/различные организации для ознакомления. Таким образом, все административные процедуры проводятся в цифровом формате. Однако специальная программа/IT-инструменты для обеспечения большей цифровизации процедуры СЭО не используется. Также, в настоящее время отсутствует централизованная электронная база данных процедур СЭО и соответствующих документов в рамках таких процедур. Более того, из вышерассмотренных схем взаимодействия можно сделать вывод, что даже в рамках одной процедуры СЭО часть документов размещается на веб-странице органа планирования или Портале, а часть, касающаяся здравоохранения, на веб-странице органа планирования и Министерства здравоохранения. При этом следует учитывать, что органами планирования могут быть различные административные органы или другие уполномоченные организации, ответственные за подготовку стратегического документа, и соответственно, на их веб-страницах размещается информация, касающаяся только их сферы компетенции. С 1 июля 2023 года Центр экологической информации и образования (www.eiec.gov.ge) отвечает за вовлечение общественности в процесс принятия решений в процедурах, предусмотренных Кодексом, обеспечение доступа к информации и организацию общественных слушаний. Согласно предоставленной Грузией в рамках подготовки данного отчёта информации, в Грузии разрабатывается Публичный портал экологической оценки, который объединит документы по ОВОС/СЭО на единой платформе и обеспечит общественности систематический и упрощенный доступ к информации, связанной с процедурами ОВОС и СЭО.

│ 33

3.4. Республика Молдова

Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО

Процедура СЭО в Республике Молдова регулируется Законом о стратегической экологической оценке (2017) с соответствующими изменениями14 (далее- Закон о СЭО). Согласно Закону о СЭО стратегическая экологическая оценка – это оценка вероятных последствий для окружающей среды, в том числе для здоровья населения, в результате реализации документов политик и планирования, которая включает определение сферы охвата доклада о СЭО и его разработку, проведение консультаций с заинтересованными органами публичной власти и с заинтересованной общественностью, а также проведение при необходимости трансграничных консультаций и принятие во внимание выводов по докладу о СЭО и результатов участия и публичных консультаций в рамках документа политик и планирования. К документам политик и планирования Закон о СЭО относит документы публичных политик и документы планирования, которые: a) должны быть разработаны в соответствии с положениями нормативных или административных актов; b) должны быть разработаны органом публичной власти и утверждены Правительством или в рамках законодательной процедуры – Парламентом; c) должны быть разработаны и утверждены органом местного публичного управления. Предварительная оценка (определение необходимости проведения СЭО) и СЭО проводятся на начальном этапе разработки концепции документа политик и планирования или изменений к нему. Процесс СЭО включает следующие этапы:

• определение сферы охвата доклада о СЭО; • разработка доклада о СЭО; • анализ качества доклада о СЭО; • анализ интеграции аспектов охраны окружающей среды в проект документа политик и

планирования; • выдача экологического заключения15; • утверждение документов политик и планирования; • мониторинг существенного воздействия документа политик и планирования на

окружающую среду, в том числе на здоровье населения.

Законом о СЭО среди полномочий инициатора и компетентного органа предусмотрено размещение информации о процедуре СЭО на их веб-страницах. В соответствии з Законом о СЭО инициатором является орган центрального или местного публичного управления, который инициирует документы политик или планирования и отвечает за

14 В данном отчёте рассматривался закон включительно с изменениями от 30.09.2022, которые вступили в силу с 21.10.2023 года. 15 Экологическое заключение – технико-правовой документ, выдаваемый компетентным органом, подтверждающий, что доклад о СЭО соответствует требованиям Закона о СЭО, а аспекты охраны окружающей среды интегрированы в документы политики или планирования, представленные для утверждения, и не имеют неблагоприятных последствий для окружающей среды, в том числе для здоровья населения.

34 │

их разработку (далее - инициатор). Компетентными органами, согласно Закону о СЭО, являются Министерство окружающей среды (МОС) – для документов политик и планирования национального уровня, включая отраслевые и межотраслевые документы, утверждаемые Правительством или Парламентом и Агентство окружающей среды (АОС) – для документов политик и планирования местного уровня, утверждаемых органом местного публичного управления. Инициатор под руководством компетентного органа информирует (через публичные объявления в местных и национальных газетах и размещение информации на официальных веб-страницах) и организует участие заинтересованной общественности на всех этапах СЭО. На всех этапах СЭО инициатор размещает и сохраняет информацию о процедуре СЭО на своей официальной веб- странице в разделе, посвященном СЭО. Для документов политик и планирования, разрабатываемых в областях сельского хозяйства, лесного хозяйства, рыболовства, энергетики, промышленности, транспорта, управления отходами, управления водными ресурсами, электронных коммуникаций, туризма, землепользования, городского и сельского планирования (документация по градостроительству и обустройству территории, включая планы по градостроительству и обустройству территории и программы градостроительства и обустройства территории на национальном, региональном, районном, местном уровнях, а также для общих и зональных градостроительных планов), которые устанавливают рамки для осуществления планируемых видов деятельности, подлежащих оценке воздействия на окружающую среду, процедура СЭО начинается с определения сферы охвата доклада о СЭО. В случае, когда эти документы политик и планирования определяют использование малых территорий на местном уровне, а также предусматривают незначительные изменения документов политик и планирования, а также когда другие документы политик и планирования, кроме вышеуказанных, устанавливают рамки для осуществления любого вида планируемой деятельности, для которой необходим разрешительный документ, проводится предварительная оценка (определение необходимости проведения СЭО). Для проведения предварительной оценки инициатор на этапе формулирования документов политики или планирования представляет в компетентный орган информацию, необходимую для предварительной оценки. Компетентный орган размещает эту информацию на своей официальной веб-странице и инициирует консультации с центральными и местными органами публичной власти, определенными в качестве заинтересованных органов публичной власти (ЗО), и с Министерством здравоохранения (МЗ) Инициатор информирует заинтересованную общественность. Заинтересованная общественность может представить свои комментарии и предложения компетентному органу в течение 10 дней со дня информирования. Компетентный орган с учетом заключения заинтересованных органов публичной власти, а также комментариев и предложений, полученных от заинтересованной общественности, принимает решение по предварительной оценке, направляет его инициатору и размещает на своей официальной веб-странице, а также по своему месту нахождения. Инициатор после получения решения по предварительной оценке размещает его на своей официальной веб-странице и информирует заинтересованную общественность в отношении

│ 35

решения по предварительной оценке, в том числе, посредством публичных объявлений в местных и национальных газетах (См. Схему 9). Для документов политик и планирования в отношении которых было принято решение о проведении СЭО, компетентный орган определяет сферу охвата доклада о СЭО и выносит решение об определении сферы охвата доклада о СЭО согласно требованиям к определению сферы охвата. Схема 9. Предварительная оценка

Подобный алгоритм взаимодействия субъектов СЭО и обнародования информации с использованием электронных ресурсов предусмотрен и на этапе определения сферы охвата доклада о СЭО (См. Схему 10). Исключение составляет то, что на стадии определения сферы охвата доклада о СЭО в Законе о СЭО не предусмотрено размещения решения компетентного органа на его веб- странице, а указано, что компетентный орган передает в письменной форме решение об определении сферы охвата доклада о СЭО инициатору, а последний размещает его на своей официальной веб-странице и информирует заинтересованную общественность. Также, на этапе определения сферы охвата доклада о СЭО результаты консультаций оформляются протоколом, который содержит перечень согласованной информации, необходимой для подготовки доклада о СЭО и являющейся его составной частью.

36 │

Схема 10. Определение сферы охвата доклада о СЭО

На следующих этапах инициатор готовит доклад о СЭО и передает на рассмотрение компетентному органу проект документа политик и планирования и доклад о СЭО (См. Схему 11), размещает их на своей официальной веб-странице, информирует заинтересованную общественность об указанных документах и сроках представления комментариев и предложений по их поводу компетентному органу, а также о проведении публичных консультаций. Компетентный орган после получения доклада о СЭО и проекта документа политик и планирования передает их Экспертной комиссии16, размещает доклад о СЭО и проект документа политик и планирования на своей официальной веб-странице, а также направляет их неправительственным организациям, содействующим охране окружающей среды. В зависимости от характера и сложности документа политик и планирования компетентный орган может привлечь к анализу качества доклада о СЭО представителей заинтересованных органов публичной власти (ЗО), а также других независимых экспертов. Комментарии и предложения заинтересованной общественности по докладу о СЭО и проекту документа политик и планирования могут быть представлены компетентному органу в течение 30

16 Экспертная комиссия создается приказом министра окружающей среды и состоит из экспертов, назначенных Министерством здравоохранения, Министерством окружающей среды, Министерством инфраструктуры и регионального развития, Агентством «Moldsilva», Национальным археологическим агентством, Академией наук Молдовы, Агентством по геологии и минеральным ресурсам, Агентством «Apele Moldovei», Государственной гидрометеорологической службой.

│ 37

дней со дня их размещения на официальной веб-странице компетентного органа. Компетентный орган направляет копии данных комментариев и предложений Экспертной комиссии. Экспертная комиссия в письменной форме представляет компетентному органу подробное и обоснованное мнение по поводу качества доклада о СЭО. В случае если качество доклада о СЭО является несоответствующим, компетентный орган после получения мнения Экспертной комиссии составляет заключение о качестве доклада о СЭО и указывает аспекты отчета, которые считает несоответствующими, со ссылкой на комментарии и предложения, полученные от заинтересованной общественности, и на мнение Экспертной комиссии, а также на рекомендуемые меры по доработке доклада о СЭО. В случае если доклад о СЭО возвращен на доработку, инициатор дорабатывает и повторно подаёт доклад. После получения заключения о качестве доклада о СЭО от компетентного органа инициатор размещает копию доклада на своей официальной веб-странице и информирует заинтересованную общественность. Одновременно с оценкой качества доклада о СЭО Экспертная комиссия оценивает проект документа политик и планирования. После признания доклада о СЭО соответствующим требуемому качеству, Экспертная комиссия представляет компетентному органу аргументированное мнение по проекту документа политик и планирования. В зависимости от мнения Экспертной комиссии по проекту документа политик и планирования, а также от результатов публичных консультаций и при необходимости от трансграничных консультаций компетентный орган выдает экологическое заключение, в письменной форме уведомляет инициатора об экологическом заключении и размещает его на своей официальной веб- странице. Инициатор также размещает экологическое заключение на своей официальной веб- странице и информирует об этом заинтересованную общественность.

38 │

Схема 11. Рассмотрение качества доклада о СЭО и проект документа политик и планирования

После утверждения в установленном действующим законодательством порядке документа политик или планирования инициатор размещает на своей официальной веб-странице и информирует заинтересованную общественность, заинтересованные органы публичной власти и предоставляет им в распоряжение экологическое заключение, а также: а) документ политик или планирования в утвержденном виде; b) информацию о том, каким образом интегрированы в документ политик или планирования вопросы окружающей среды, об экологических, в том числе связанных со здоровьем населения, последствиях документа политик или планирования; с) информацию о консультациях с общественностью и заинтересованными органами публичной власти, о результатах трансграничных консультаций. Инициатор в пределах своей компетенции и в соответствии с установленной программой проводит мониторинг существенных экологических, в том числе связанных со здоровьем населения, последствий реализации документа политик или планирования с целью обнаружения на ранней стадии непредусмотренных неблагоприятных последствий и обеспечения возможности для принятия надлежащих мер по исправлению положения. Инициатор представляет результаты мониторинга в соответствии с графиком и формой, которые предусмотрены утвержденным документом политик и планирования, компетентному органу и публикует их на своей официальной веб-странице.

│ 39

Предпосылки для создания электронного регистра СЭО Общую правовую основы для разработки и функционирования электронных регистров в Молдове формируют Закон об информатизации и государственных информационных ресурсах (2003), Закон о регистрах (2007), Постановление Правительства № 562 от 22.05.2006 о создании государственных автоматизированных информационных систем и ресурсов, Постановление Правительства № 153 от 25.08.2021 об утверждении Концепции Информационной системы «Регистр государственных информационных ресурсов и систем» и Положения о порядке ведения Регистра государственных информационных ресурсов и систем. Законом о СЭО также предусмотрено, что компетентный орган создает, развивает и ведет электронный регистр учета информации о СЭО и предоставляет его в распоряжение общественности путем размещения на официальной веб-странице в разделе, посвященном СЭО (ст. 84)17. В настоящее время в Молдове отсутствует централизованная электронная база данных процедур СЭО и соответствующих документов в рамках таких процедур. Из вышерассмотренных схем взаимодействия можно сделать вывод, что информация и документы в рамках процедуры СЭО размещаются на веб-страницах инициатора и компетентного органа. При этом следует учитывать, что инициаторами могут быть различные органы центрального или местного публичного управления, которые инициируют документы политик или планирования и отвечают за их разработку, соответственно, на веб-страницах инициаторов размещается информация, касающаяся только их сферы компетенции. Также на различных веб-страницах размещается информация в зависимости от статуса документа и вида компетентного органа, которыми проводится рассмотрение: для документов политик и планирования национального уровня, включая отраслевые и межотраслевые документы, утверждаемых Правительством или Парламентом – на веб-странице Министерства окружающей среды, а для документов политик и планирования местного уровня, утверждаемых органами местного публичного управления – на веб-странице Агентства окружающей среды.

17 Данное положение вступило в силу с 21.10.2023 года.

40 │

3.5. Украина

Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО

Процедура СЭО в Украине регулируется Законом Украины «О стратегической экологической оценке» (2018)18. Также, принято ряд подзаконных актов, регулирующих отдельные вопросы осуществления СЭО, среди которых Порядок осуществления мониторинга последствий выполнения документа государственного планирования для окружающей среды, в том числе для здоровья населения, утверждённый Постановлением Кабинета Министров Украины от 16.12.2020 №1272, Порядок ведения Единого регистра стратегической экологической оценки, утвержденный Постановлением Кабинета Министров Украины от 2 мая 2023 г. № 430 и другие. Правовую основу и общие требования в отношении разработки и функционирования государственных электронных регистров в Украине определяет Закон Украины «О публичных электронных регистрах» (2021). Согласно Закону Украины «О стратегической экологической оценке» СЭО это процедура определения, описания и оценки последствий выполнения документов государственного планирования для окружающей среды, в том числе для здоровья населения, оправданных альтернатив, разработки мер по предотвращению, уменьшению и смягчению возможных негативных последствий, которая включает определение сферы охвата СЭО, составление отчета о СЭО, проведение общественного обсуждения и консультаций (при необходимости - трансграничных консультаций), учет в документе государственного планирования отчета о СЭО, результатов общественного обсуждения и консультаций, информирование об утверждении документа государственного планирования и осуществляется в порядке, определенном настоящим Законом. Документами государственного планирования в соответствии с Законом Украины «О стратегической экологической оценке» являются стратегии, планы, схемы, градостроительная документация, общегосударственные программы, государственные целевые программы и другие программы и программные документы, включая изменения к ним, разрабатываемые и/или подлежащие утверждению органом государственной власти, органом местного самоуправления. Этапы СЭО включают: 1) определение сферы охвата СЭО; 2) составление отчета о СЭО; 3) проведение общественного обсуждения и консультаций с уполномоченными органами в сфере охраны окружающей среды и здоровья, трансграничных консультаций; 4) учет отчета о СЭО, результатов общественного обсуждения и консультаций; 5) информирование об утверждении документа государственного планирования; 6) мониторинг последствий выполнения документа государственного планирования для окружающей среды, в том числе для здоровья населения. До мая 2023 года обмен информацией и документами в рамках процедуры СЭО осуществлялся путём пересылки соответствующих документов в электронной или бумажной форме между субъектами СЭО посредством электронной или обычной почты. В целях проведения консультаций

18 В данном Отчёте рассматривалась редакция Закона по состоянию на 9.07.2023 - https://zakon.rada.gov.ua/laws/show/2354-19#Text

│ 41

заказчики19 подавали соответствующие документы уполномоченным органам на национальном уровне (Министерство защиты окружающей среды и природных ресурсов и Министерство охраны здоровья) или уполномоченным территориальным органам (областного уровня). Обнародование документов осуществлялось на веб-страницах заказчиков и соответствующих органов, с которыми проводились консультации. Также заказчики дополнительно осуществляли информирование общественности посредством печатных средств массовой информации. Таким образом, документы по СЭО размещались на веб-страницах многочисленных и разнообразных органов (в зависимости от заказчика и статуса документа государственного планирования). Более того, в рамках одной процедуры СЭО часть документов размещалась только на веб-странице заказчика, а другая часть – исключительно на веб-страницах органов, с которыми проводились консультации. В мае 2023 году вступили в силу изменения к Закону Украины «О стратегической экологической оценке», внедрившие положения об Едином регистре СЭО. Вследствие таких изменений процедура СЭО и обмен документами на всех этапах осуществляется через электронный регистр – Единый регистр СЭО (далее – Регистр). С мая текущего года все документы по СЭО, создаваемые соответствующими органами на всех уровнях, подаются в электронной форме20 и находятся в Регистре в свободном доступе21. Обнародование документов по СЭО в сети Интернет осуществляется посредством Регистра (авторизированный доступ), а также на официальных веб- страницах заказчиков (неавторизированный доступ). На местном уровне в сельских населенных пунктах информирование также осуществляется через доски объявлений.

Единый регистр СЭО Согласно законодательству Украины, Регистр – это единая информационно-коммуникационная система, обеспечивающая создание, просмотр, сбор, внесение, накопление, обработку, использование, рассмотрение, хранение, защиту, учет и предоставление информации в сфере СЭО. Регистр был создан с использованием программного обеспечения, обеспечивающего его совместимость и электронное информационное взаимодействие в режиме реального времени с другими электронными информационными системами и сетями, составляющими информационный ресурс государства, в том числе с Единым регистром по оценке воздействия на окружающую среду, градостроительным кадастром, Государственным земельным кадастром и т.д. Регистр являет собой электронную базу данных информации и документов, создаваемых и загружаемых через электронные кабинеты пользователей, которые обеспечивают электронное взаимодействие между субъектами СЭО.

19 Заказчик - орган исполнительной власти или орган местного самоуправления, отвечающий за разработку документов государственного планирования и осуществляющий общее руководство и контроль за их выполнением, или другой определенный законодательством заказчик документов государственного планирования; 20 За исключением графических материалов градостроительной документации, которые подаются в бумажной форме. На государственном уровне существуют предпосылки для перехода на цифровой формат и в части графических материалов градостроительной документации. 21 В настоящее время доступ к информации в Регистре требует авторизации пользователя, что может не в полной мере соответствовать подходам «свободного доступа» общественности к регистрам, предусмотренным Орхусской конвенцией.

42 │

Регистр также является функциональным модулем Единой экологической платформы "ЭкоСистема", предназначенным для официального обнародования документации о СЭО, обеспечении информирования субъектов СЭО и других заинтересованных сторон о ходе и результатах осуществления СЭО. Регистр ведется для обеспечения:

• доступа субъектов СЭО к актуальной информации об осуществлении СЭО (нормативно- правовые акты, методическое обеспечение, справочная информация);

• электронного взаимодействия между субъектами СЭО, представления и обработки сведений заказчиками документов государственного планирования и органами, с которыми проводятся консультации, через электронный кабинет пользователя;

• осуществления консультаций с уполномоченными органами и трансграничных консультаций;

• свободного доступа всех заинтересованных сторон к информации о СЭО документов государственного планирования;

• проведения общественного обсуждения в процессе СЭО и предоставления предложений и замечаний общественности.

Держателем и администратором Регистра, обеспечивающим ведение Регистра, а также ответственным за техническое, технологическое и программное обеспечение Регистра, хранение и защиту данных, содержащихся в Регистре, является Министерство защиты окружающей среды и природных ресурсов Украины. Заказчики и органы, с которыми проводятся консультации, имеют доступ в Регистр с правом внесения сведений в соответствии со своими полномочиями через электронные кабинеты пользователей. При внесении в Регистр заказчиком или органом, с которым проводятся консультации, соответствующих сведений, программные средства ведения Регистра автоматически создают сообщения, направляемые через электронные кабинеты пользователей соответственно органам, с которыми проводятся консультации, или заказчику. Такие сообщения включают дату и время размещения в Регистре соответствующих сведений и ссылку для ознакомления с ними в Регистре. Общественность имеет доступ к информации, размещённой в Регистре, через Интернет (авторизированный доступ). Также информирование общественности осуществляется через другие средства (веб-страницы заказчиков, доски объявлений). Предложения и замечания общественности в процессе СЭО подаются в электронной или бумажной форме заказчику, который вносит их в Регистр (См. также Схему 12). В Регистр вносятся: 1) заявление об определении сферы охвата СЭО и уведомление об его обнародовании; 2) проект документа государственного планирования, отчет о СЭО и уведомление об обнародовании указанных документов; 3) замечания и предложения общественности; справка об общественном обсуждении; 4) замечания и предложения органов исполнительной власти; справка о консультациях; 5) утвержденный документ государственного планирования (кроме информации, которая согласно закону составляет государственную тайну или относится к информации с ограниченным доступом) и решение об его утверждении; 6) информация о трансграничных консультациях (в случае их проведения);

│ 43

7) результаты мониторинга последствий выполнения документа государственного планирования для окружающей среды, в том числе для здоровья населения, информация о не предусмотренных отчетом о СЭО негативных последствиях выполнения документа государственного планирования для окружающей среды, в том числе для здоровья населения (в случае их выявление), и о мерах, предпринятых для их устранения; 8) рекомендации по необходимости осуществления СЭО незначительных изменений к документу государственного планирования, относительно которого ранее осуществлялась СЭО, и решение о неосуществлении СЭО таких изменений. Информация, внесенная в Единый регистр СЭО, является публичной и открытой. Доступ к ней обеспечивается через Интернет, однако, как указывалось выше, требует авторизации. Схема 12. Взаимодействие субъектов процедуры СЭО после внедрения Регистра

3.6. Заключение

Во всех рассматриваемых странах, кроме Украины, в настоящее время отсутствуют централизованные электронные регистры СЭО. Доступ к информации и документам в процессе СЭО в рассматриваемых странах осуществляется, в том числе, путём их размещения на официальных веб-страницах органов планирования, соответствующих природоохранных ведомств, органов здравоохранения и других органов. При этом органами планирования могут быть различные административные органы или другие уполномоченные организации, ответственные за подготовку планов и программ, и соответственно, на их веб-страницах размещается информация, касающаяся только их сферы компетенции. А в части размещения информации в рамках СЭО на веб-страницах природоохранных органов и органов здравоохранения, в ряде стран существует разделение по статусу плана или программы и уровню уполномоченного органа (общенациональный, региональный/местный), с которым проводится консультирование, и информация размещается также на различных веб-страницах соответствующих органов.

44 │

Распространение информации на веб-страницах различных органов само по себе не противоречит подходам к широкому информированию общественности, однако необходимость поиска информации на веб-страницах различных органов (в отдельных случаях даже в рамках одной процедуры СЭО) может существенно снижать эффективность информирования и возможности для участия общественности. Создание централизованной базы данных информации по СЭО может способствовать решению этой проблемы. Во всех рассматриваемых странах происходят процессы цифровизации, многие страны переходят на электронный документооборот, а также создают системы электронного правительства. Таким образом в странах существуют предпосылки для создания электронных регистров СЭО.

4. Рекомендации

4.1. Юридические рекомендации

При разработке и функционировании электронных реестров СЭО целесообразно учитывать следующие рекомендации:

a) При рассмотрении вопроса о создании электронных регистров СЭО рекомендуется разрабатывать системы, которые смогут обеспечить не только накопление и доступ к информации о СЭО, а и более эффективное взаимодействие между субъектами СЭО, консультации и обмен документами, а также содействовать анализу и сбору информации в целях выполнения обязательств в рамках Протокола по СЭО;

b) При создании электронного регистра СЭО следует учитывать национальную специфику и особенности процедуры СЭО в конкретных странах, уровни принятия решений, компетенцию уполномоченных органов, а также брать во внимание ключевые функции и рекомендации, предусмотренные в разделе 2 данного отчёта;

c) Электронный регистр СЕО может быть как самостоятельно функционирующей системой, так и быть объединённым с другими регистрами (например, по ОВОС) или интегрирован в существующие электронные системы электронного правительства или другие базы данных. Так, например, в Грузии разрабатывается публичный портал экологической оценки, который объединит документы по ОВОС/СЭО на единой платформе и обеспечит общественности систематический и упрощенный доступ к информации, связанной с процедурами ОВОС и СЭО;

d) Для внедрения электронных регистров СЭО кроме привлечения технических решений и финансовых ресурсов требуется также изменение или дополнение соответствующего законодательства о СЭО и/или разработка и принятие соответствующих подзаконных актов;

e) При подготовке изменений к законодательству и/или разработке новых актов рекомендуется, чтобы нормы законодательства обеспечивали правовую основу для своевременного внесения в регистр информации и документов, соблюдения сроков, предусмотренных процедурой СЭО, хранения и защиты информации. Так, например, в Украине проведение процедуры СЭО прямо связано с внесением информации и документов в Единый электронный регистр СЭО, а сроки, предусмотренные процедурой СЭО, отсчитываются от даты размещения/обнародования информации в Едином электронном регистре СЭО;

│ 45

f) Электронный регистр СЭО должен по меньшей мере обеспечивать доступность информации по мере поступления на протяжении всего процесса СЭО, а также доступ к информации по мониторингу существенных экологических, в том числе связанных со здоровьем населения, последствий осуществления планов и программ;

g) В законодательстве также целесообразно определять сроки хранения и доступности в электронном регистре СЭО информации по завершённым процедурам СЭО, а также информации по мониторингу последствий осуществления планов и программ, после завершения сроков их действия и/или завершения мероприятий по мониторингу.

В отношении некоторых стран следует дополнительно отметить следующее. В Грузии и Республике Молдова в 2023 году законодательство было дополнено отдельными положениями о регистрах (Подробнее, см. части по Грузии и Республике Молдова), однако данные положения требуют детализации в соответствующих законах и/или подзаконных актах, в частности, что касается порядка создания и функционирования таких регистров, взаимодействия субъектов в рамках процедуры СЭО. В Украине были приняты изменения к законодательству, соответствующий подзаконный акт регулирующий порядок функционирования Единого электронного регистра СЭО, а также создан и начал функционировать централизованный электронный регистр СЭО (Подробнее, см. часть по Украине). В дальнейшем, по результатам практического применения данного регистра может потребоваться доработка как подзаконных актов, так и функционала регистра, в том числе, особого внимания требует вопрос исключительно авторизированного доступа общественности к регистру, что может не в полной мере соответствовать подходам «свободного доступа» общественности к регистрам, предусмотренным Орхусской конвенцией. 4.2. Технические рекомендации

Создание программного обеспечения для автоматизации процесса стратегической экологической оценки требует комплексного подхода, который включает в себя разработку не только самого программного обеспечения, но и соответствующих реестров данных.

Автоматизация процесса стратегической экологической оценки может значительно упростить и ускорить обмен информацией, а также повысить прозрачность процедур во всех странах-участниках. При этом важно, чтобы автоматизированная система была максимально интегрирована и согласована с реестрами данных и требованиями Протокола, а также обеспечивала возможность эффективного обмена данными между странами и Секретариатом Конвенции.

Ниже приведено несколько общих рекомендаций для планирования автоматизации процесса стратегической экологической оценки и создания единых реестров данных.

В рамках такой автоматизированной системы центральное место должна занимать цифровая платформа, доступная как для властей, так и для общественности. Эта платформа может включать в себя модули для подачи инициаторами заявок, проведения общественных консультаций и обсуждений, а также для мониторинга результатов СЭО. Благодаря централизации данных, участники процесса смогут в режиме реального времени отслеживать статус заявок и документов, а также получать уведомления о необходимости их доработки или о новых требованиях законодательства.

Для обеспечения согласованности с Протоколом важно внедрить единые стандарты обмена данными, которые будут соответствовать международным требованиям и правилам. Это позволит обеспечить сопоставимость и совместимость информации на международном

46 │

уровне и упростить трансграничный обмен данными. Кроме того, при автоматизации процесса стратегической экологической оценки следует учитывать уже имеющиеся национальные системы и реестры данных. Например, реестры СЭО следует интегрировать с системой автоматизированного документооборота учреждения при ее наличии, реестрами национального законодательства и экологическими реестрами доступными в стране.

Рекомендуется предусмотреть механизмы обратной связи и поддержки пользователей, что позволит оперативно реагировать на возникающие вопросы и проблемы и будет способствовать непрерывному улучшению программного обеспечения и реестров данных и увеличению их эффективности.

Важным элементом автоматизации является также модуль для проведения экспертных оценок и выдачи заключений. Эксперты должны иметь возможность взаимодействия с системой для загрузки своих отчетов и заключений, что повысит оперативность процесса утверждения документов, иметь доступ к другим национальным экологическим реестрам, в том числе, к реестрам связанным со здоровьем населения и данными мониторинга окружающей среды. Реестры могут быть интегрированы с геоинформационными системами для визуализации зон влияния планируемых проектов и программ, что позволит лучше оценивать их потенциальное воздействие на окружающую среду.

В процессе создания реестров стратегической экологической оценки и их автоматизации ключевым аспектом является защита данных. Защита данных обеспечивает не только безопасность и конфиденциальность информации, но и поддерживает доверие всех заинтересованных сторон к процессу СЭО. Учитывая это, рекомендации по безопасности должны включать ряд мер, направленных на защиту информации на всех этапах её жизненного цикла.

В первую очередь, конфиденциальность данных можно гарантировать с помощью шифрования как хранящихся, так и передаваемых данных. Это означает, что информация реестров СЭО должна быть надежно зашифрована, чтобы предотвратить её утечку или несанкционированный доступ. Важно также обеспечить разграничение доступа, предоставляя разные уровни доступа различным группам пользователей и устанавливая строгие процедуры аутентификации и авторизации, основанные на использовании паролей, биометрических данных или электронных ключей.

Чтобы поддерживать целостность данных, необходимо внедрить системы, которые могут отслеживать и верифицировать любые изменения в данных, включая регулярные аудиты и проверки. Это обеспечит точность и актуальность информации в реестрах СЭО, а любые изменения будут документированы и могут быть отслежены до источника.

Доступность данных крайне важна для оперативности процесса СЭО. Это означает, что данные должны быть защищены от атак и сбоев, которые могут нарушить доступ к ним. Резервное копирование и стратегии восстановления после сбоев должны быть интегрированы в инфраструктуру реестров, чтобы в случае любых аварийных ситуаций данные могли быть быстро восстановлены.

Технологические решения, такие как брандмауэры, антивирусное программное обеспечение, а также виртуализация и облачные технологии, могут служить для укрепления общей безопасности системы. Эти инструменты помогут оградить инфраструктуру реестров от внешних угроз и обеспечить их стабильную работу.

Соответствие международным стандартам защиты данных, таким как Общий регламент по защите данных Европейского Союза (GDPR), и национальному законодательству является обязательным. Это требует регулярного пересмотра политик и процедур, чтобы убедиться,

│ 47

что реестры СЭО соответствуют текущим нормативным требованиям и лучшим практикам защиты данных.

Следует предусмотреть регулярное обучение пользователей программного обеспечения для эффективного взаимодействия с реестрами данных и использования всех их возможностей. Обучение персонала может включать в себя тренинги, семинары и информационные сессии, направленные на повышение компетенций пользователей в области безопасности, например обучение правильному обращению с конфиденциальной информацией, правилам создания и использования паролей, правилам поиска информации, использованию геоинформационных систем и т.д.

В заключение, для поддержания доверия к процессу СЭО и его результатам, важно не только создать надежную систему защиты данных, но и обеспечить прозрачность и доступность информации для всех заинтересованных сторон: международных организаций, государственных органов, научных и экспертных организаций, общественности, чье участие и вовлеченность являются ключевыми в процессе экологической оценки.

Приложение. Картирование процессов: блок-схемы процессов/функций регистров СЭО

│ 49

Протокол по СЭО

50 │

Армения

│ 51

Азербайджан

52 │

Грузия

│ 53

Республика Молдова

  • 1. Цифровая трансформация в отдельных странах Восточной Европы и Кавказа
    • 1.1. Введение
    • 1.2. Армения
    • 1.3. Азербайджан
    • 1.4. Грузия
    • 1.5. Республика Молдова
    • 1.6. Украина
    • 1.7. Заключение
  • 2. Ключевые элементы для создания модельного электронного регистра СЭО
  • 3. Национальные процедуры СЭО и предпосылки для создания электронных регистров
    • 3.1. Армения
      • Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО
      • Предпосылки для создания электронного регистра СЭО
    • 3.2. Азербайджан
      • Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО
      • Предпосылки для создания электронного регистра СЭО
    • 3.3. Грузия
      • Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО
      • Предпосылки для создания электронного регистра СЭО
    • 3.4. Республика Молдова
      • Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО
      • Предпосылки для создания электронного регистра СЭО
    • 3.5. Украина
      • Законодательная основа СЭО и взаимодействие субъектов в рамках процедуры СЭО
      • Единый регистр СЭО
    • 3.6. Заключение
  • 4. Рекомендации
    • 4.1. Юридические рекомендации
    • 4.2. Технические рекомендации
  • Приложение. Картирование процессов: блок-схемы процессов/функций регистров СЭО

Using administrative data sources to produce official statistics on refugees, IDPs and(or) stateless persons (Azerbaijan)

Languages and translations
English

Using administrative data sources to

produce official statistics on refugees, IDPs

and(or) stateless persons

State Statistical Committee of the Republic of Azerbaijan

Chief advisor of Population statistics department Etibar Khalilov

State Statistical Committee of the Republic of Azerbaijan

Legal basis for migration statistics

Law on passports

Law on registration by place of residence and location

Law on the citizenship of the Republic of Azerbaijan

Law on the status of refugees and internally displaced persons

Migration Code of the Republic of Azerbaijan

State Statistical Committee of the Republic of Azerbaijan

Sources and data holders

➢ Local executive authorities of the Republic of Azerbaijan

➢ State Committee for Affairs of Refugees and IDPs of the

Republic of Azerbaijan

➢ State Migration Service of the Republic of Azerbaijan

➢ Ministry of Internal Affairs of the Republic of Azerbaijan

➢ State Statistical Committee of the Republic of Azerbaijan

State Statistical Committee of the Republic of Azerbaijan

In accordance with the law of the Republic of Azerbaijan “On status of refugees and internally displaced (persons displaced within the country)

persons”

“Refugee” a person who is not a citizen of

the Republic of Azerbaijan, who is outside the

country of his citizenship due to a well-founded

fear of becoming a victim of persecution on the

basis of race, nationality, religious beliefs,

membership of a particular social group or

political opinion, and who cannot benefit from

the protection of this country out of the same

fear, or is a person who is unwilling to do so.

Internally Displaced Person (IDP)

(person displaced within the country) is a

person forced to leave his permanent place of

residence and move to another place as a result

of military aggression, natural and man-made

disaster on the territory of the Republic of

Azerbaijan.

State Statistical Committee of the Republic of Azerbaijan

In order to get timely, qualitative and detailed statistical

information on refugees and IDPs it is necessary to define data

sources. Sources for obtaining of this statistical information could

be divided into three parts:

administrative data

population census

statistical observations

Organization of refugees and internally displaced persons statistics in official statistics

State Statistical Committee of the Republic of Azerbaijan

population census

Organization of refugees and internally displaced persons statistics in official statistics

In the 2009 and 2019 census questionnaires, to determine the

status of respondents, appropriate section included the question

“Are you a refugee or internally displaced person?”. The name of

the country that the refugee left and the name of the administrative

unit that the IDPs left are obtained from this section.

State Statistical Committee of the Republic of Azerbaijan

Statistical information on refugees and IDPs is obtained by the State Statistical

Committee of the Republic of Azerbaijan since 1990. At present this statistical

information is mainly obtained based on administrative data, i.e. based on

quarterly official statistical reporting forms namely:

Data on refugees and IDPs based on administrative sources

“Number of refugees and IDPs” (1-refugee and IDP )

“Number and composition of refugees and IDPs” (2-refugee and IDP)

“Dislocation and migration of refugees and IDPs” (4-refugee and IDP)

administrative data

State Statistical Committee of the Republic of Azerbaijan

Data on refugees and IDPs based on administrative sources

State Statistical Committee aggregates quarterly and annual data on actual

residence of refugees and IDPs based on reports from the regional

authorities and presents it to the State Committee for Affairs of Refugees

and IDPs of the Republic of Azerbaijan.

This information are obtained from local executive authorities and then

submitted to other relevant entities and published in aggregated form.

Data reflects the number of refugee and IDP families and family members with

indication of the region and city they left, number by temporary place (region,

city) of residence where IDPs temporarily reside.

State Statistical Committee of the Republic of Azerbaijan

Data on IDPs obtained from statistical reports covers main indicators such as distribution

of IDPs by number of families and family members, types and conditions of buildings

where they settled, sex-age and national structure, education, occupational composition,

sources of income and other.

Moreover, all information on this group of population is prepared based on population

living standards and household budget surveys.

Data on refugees and IDPs based on administrative sources

State Statistical Committee of the Republic of Azerbaijan

Data on refugees and IDPs based on administrative sources

Statistical information on IDPs is also obtained from report form

on “Registration of Households” fulfilled by representative offices of

administrative territorial units of Republic of Azerbaijan. Administrative

territorial units are managed by local executive authorities.

Administrative territorial units check information in the

electronic system throughout the year to generate effective statistics on

household members. Members of IDP households answer question on

new place of residence. SSC have access to this e-system and help

administrative territorial units in methodological issues.

State Statistical Committee of the Republic of Azerbaijan

In accordance with Migration Code of the Republic of Azerbaijan

A stateless person is a person who is not considered a citizen of any state in accordance with

its legislation.

Stateless persons who have received a permit for permanent residence in the Republic of

Azerbaijan or whose period of permanent residence has been extended in the Republic of Azerbaijan

are considered permanent residents of the Republic of Azerbaijan.

Stateless persons arriving in the Republic of Azerbaijan according to another legal basis

(tourism; arriving in connection with private business; sending on a business mission; and other

analogous cases of a temporary nature) are considered temporarily residing in the Republic of

Azerbaijan.

Legislation on the legal status of stateless persons

The legal status of stateless persons is defined by the Constitution of the Republic of Azerbaijan,

Migration Code of the Republic of Azerbaijan and other legislative acts of the Republic of Azerbaijan

adopted in accordance with these laws, and international treaties to which the Republic of Azerbaijan

is party.

State Statistical Committee of the Republic of Azerbaijan

Organization of work with stateless persons in official statistics

Statistical information on Stateless persons obtained from population census.

In the questionnaires of the population census of 2009 and 2019, in order to determine the

respondent’s citizenship, to the section “Citizenship” were entered the column “Citizen of the

Republic ofAzerbaijan”, “Citizen of other country”, “No citizenship” and “Unknown”.

Census question on citizenship

State Statistical Committee of the Republic of Azerbaijan

Organization of work with stateless persons in official statistics

Data on stateless persons are also obtained on the basis of the official

statistics reporting “On foreigners and stateless persons in the Republic of

Azerbaijan” (1-migration quarterly (annual)) provided by the State

Migration Service of the Republic of Azerbaijan to the State Statistical

Committee.

This statistical report shows data on the number of stateless persons by

sex, who received permission for permanent and temporary residence in

the Republic of Azerbaijan.

State Statistical Committee of the Republic of Azerbaijan

“The Great Return” - state program of Republic of Azerbaijan on IDPs

Now the government is engaged in the return of IDPs to their homes and

creating the necessary living conditions for them. At the same time administrative

territorial units register IDPs in e-system of “Registration of Households” who

returns homes and begins new life.

By 2026, 34,5 thsd families or 140 thsd of IDPs will be resettled with the

implementation of the I State Program on the “Great Return”. Every year, billions

of funds are allocated from the state budget for the implementation of the State

Program. 4 billion manats (National Currency of Azerbaijan) have been allocated

from the 2024 state budget for the State Program.

State Statistical Committee of the Republic of Azerbaijan

Outcomes and

challenges

International

methodologies

should be widely

used for

organization of

statistics on

refugees and

internally

displaced persons

Official statistical

bodies should

constantly ensure

the maintenance of

statistics in this

area and ensure

that the current

situation is brought

to the attention of

the world

community at

international

events.

World-wide unions

have to increase

their efforts to

encourage

returning of

refugees and IDPs

to their homes with

the cooperation of

the UN, influential

organizations and

countries.

1 2 3

State Statistical Committee of the Republic of Azerbaijan

THANK YOU FOR

YOUR

ATTENTİON

  • Slide 1
  • Slide 2
  • Slide 3
  • Slide 4: In accordance with the law of the Republic of Azerbaijan “On status of refugees and internally displaced (persons displaced within the country) persons”
  • Slide 5: Organization of refugees and internally displaced persons statistics in official statistics
  • Slide 6: Organization of refugees and internally displaced persons statistics in official statistics
  • Slide 7
  • Slide 8
  • Slide 9
  • Slide 10: In accordance with Migration Code of the Republic of Azerbaijan
  • Slide 11: In accordance with Migration Code of the Republic of Azerbaijan
  • Slide 12
  • Slide 13
  • Slide 14
  • Slide 15
  • Slide 16
Russian

Использование источников

административных данных для

подготовки официальной статистики по

беженцам, ВПЛ и (или) лицам без

гражданства

Государственный комитет по статистике

Азербайджанской Республики

Главный советник отдела статистики населения Этибар

Халилов

Государственный комитет по статистике

Азербайджанской Республики

Правовая основа миграционн ой статистики

Закон о паспортах

Закон о регистрации по месту жительства и месту пребывания

Закон о гражданстве Азербайджанской Республики

Закон о статусе беженцев и внутренне перемещенных лиц

Миграционный кодекс Азербайджанской Республики

Государственный комитет по статистике

Азербайджанской Республики

Источники и держатели данных

➢ Местные органы исполнительной власти Азербайджанской Республики

➢ Государственный комитет по делам беженцев и вынужденных переселенцев

Азербайджанской Республики

➢ Государственная миграционная служба Азербайджанской Республики

➢ Министерство внутренних дел Азербайджанской Республики

➢ Государственный комитет по статистике Азербайджанской Республики

Государственный комитет по статистике

Азербайджанской РеспубликиВ соответствии с законом Азербайджанской Республики "О статусе беженцев и внутренне перемещенных лиц (лиц, перемещенных внутри

страны)"

"Беженец" - лицо, не являющееся

гражданином Азербайджанской Республики,

находящееся за пределами страны своего

гражданства в связи с обоснованным страхом

стать жертвой преследования по признаку расы,

национальности, религиозных убеждений,

принадлежности к определенной социальной

группе или политических убеждений, и не

могущее воспользоваться защитой этой страны в

связи с тем же страхом, либо являющееся лицом,

не желающим этого делать.

Внутренне перемещенное лицо

(ВПЛ) (лицо, перемещенное в пределах

страны) - это лицо, вынужденное покинуть

постоянное место жительства и переехать в

другое место в результате военной

агрессии, стихийного бедствия или

техногенной катастрофы на территории

Азербайджанской Республики.

Государственный комитет по статистике

Азербайджанской Республики

Для получения своевременной, качественной и подробной

статистической информации о беженцах и ВПЛ необходимо

определить источники данных. Источники получения этой

статистической информации можно разделить на три части:

администрат ивные данные

перепись населения

статистически е наблюдения

Организация статистики беженцев и внутренне перемещенных лиц в официальной статистике

Государственный комитет по статистике

Азербайджанской Республики

перепись населения

Организация статистики беженцев и внутренне перемещенных лиц в официальной статистике

В анкетах переписи населения 2009 и 2019 годов для

определения статуса респондентов в соответствующий раздел

включен вопрос "Вы беженец или внутренне перемещенное

лицо?". Из этого раздела можно узнать название страны,

которую покинул беженец, и название административной

единицы, которую покинули ВПЛ.

Государственный комитет по статистике

Азербайджанской Республики

Статистическая информация о беженцах и ВПЛ собирается Государственным

комитетом по статистике Азербайджанской Республики с 1990 года. В настоящее

время эта статистическая информация получена в основном на основе

административных данных, т.е. на основе ежеквартальных официальных форм

статистической отчетности, а именно:

Данные о беженцах и ВПЛ на основе административных источников

"Количество беженцев и ВПЛ" (1-беженцы и ВПЛ )

"Количество и состав беженцев и ВПЛ" (2-беженцы и ВПЛ)

"Перемещение и миграция беженцев и ВПЛ" (4-беженцы и ВПЛ)

администрат ивные данные

Государственный комитет по статистике

Азербайджанской Республики Данные о беженцах и ВПЛ на основе административных источников

Государственный комитет по статистике обобщает квартальные и годовые

данные о фактическом проживании беженцев и вынужденных переселенцев

на основе отчетов региональных властей и представляет их в

Государственный комитет по делам беженцев и вынужденных переселенцев

Азербайджанской Республики.

Эта информация поступает от местных органов исполнительной власти,

затем передается в другие соответствующие организации и публикуется в

агрегированном виде.

Данные отражают количество семей и членов семей беженцев и ВПЛ с указанием

региона и города, который они покинули, количество по временному месту (регион,

город) проживания, где ВПЛ временно проживают.

Государственный комитет по статистике

Азербайджанской Республики

Данные о ВПЛ, полученные из статистических отчетов, охватывают такие

основные показатели, как распределение ВПЛ по количеству семей и членов семей,

типы и условия зданий, в которых они поселились, половозрастная и национальная

структура, образование, профессиональный состав, источники дохода и другие.

Кроме того, вся информация по этой группе населения готовится на основе данных

обследований уровня жизни населения и бюджетов домашних хозяйств.

Данные о беженцах и ВПЛ на основе административных источников

Государственный комитет по статистике

Азербайджанской Республики

Данные о беженцах и ВПЛ на основе административных источников

Статистическая информация о ВПЛ также получена из формы отчета

"Регистрация домохозяйств", заполненной представительствами административно-

территориальных единиц Азербайджанской Республики. Административно-

территориальные единицы находятся в ведении местных органов исполнительной

власти.

Административно-территориальные единицы проверяют информацию в

электронной системе в течение всего года для составления эффективной статистики о

членах домохозяйств. Члены домохозяйств ВПЛ отвечают на вопрос о новом месте

жительства. ЦСБ имеет доступ к этой электронной системе и помогает

административно-территориальным единицам в методологических вопросах.

Государственный комитет по статистике

Азербайджанской Республики В соответствии с Миграционным кодексом Азербайджанской Республики

Лицо без гражданства - это человек, который не считается гражданином какого-либо

государства в соответствии с его законодательством.

Лица без гражданства, получившие разрешение на постоянное проживание в Азербайджанской

Республике или срок постоянного проживания которых в Азербайджанской Республике продлен,

считаются постоянно проживающими в Азербайджанской Республике.

Лица без гражданства, прибывшие в Азербайджанскую Республику на ином законном основании

(туризм; прибытие в связи с частной предпринимательской деятельностью; направление в служебную

командировку; другие аналогичные случаи временного характера), считаются временно проживающими

в Азербайджанской Республике.

Законодательство о правовом статусе лиц без гражданства

Правовой статус лиц без гражданства определяется Конституцией Азербайджанской Республики,

Миграционным кодексом Азербайджанской Республики и другими законодательными актами

Азербайджанской Республики, принятыми в соответствии с этими законами, а также международными

договорами, участником которых является Азербайджанская Республика.

Государственный комитет по статистике

Азербайджанской Республики Организация работы с лицами без гражданства в официальной статистике

Статистическая информация о лицах без гражданства получена в результате переписи

населения.

В анкетах переписи населения 2009 и 2019 годов для определения гражданства респондента

в раздел "Гражданство" были внесены графы "Гражданин Азербайджанской Республики",

"Гражданин другой страны", "Без гражданства" и "Неизвестно".

Вопрос переписи населения о гражданстве

Государственный комитет по статистике

Азербайджанской Республики Организация работы с лицами без гражданства в официальной статистике

Данные о лицах без гражданства также получены на основе

официальной статистической отчетности "Об иностранцах и лицах

без гражданства в Азербайджанской Республике" (1-миграция

ежеквартально (ежегодно)), предоставляемой Государственной

миграционной службой Азербайджанской Республики в

Государственный статистический комитет.

В данном статистическом отчете представлены данные о

количестве лиц без гражданства по полу, получивших разрешение на

постоянное и временное проживание в Азербайджанской

Республике.

Государственный комитет по статистике

Азербайджанской Республики "Великое возвращение" - государственная программа Азербайджанской Республики по вынужденным переселенцам

Сейчас правительство занимается возвращением ВПЛ в свои дома и

созданием для них необходимых условий жизни. В то же время административно-

территориальные единицы регистрируют в электронной системе "Регистрация

домохозяйств" ВПЛ, которые возвращаются домой и начинают новую жизнь.

К 2026 году в рамках реализации I Государственной программы "Великое

возвращение" будет переселено 34,5 тыс. семей или 140 тыс. вынужденных

переселенцев. Ежегодно на реализацию Государственной программы из

государственного бюджета выделяются миллиардные средства. Из государственного

бюджета на 2024 год на реализацию Государственной программы выделено 4

миллиарда манатов (национальная валюта Азербайджана).

Государственный комитет по статистике

Азербайджанской Республики

Итоги и проблемы

Для

организации

статистики по

беженцам и

внутренне

перемещенным

лицам следует

широко

использовать

международные

методологии

Официальные

статистические

органы должны

постоянно

обеспечивать

ведение статистики

в этой области и

следить за тем,

чтобы текущая

ситуация

доводилась до

сведения мирового

сообщества на

международных

мероприятиях.

Профсоюзы всего

мира должны

активизировать

свои усилия по

содействию

возвращению

беженцев и ВПЛ

в свои дома при

сотрудничестве с

ООН,

влиятельными

организациями и

странами.

1 2 3

Государственный комитет по статистике

Азербайджанской Республики

СПАСИБО ВЫ НА

ВАШ

ATTENTİON

  • Slide 1
  • Slide 2
  • Slide 3
  • Slide 4: В соответствии с законом Азербайджанской Республики "О статусе беженцев и внутренне перемещенных лиц (лиц, перемещенных внутри страны)"
  • Slide 5: Организация статистики беженцев и внутренне перемещенных лиц в официальной статистике
  • Slide 6: Организация статистики беженцев и внутренне перемещенных лиц в официальной статистике
  • Slide 7
  • Slide 8
  • Slide 9
  • Slide 10
  • Slide 11: В соответствии с Миграционным кодексом Азербайджанской Республики
  • Slide 12
  • Slide 13
  • Slide 14
  • Slide 15
  • Slide 16

EVAL_ENV_FinalPR_DamSafety_31Dec2024

Evaluation of the Project
Capacity Building for Cooperation on Dam Safety in Central Asia (July 2021)
Final Progress Report as of 31 December 2023

Languages and translations
English

1

Evaluation of the Project Capacity Building for Cooperation on Dam Safety in Central Asia (July 2021)

Final Progress Report as of 31 December 2023

# Recommendation Management response Responsibility Date of implementation/comments

1 UNECE to continue supporting national and regional capacities on hydraulic and hydro- electric structures and cooperation on dam safety, in close cooperation with relevant global and regional organizations, and donors.

UNECE partially accepts the recommendation. Depending on available funding UNECE will continue to support cooperation on dam safety if a proper regional institution willing to address the dam safety challenges is allocated. However, supporting capacities on hydro-electric structures has never been the focus of UNECE initiative. UNECE will explore cooperation opportunities with the International Fund for Saving the Aral Sea (IFAS) which is mandated to address the dam safety challenges in the region.

Project Manager Closed without implementation in June 2023 During the reporting period efforts were made to identify a regional institution mandated to address the dam safety challenges in Central Asia. On 27 July 2021, a letter was sent to the Chair of the Executive Committee of the International Fund for Saving the Aral Sea (IFAS) with the proposal to lead and coordinate the work on dams’ challenges in the region. A reminder was sent on 3 December 2021. As of June 2023, no response has been received regarding the letter addressed to the IFAS. In addition, the project manager raised this issue during a meeting with the Executive Director of IFAS. Despite the expressed importance of this area for the region, the Executive Director did not welcome the inclusion of this matter in IFAS’ activities. .

2 UNECE Secretariat to present the findings of this review to the Committee on Environmental Policy to continue promoting dam safety awareness across all government agencies and among general public.

UNECE accepts the recommendation. The results of the evaluation and the management response will be presented at the next CEP meeting in November 2021.

Programme Manager

Implemented as of 30 June 2022 The results of the evaluation and the management response were presented at the Committee on Environmental Policy, 3-5 November 2021 https://unece.org/sites/default/files/2021- 12/Item_3_presentation_Dam_Safety_project.pdf

3 During the future similar project designs UNECE should include explicit provisions to address gender equality, vulnerable people and disabled in project activities, outcomes, as well as reporting. One of the directions

UNECE partially accepts the recommendation. 1. Gender assessment is part of the standard template of the concept note and it was addressed at the design stage of the project. As to the vulnerable and disable people this aspect

Project Manager and PMU

Partially implemented as of 31 December 2023 1. The updated ECE Concept Note and Project Document template includes requests to assess stakeholders’ vulnerabilities and how the intervention attempts to bridge them. It also asks an assessment of how the project addresses gender equality, human rights,

2

# Recommendation Management response Responsibility Date of implementation/comments could be to make a study/assessment of how gender roles, as well as situation of vulnerable/disabled people could be influenced by displacement and environmental migration as a consequence of a major dam accident. National crisis management agencies should be actively integrated in these efforts

was not part of the project template and can be added subject to amendment of the concept template.

2. Follow-up projects on dam safety will assess in the inception phase the potential impact of the project on gender equality and the situation of the vulnerable groups, as reflected in the 2030 Agenda for Sustainable Development.

Leaving no one behind and disability inclusion. 2. As of 31 December 2023 there was no follow up project on dam safety. This recommendation is closed as not implemented.

5 In the future projects UNECE should systematically assess, monitor and document the usefulness and expected impact of the capacity development workshops by canvassing feedback from the participants through a well-tailored questionnaires, quiz approach application and internet based tools as e.g. Survey Monkey. These assessments should include a space for qualitative comments and improvement proposals by participants

UNECE accepts the recommendation. This recommendation will be brought to the attention of the Director of Environment Division and will apply in future Dam Safety projects if any.

Project Manager Implemented as of 30 June 2022 The recommendation was brought to the attention of the Director of Environment Division and will apply in future dam safety projects.

6 The purposes of sustainability in development of legal, regulatory and technical frameworks on dam safety in Central Asia, UNECE should maintain frequent engagement with Governments and other partners through, but not limited to, advisory missions

UNECE accepts the recommendation. The challenges and the need to strengthen cooperation in the area of dam safety in the region as well as the results of the UNECE initiative will be presented at the various events (SPECA Working Group on Water, Energy and Environment, relevant regional platforms; Water MoP) conducted in the region. UNECE will also provide advisory, intermediary, mediation services to other donor and development organizations (e.g. IWAC, GIZ, AFD) interested in addressing the Central Asian countries' dam safety challenges. In addition, it will be implemented through various relevant missions including conducted by the Regional Advisor on Environment.

Programme Manager and

Regional Adviser

Implemented as of 31 December 2023 The project results, lessons learnt as well as the dam safety challenges in the region are regularly presented at the regional and national events that are conducted in Central Asia. The final results of the project were presented at the SPECA Working Group on Water, Energy and Environment, 10-11 November 2021. The project manager is regularly contacted and consulted by governments and regional organizations that are engaged or plan to engage in the dam safety activities in the region. For example, inputs were provided to the work plan of the International Water Assessment Center (IWAC). UNECE has continued to raise awareness about the project results in subsequent years. A presentation was delivered at the 25th meeting of the SPECA WG on Water, Energy and Environment. UNECE staff attended

3

# Recommendation Management response Responsibility Date of implementation/comments the 2nd International Training Workshop on the Safety of Hydraulic Structures (https://iwac.kz/index.php/en/news/340-central-asian- experts-will-share-experiences-in-ensuring-the-safety-of- hydraulic-structures-at-a-workshop-in-bratislava ) organized by the International Water Assessment Center in cooperation with Government of Slovakia Cooperation on water resources and dam safety was a topic for discussion of a meeting between the UNECE ES and a staff member with the Slovak organizations dealing with these issues (23 November 2022). UNECE received a letter from the Ministry of Water Resources of Uzbekistan with a request to support addressing dam safety in the country. On 20 April 2023 the need to strengthen dam safety cooperation was discussed at the regional meeting organized by the UN Regional Centre for Preventive Diplomacy in Central Asia (UNRCCA). Initiating new activities on dam safety was also discussed with the OSCE Office in Dushanbe on 5 May 2023.

Presentation (State Statistical Committee, Azerbaijan)

Languages and translations
English

Alternative Child Care

in the Republic of Azerbaijan

Geneva, March 04-06, 2024

The State Statistical Committee of the Republic of Azerbaijan

National Policy Regarding

Child Welfare 2

The State National Policy regarding children is

aimed at these outcomes:

► For every child to grow and be brought up in

an adequate living conditions and all material

needs fulfilled.

► For every child to receive education based on

the progressive demand.

► For each child to be formed into a good

citizen.

Goals of Alternative Care

 Protection of the rights and legitimate interests of children who have lost their parents and children deprived of parental care, children with disabilities;

 Ensuring a decent life for children who have lost their parents and children deprived of parental care, children with disabilities;

 Inclusivity;

3

Regulatory Rights Framework

➢ Law on the Rights of the Child

➢ Family Code

➢ Law on “pocial protection of children Who Have deceased parents and those

deprived of parental care"

➢ State Programme for the Transfer of Children from State Child Care Institutions

to Families (De-institutionalization) and Alternative Care in the Republic of

Azerbaijan

➢ UN Convention on the Rights of the Child

➢ State strategy for the Development of Education in the Republic of Azerbaijan

4

State Strategy on

Education Development

► Preparation and implementation of developmental and inclusive

educational programs for children in need of special care

► Organization of inclusive education with the aim to prevent

children with disabilities from entering special schools

5

The main duties related to children in need of special care are:

State Program for the Transfer of Children

from State Child Care Institutions to

Families of Alternative Care

Main duties:

► Placement of children in public institutions to local families

► Provision of mechanisms for the organization of alternative care

► Making sure that children are covered by alternative care within

their living environment and do not end up in public child care

institutions

► After the transfer of children with disabilities to families,

measures are taken to integrate them into general education

schools and organize joint education with healthy children

6

► Improvement of the legislative framework

► Transformation (re-profiling) of public child care institutions

► Development of the alternative care system

► Gradual decentralization of the care system

► Encouraging public to adopt, develop foster families, strengthen

guardianship, and support for extended families

► Providing employee trainings for institutions and guardianship

bodies

7 State Programme for Deinstitutionalization

and Alternative Care developed as a guide for

the implementation of the National Plan for

the Transformation of Public Child Care

Institutions.

Main Duties:

Data sources

Child care facilities – information collection in

the form of statistical reporting

Households - The State Statistical Committee

conducts sample surveys of households on

various topics

Census

Administrative data of the relevant government

agencies

8

Data collection

The State Statistical

Committee

Local Statistical Authorities

Local Executive Authorities

Enterprises and

organizations

Households

Ministry of Health

Ministry of Labour and

Social Protection of the

Population

Ministry of Education

Ministry of Justice

9

Electronic Registration of Children

Without Parental Care

➢ Since 2018, the Ministry of Labor and Social Protection

has been maintaining a centralized electronic record of

children left without parental care, as well as persons wishing

to adopt them.

➢ Since 2022, the Statistics Committee has been receiving

data from the Ministry of Labor and Social Protection from

the Register of Children Without Parental Care.

10

State Educational Institutions

2005 2010 2015 2020 2022

Number of orphanages (0-3 years

old) 4 4 4 4 4

The number of children 156 143 179 82 77

Orphans and children left without

parental care

59 43 - 2 23

Number of orphanages (over 3 years) 6 6 8 9 12

The number of children 734 513 681 636 769

Number of special boarding schools

for children with disabilities

12 11 11 9 9

The number of children 2915 2537 2468 2288 2308

The number of orphans and children

left without parental care

280 162 82 28 110

11

2005 2010 2015 2020 2022

Number of social service institutions for

persons under 18 years of age who have

been diagnosed with a disability 2 2 2 2 2

Number of children 298 267 320 326 292

Orphans and children left without

parental care 61 44 39 39 31

Number of boarding schools for lost

children, orphans, and children left

without parental care 2 2 3 3 3

Number of children 330 398 401 449 419

12 State Educational Institutions

Adoption 13

Total in

2022

including by age

up to 1

year

1-5

years

6-10

years

11-17

years

Number of children adopted 358 68 188 72 30

Of which:

Girls 170 29 98 31 12

Boys 188 39 90 41 18

Of which, adopted by foreign

citizens and stateless persons

5 1 - - 4

of which:

Girls 1 1 - - -

Boys 4 - - 2 4

14Adoption

Data Dissemination

Every year, the State Statistics

Committee publishes a statistical

compilation titled "Children in

Azerbaijan" in Azerbaijani and

English and makes it available to

State bodies and other users,

reflecting a number of indicators

relating to children.

15

Compiled Statistics "Children in

Azerbaijan"

16

1. Demographics

2. Education and Culture

3. Therapeutic and preventive services.

4. Social protection

5. Disease and mortality

6. Crime

7. Economic Regions and

Administrative Regions Strategic Unit

Areas

Official website (www.stat.gov.az)

E-version of the compiled statistics

“Children in Azerbaijan” is posted on the website of the

Committee www.stat.gov.az.

17

18

THANK YOU FOR YOUR ATTENTION!

Russian

Альтернативный уход за детьми

в Азербайджанской Республике

Женева, 04-06 марта 2024 г.

Государственный Комитет по Статистике Азербайджанской Республики

Национальная политика по

отношению к детям

 Государственная политика по отношению к детям направлена ​​на то, чтобы:

каждый ребенок рос и воспитывался в необходимых материальных и бытовых условиях;

каждый ребенок получал образование, основанное на прогрессивных требованиях;

каждый ребенок формировался как достойный гражданин.

2

Цели альтернативного ухода

 Охрана прав и защита законных интересов детей потерявших родителей и детей, лишенных родительской опеки, детей с ограниченными возможностями здоровья;

 Обеспечения достойной жизни детям потерявших родителей и детям, лишенных родительской опеки, детей с ограниченными возможностями здоровья;

 Инклюзивность;

3

Нормативно-правовая база

➢ Закон «О правах ребенка»

➢ Семейный Кодекс

➢ Закон «О социальной защите детей, потерявших родителей и лишенных родительской опеки»

➢ Государственная Программа по передаче детей из государственных детских учреждений в семьи (де- институционализация) и альтернативной заботе в Азербайджанской Республике

➢ Конвенция ООН «О правах детей»

➢ Государственная Стратегия развития образования в Азербайджанской Республике

4

Государственная Стратегия

развития образования

Основные задачи связанные с детьми нуждающимися в особой заботе:

➢ подготовка и внедрение развивающих и инклюзивных учебных программ для детей, нуждающихся в особой заботе

➢ организация инклюзивного образования с целью предотвращения попадания детей с ограниченными возможностями здоровья в школы-интернаты

5

Государственная Программа по передаче

детей из государственных детских

учреждений в семьи альтернативной заботе

Основные задачи:

➢ передача детей, находящихся в государственных детских учреждениях, в семьи

➢ обеспечение механизмов организации альтернативной заботы

➢ создание такой среды, чтобы дети были охвачены альтернативной заботой на проживаемых территориях и не попадали в государственные детские учреждения

➢ после передачи детей с ограниченными возможностями здоровья в семьи, проведение мероприятий по интеграции их в общеобразовательные школы и организации совместного обучения со здоровыми детьми

6

В качестве руководства для реализации Государственной программы деинституционализации и альтернативного попечения разработан Национальный план преобразования государственных детских учреждений.

Задачи:

➢ улучшение законодательной базы

➢ преобразование (перепрофилирование) государственных детских учреждений

➢ развитие системы альтернативного попечения

➢ постепенная децентрализация системы попечения

➢ поощрение усыновления, развитие приемных семей, усиление опеки и поддержки расширенным семьям

➢ обучение сотрудников учреждений и органов опеки и попечительства

7

Источники данных

Детские учреждения – информация

собирается в виде статистической отчетности

Домохозяйства - Госкомстат проводит

выборочные обследования домохозяйств по

разным темам

Перепись населения

Административные данные соответствующих

государственных органов

8

Сбор данных

Госкомстат

Местные статорганы

Местные исполнительные власти

Предприятия и

организации

Домохозяйства

Министерство

Здравоохранения

Министерство Труда и

Социальной Защиты

Населения

Министерство

Образования

Министерство Юстиции

9

Электронный учет детей оставшихся

без попечения родителей

➢ Соответствующим Указом Президента С 2018 года в Министерстве Труда и Социальной Защиты ведется централизованный электронный учет детей оставшихся без попечения родителей, а также лиц, предъявляющие желание их усыновлению.

➢ С 2022 года Комитет по Статистике получает от Министерства Труда и Социальной Защиты данные из регистра о детях оставшихся без попечения родителей..

10

Государственные учебно-воспитательные

учреждения

2005 2010 2015 2020 2022

Число домов ребенка 4 4 4 4 4

в них детей, человек 156 143 179 82 77

из них дети-сироты и дети,

оставшиеся без родительской

опеки 59 43 - 2 23

Число детских домах 6 6 8 9 12

в них детей, человек 734 513 681 636 769

Число специальных школ-

интернатов для детей с ограничен-

ными возможностями здоровья

12 11 11 9 9

в них детей, человек 2915 2537 2468 2288 2308

из них дети-сироты и дети,

оставшиеся без родительской

опеки 280 162 82 28 110

11

2005 2010 2015 2020 2022

Число предприятий социальной

сферы обслуживания для лиц до 18

лет, у которых установлена

инвалидность 2 2 2 2 2

в них детей, человек 298 267 320 326 292

из них дети-сироты и дети,

оставшиеся

без попечения родителей 61 44 39 39 31

Число школ-интернатов для детей

потерявших сирот и детей,

оставшихся без родительской опеки 2 2 3 3 3

в них детей, человек 330 398 401 449 419

12 Государственные учебно-воспитательные

учреждения

Усыновление 13

Всего в

2022 году

в том числе по

возрасту

до 1

года

1- 5

лет

6-10

лет

11-17

лет

Усыновлено детей, человек 358 68 188 72 30

из них:

девочки 170 29 98 31 12

мальчики 188 39 90 41 18

Из общего числа, усыновлено

иностранными гражданами

и лицами без гражданства 5 1 - - 4

из них:

девочки 1 1 - - -

мальчики 4 - - 2 4

14Усыновление

Распространение данных Ежегодно Государственным

Комитетом по Статистике

Азербайджанской Республики

издается на азербайджанском и

английском языках и

представляется государственным

органам и прочим пользователям

статистический сборник “Дети в

Азербайджане”, отражающий в

себе ряд показателей,

относящихся к детям.

15

Cтатистический сборник “Дети

в Азербайджане”

16

1.Демографические показатели

2. Образование и культура

3. Лечебно-профилактические

услуги.

4. Социальная защита

5. Заболевания и смертность

6. Преступность

7. Экономические районы и адми-

нистративные территориальные

единицы

Официальный сайт (www.stat.gov.az)

Электронный вариант статистического сборника

Дети в Азербайджане” размещается на интернет-странице

Комитета www.stat.gov.az.

17

18

БЛАГОДАРЮ ЗА ВНИМАНИЕ!

  • Slide 1: Альтернативный уход за детьми в Азербайджанской Республике
  • Slide 2: Национальная политика по отношению к детям
  • Slide 3: Цели альтернативного ухода
  • Slide 4: Нормативно-правовая база
  • Slide 5: Государственная Стратегия развития образования
  • Slide 6: Государственная Программа по передаче детей из государственных детских учреждений в семьи альтернативной заботе
  • Slide 7
  • Slide 8: Источники данных
  • Slide 9: Сбор данных
  • Slide 10: Электронный учет детей оставшихся без попечения родителей
  • Slide 11: Государственные учебно-воспитательные учреждения
  • Slide 12: Государственные учебно-воспитательные учреждения
  • Slide 13: Усыновление
  • Slide 14:
  • Slide 15: Распространение данных
  • Slide 16: Cтатистический сборник “Дети в Азербайджане”
  • Slide 17
  • Slide 18

Supporting Innovative High-Growth Enterprises in the SPECA sub-region, UNECE Policy Handbook

The UNECE region, including the participating countries of the United Nations Special Programme for the Economies of Central Asia (SPECA), have recently faced a number of challenges, such as the COVID-19 pandemic and resulting economic downturn, the impact of military conflict in Ukraine, and the triple planetary crisis of climate change, biodiversity loss and pollution. This has negatively impacted progress towards the Sustainable Development Goals (SDGs) and made the search for effective policy solutions a priority.

Supporting Innovative High-Growth Enterprises in the SPECA sub-region, UNECE Policy Handbook

The UNECE region, including the participating countries of the United Nations Special Programme for the Economies of Central Asia (SPECA), have recently faced a number of challenges, such as the COVID-19 pandemic and resulting economic downturn, the impact of military conflict in Ukraine, and the triple planetary crisis of climate change, biodiversity loss and pollution. This has negatively impacted progress towards the Sustainable Development Goals (SDGs) and made the search for effective policy solutions a priority.