Montenegro

This assignment consists of an evaluation of the European Union’s Budget Support in Montenegro, focusing on three strategic areas: Public Administration Reform (PAR), Integrated Border Management (IBM), and the response to COVID-19. It covers four sector budget support programmes implemented between 2016 and 2026, including the PAR programme, two IBM programmes, and COVID-19 crisis response support. The evaluation will assess the relevance, effectiveness, efficiency, coherence, EU added value and sustainability of these interventions, with particular attention to their contribution to Montenegro’s EU accession agenda. It will examine how EU financial support, policy dialogue and complementary technical assistance have supported reforms, strengthened national systems, and contributed to governance, security, economic recovery and social resilience.

The work will combine document review, interviews, field visits, quantitative analysis and stakeholder consultation, structured around reconstructed intervention logics and evaluation questions. The evaluation will provide evidence-based findings, conclusions and recommendations for future EU support in Montenegro.

Methodologically, the assignment will follow the OECD-DAC approach to budget support evaluation, structured around the levels of the intervention logic: inputs, direct outputs, induced outputs, outcomes and contextual impact indicators. The evaluation will not aim to prove direct causality for higher-level results, but will assess the plausibility and consistency of EU contributions in relation to observed changes. Evidence will be gathered through a combination of documentary review, administrative and statistical data, interviews with EU, government, donor and civil society stakeholders, focus groups and field visits.

The evaluation covered all European Union (EU) external action support to IPA Roma inclusion projects in the Western Balkans and Türkiye between 2016 and 2023. The main objectives of this assignment were to: i) provide an independent and evidence-based assessment of the EU's past and current external action support in the area of Roma inclusion; and ii) provide lessons learnt and recommendations at EU level to inform ongoing and future decision-making in improving current and future EU strategies, programmes and actions on Roma inclusion.

The evaluation was centred around three main thematic areas:

• Civil rights (registration, data collection, monitoring, reporting, fight against discrimination and antigypsyism);
• Economic and social rights (health, employment, education and housing, including access to public services);
• Roma inclusion in governments' green and digital agendas.

Additionally, the evaluation addressed broader cross-cutting issues such as human rights, the need to ensure appropriate treatment for vulnerable groups including children, fight against environmental poverty and climate change, social inclusion and gender equality.

The analysis covered both spending and non-spending actions and examined all funding mechanisms and modalities of implementation used under IPA II and III.

The evaluation covered all EU external action support to connectivity in the Enlargement and Neighbourhood regions between 2014 and 2022. It served a dual purpose of accountability and learning. The main objectives are to: i) provide an independent and evidence-based assessment of the EU’s past and current external action support in the area of connectivity in the transport and energy sectors, and ii) based on this assessment, provide lessons learnt and recommendations for decision-makers at EU level to inform ongoing and future decision making.

The specific objectives of the evaluation were:

• To assess whether the EU support on connectivity has contributed to achieve the objectives articulated in the EU’s strategic policy framework and whether those results justify and strengthen the approach followed for the current programming period;
• To assess the extent that the EU support has contributed to enhance connectivity in the energy and transport sectors at both policy and implementation level;
• To assess the EU’s toolbox especially grants, technical assistance, blending of loans, risk capital, equity and guarantees and whether it has contributed to catalysing investments on connectivity;
• To identify how the most recent engagement in connectivity supports the operationalisation of the Global Gateway, and derives lessons learnt to further strengthen Global Gateway implementation in the context of the Economic Investment Plans (EIPs);
• To assess the interplay between regional, multi country and bilateral interventions and the synergies created to stimulate regional integration;
• To assess the Working Better Together approach and the private-public dialogues and partnerships and their impact on the effectiveness, predictability and sustainability of long-term interventions;
• To assess the extent to which the EU connectivity support in energy and transport sectors have contributed to supporting the twin green and digital transitions.

The analysis covered both spending and non-spending actions and examines all funding mechanisms and modalities of implementation (including grants for investments and technical assistance, financial instruments such as guarantees, budget support) used under the Instrument for Pre-accession Assistance (IPA) II and III, the European Neighbourhood Instrument (ENI), and the Neighbourhood, Development and International Cooperation Instrument – Global Europe (NDICI-GE). This also included operations under the Western Balkans Investment Framework (WBIF), the Neighbourhood Investment Facility (NIF) and the European Fund for Sustainable Development (Plus) (EFSD / EFSD+).

The objective of this facility is to improve the overall quality of the NDICI Rapid Response and Peace, Stability and Conflict Prevention thematic areas by providing high quality research services.

We will provide the FPI with flexible and ad hoc research services and prepare literature reviews and rapid assistance reports in the thematic areas specified below.

The main objectives of this evaluation were to provide the relevant services of the European Union and the interested stakeholders with:

• an overall independent assessment of the performance of the TACSO programme (in its three phases), paying particular attention to its different levels of results measured against its expected objectives; and the reasons underpinning such results
• key lessons learned, conclusions and related recommendations in order to improve current and future interventions.

In particular, this evaluation served:

• To inform the implementation of the programme fourth phase, which will be implemented through an extension of the current contract (TACSO 3);
• To inform the decision-making on the appropriateness of a potential fifth phase of the programme, to be launched with the next programming exercise (CSF 2024-2026);
• To inform the potential programme exit strategy to ensure the sustainability of its results.

The main users of this evaluation are the relevant EU services (DG NEAR HQ and EU Delegations of the targeted region) and the implementing partners.

The overall objective of the project of which this contract is part is to support evaluation of the achieved results for activities implemented under specific projects implemented under indirect management and to identify the weaknesses in the implementation of activities as pre-condition for the long-term success of all EU funded activities.

The purposes of this contract are as follows:

• to assess the ongoing implementation of proposed projects and determine the extent to which the objectives of the projects are being achieved;
• to provide recommendations for more efficient implementation of forthcoming contracts under indirect management.

Results to be achieved by this contract are grouped into two result areas with pertaining results:

Result area 1: Interim evaluation of selected activities conducted, and evaluation reports prepared;

Result area: Evaluation reports presented to the key stakeholders and published.

The main aim of this project was to enhance the promotion of Montenegrin companies and their products at an international level, with specific focus on SMEs. The project has establish the best possible conditions for increasing the export oriented activities of Montenegrin SMEs with respect to its geographical position and sector.

Background

In March 2017, the UfM Climate Change Expert Group (UfM CCEG) adopted a 5-years work-program, which stress the importance to strengthen climate finance in the Euro-Mediterranean region to address climate change issues. In April 2019, UfM Member States agreed to pursue the assessment of climate finance in the UfM region, with the view of providing sound basis for a renewed climate mandate of the UfM by 2020, with the new UfM Ministerial Declaration on Climate Change.

Target Groups

Target gruops of tehis technical assistance are: UfM Institutions, UfM Member States, International Financial Institutions, International Climate Change Institutions, Civil society representatives, Private sector Organisations and Institution interested in UfM regional integration.

Objectives

• The primary objective of this technical assistance is to enhance the knowledge of UfM Member States, International Financial Institutions, civil society representatives, private sector organisations and any other institution interested in the UfM regional integration, regarding the level of financial commitments made in 2018 and in 2019 to address climate change issues in the UfM region.
• The second objective is to explore the feasibility of financial mechanisms, which could be implemented in the UfM region with the aim of enhancing the mobilisation of climate finance for specific population or sectors, while building on lessons learned in some specific sectors (in particular in the water sector) or potentially in other regions, such as Caribbean or Latin America.

Expected Results

• R1 - to have a precise figure of financial commitments made to recipients in UfM developing countries in 2018 and in 2019, by public/private donors from developed/developing countries;
• R2 - to have a precise view of commitments and disbursements made by national governments, local authorities, private sector and, if possible, households addressing climate change in UfM developing countries;
• R3 - to have the methodology and process in place for monitoring, reporting and verifying climate finance commitments made in the UfM region, especially by ensuring the involvement of all relevant actors, including the scientific community, the private sector and NGOs;
• R4 - quantitative and qualitative analysis of financial mechanisms, which could be implemented in the UfM region with the aim of enhancing the mobilisation of climate finance for specific population or sectors.

The purpose of this project was to contribute to the modernization of the vocational education and training system of Montenegro by equipping learners for lifelong learning and employment in line with modern rapid technological development, new labour market jobs and thereby overcomes the lack of skills needed in the labour market.

These objectives were achieved through the development of professional qualifications and training of VET teachers and in-company trainers in vocational and practical education as well as through the improvement of student-centred learning and practical teaching and learning.

The development of qualifications and programmes enabled an adaptation to current labour market and societal needs, in line with the National Qualifications Framework (NQF). The modernization of the curriculum and increase in the scope and quality of work-based learning empowers youth and young adults to take up the labour market challenges and it facilitates the transition from school to work and/or further education and lifelong learning.

The objectives of the evaluation were to:

• support the decision-making, both at EU strategic (planning) level, and at the level of the design of the action programmes /interventions;
• feed into the reflection on the revision of future development policies in view of the recent adoption of the 2030 Sustainable Development Agenda;
• demonstrate whether already on-going/planned IPA (I) II, ENPI/ENI, DCI and EDF action programmes/interventions in the area of social protection have taken on board the latest lessons learnt from the previous programming period;
• provide recommendations on the best way to approach and improve its support to the development of social protection systems, in partner countries both in terms of the use of policy dialogue and financial assistance;
• contribute to the preparation and adjustment of action programmes, namely for IPA II, ENI, DCI and EDF assistance to the extent possible;
• feed in the Mid-term Review of financial instruments;
• feed in Mid-term Reviews of country/region multi-annual indicative programmes.

The evaluation used the five standard OECD/DAC evaluation criteria, namely relevance, effectiveness, efficiency, impact and sustainability. In addition, the evaluation also looked at the EU added value and the 3Cs.

The evaluation focused on EU support (policy dialogue and financial assistance) provided for the development of social protection systems in partner countries, more specifically on the social insurance and social assistance measures to address the following risks and needs: unemployment, parental responsibilities, sickness and healthcare, work accidents/employment injuries disability, loss of a spouse or parent, old age, housing and social exclusion.

The evaluation covered:

• IPA I beneficiary countries;
• ENPI partner countries;
• Countries covered by DCI and EDF.

The evaluation was conducted in four phases

• An Inception Phase, focusing on the design and the methodology of the evaluation including the reconstruction of the intervention logic and the compilation of an inventory;
• A Desk Phase, focusing on gathering and analysing existing data and information (through literature/document reviews and interviews/focus groups/surveys);
• A Field Phase, including preparation of field missions, visits to the case study countries (Bosnia and Herzegovina, El Salvador, Ethiopia, Turkey, Kyrgyzstan, Macedonia*, Malawi, Moldova, Montenegro, Morocco, Palestine*) and drafting of country reports;
• A Synthesis Phase, producing a final report and dissemination of results to concerned stakeholders.

The purpose of the evaluation was to provide an overall independent assessment of the regional organisations and fora in the Western Balkans and the instrumental role they play in the regional integration and accession perspective including recommendations on possible streamlining.

The specific objectives of the evaluation were the following:

• Mapping of existing regional organisations and initiatives;
• Assessment of administrative and financial costs for its member countries of each organisation/initiative;
• Assessment of benefits for the region/countries of each organisation/initiative;
• Proposal for possible streamlining.

The evaluation mapped 56 organisations in the six Western Balkan IPA countries that were still active in June 2016 (cut-off date) and analysed 31 of them in detail. The key sectors included: connectivity (transport, energy and environment), competitiveness (industry and private sector development, trade), skills and mobility (human capital development, higher education), good governance, fighting crime and security cooperation.

The Evaluation of the Instrument for Pre-accession Assistance (IPA II) assessed whether IPA II is fit for purpose to deliver EU resources towards EU's external policy, both at start of the planning period (2014) and at the time of the evaluation (2017), and considered the place of IPA II – its complementarities and synergies – within the wider set of external financing instruments.

The objective of the IPA II evaluation was: 1) to provide the relevant external relations services of the European Union and the wider public with an independent assessment of the IPA II instrument, including complementarities/synergies between IPA II and the other EFIs; and 2) to inform the future programming and implementation of the current EFIs, as well as the next generation of the EFIs.

The evaluation was evidence-based using a non-experimental mixed methods approach combining quantitative and, more importantly, qualitative data and was guided by Evaluation Questions (EQ) covering EU evaluation criteria (relevance; effectiveness, impact and sustainability; efficiency; added value; coherence, consistency, complementarity and synergies; leverage).

The main analytical tools consisted of a rigorous assessment of documentation, analysis of statistics and quantitative data and a consultation of stakeholders (via interviews, group discussions, the online-survey focusing on EU Delegations and the Open Public Consultation).

The report is available at:

https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/final_report_ipa_ii_evaluation.zip