External Final Evaluation of “Labour Migration Programme - Central Asia” at IOM

London, England, United Kingdom -

Full Time

Start Date

Immediate

Expiry Date

20 Aug, 26

Salary

0.0

Posted On

22 May, 26

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Research Design, Qualitative Research, Quantitative Research, Mixed-Method Research, Theory Of Change, OECD DAC Evaluation Criteria, Project Evaluation, Migration Governance, Labour Migration, Institutional Capacity Development, Data Collection, Stakeholder Engagement, Gender Analysis, Human Rights-Based Approach, Monitoring And Evaluation, Contribution Analysis

Industry

International Affairs

Description

Introduction The International Organization for Migration (IOM), the United Nations (UN) Migration Agency, was established in 1951 and is the leading intergovernmental organization in the field of migration, working closely with governmental, intergovernmental, and non-governmental partners. With 175 member states, 8 states holding observer status, and offices in 172 countries, IOM is dedicated to promoting humane and orderly migration for the benefit of all. IOM established its office in Kazakhstan in 1997. Since then, the Organization has been promoting humane and orderly migration for the benefits of all, providing services and advice to the government as well as to migrants. As the Country Office with Coordinating Functions for Central Asia, the IOM mission in Kazakhstan helps to address specific sub-regional migration issues and emerging trends in Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan, establishing priorities for project development in the country offices in the context of sub-regional strategies, policies and consultative processes. Since June 2022, IOM has been implementing the regional project “Labour Migration Programme - Central Asia” (Phase 1), funded by the Swiss Agency for Development and Cooperation (SDC), which aims to contribute to the improvement of labour migration management in order to use the human mobility impact on development in countries of origin, as well as in countries of destination. Project participating countries: Kazakhstan, Kyrgyz Republic, Tajikistan and Uzbekistan. Through six key outcomes, the project aims to support the efforts of governments and other stakeholders in target countries to develop coordination mechanisms to identify problems and regularly adapt and improve labour migration policies and processes at the national and regional levels. More specifically, the project aims to achieve this objective with six key outcomes: I Governments and other stakeholders develop coordination mechanisms to identify challenges and regularly adapt and improve labour migration policy and processes in the region. II. SRAs, PRAs and vocational training centres improve services to better anticipate and respond to labour market needs as well as needs and priorities of prospective migrant workers. III. Employers in targeted sectors in destination countries adjust their recruitment practices to engage in managed ethical labour migration programmes in Central Asia. IV. Prospective migrant workers are better informed and prepared to protect themselves in recruitment and migration. V. Migrants and their families achieve greater financial inclusion and uptake of financial services to sustainably improve their livelihoods. VI. Governments integrate data and research on return migration into their economic planning and targeted social protection initiatives Specific Objectives of the Final Evaluation The specific objectives of the evaluation are to: * Assess the extent to which the project has achieved its stated objectives (outcome and impact levels) and outputs, while identifying both negative and positive factors that have facilitated or hampered progress in achieving the project outcomes, including external factors/environment, weakness in design, management, and resource allocation. * Measure project’s degree of implementation, efficiency and quality delivered on intended and unintended results (outputs) and specific objectives (outcomes), against what was originally planned. * Provide summative evaluative evidence on the contribution of the project towards supporting efforts of governments and other stakeholders in target countries to develop coordination mechanisms to identify problems and regularly adapt and improve labour migration policies and processes at the national and regional levels * Assess the management and financial efficiency of the project. * Assess the extent to which the project's outcomes will be sustainable (without the need for external support) and contribute towards the objective of the project. * Assess project’s contribution to gender equality and women’s empowerment and the broader “Leave No One Behind” agenda. * Generate substantive evidence-based knowledge by identifying best practices and lessons learned that could be useful for remaining project implementation, other interventions at national (scale up) and international level (replicability). * Provide a forward-looking perspective for IOM’s positioning in relation to labour migration for subsequent phases. * Provide formative recommendations toward design and implementation arrangements of MEAL for subsequent phases. Through a participatory approach, the evaluator will actively engage relevant stakeholders, including IOM project staff, focal points of the national counterparts and beneficiaries from different activities. Recommendations, emerging from the evaluation, should be strongly linked to the findings of the evaluation and should provide clear guidance to IOM and its stakeholders on how they can address them. Furthermore, the evaluation will have a focus on what worked, what did not work, and why, based on feedback against evidence and provide actionable recommendations, highlight best practices, share lessons learned, and offer valuable insights, which will inform the remainder of the implementation of the project or potential subsequent phases. An essential aspect of the evaluation involves analysing the integration of IOM cross-cutting themes of gender equality, non-discrimination and human rights-based approach into project activities and implementation. Additionally, the evaluator shall contribute to the following activities, taking a lead role in their implementation: * Review of Monitoring and Evaluation (M&E) approach, with recommendations aimed at contributing to overall programme strengthening. * Review of Monitoring, Evaluation, Accountability and Learning (MEAL) support activities assigned to the staff and/or consultants. * Review of application of IOM’s Project Handbook for programme design, implementation, and MEAL under relevant evaluation questions; and ensuring the uploading of monitoring data, reports, and evaluation documentation in line with IOM Evaluation Policy. * Review of programme risk register, including recommendations for improvements of the Project Risk Register Matrix and related risk management strategies and plans, where appropriate. Evaluation Criteria The OECD-DAC evaluation criteria, relevance, coherence, effectiveness, efficiency, likely impact, and sustainability will guide the final evaluation. All assessments will be conducted in accordance with the OECD-DAC definitions to ensure methodological rigour and consistency with international standards. • Relevance: The extent to which the intervention's objectives and design respond to the needs, policies, and priorities of beneficiaries at the global, national, and institutional levels, and continue to do so as circumstances evolve. • Coherence: The extent to which the intervention is compatible with other interventions within the same country, sector, or institution. • Effectiveness: The extent to which the intervention has achieved, or is expected to achieve, its stated objectives and results, including any differential results across groups. • Efficiency: The extent to which the intervention delivers, or is likely to deliver, results in a cost-effective and timely manner. • (Likely) Impact: The extent to which the intervention has produced or is expected to produce significant positive or negative, intended or unintended, higher-level effects. • Sustainability: The extent to which the benefits of the intervention continue or are likely to continue. Evaluation Questions In the light of the evaluation parameters, the Evaluator is expected to analyze data and share their findings, conclusions and recommendations generated by this analysis. As a reference point for the evaluation, the evaluator is provided with indicative evaluation questions below; which are expected to be amended, elaborated and submitted as part of the Inception Report and shall be included as an annex to the final evaluation report described below. In consultation with the Evaluation Manager, in the inception phase, the evaluation team will further develop and refine the evaluation questions to ensure detailed and specific information is gathered for each criteria. The evaluation matrix will be reviewed collaboratively with the Evaluation Manager to the evaluation criteria and questions should be agreed upon and reflected in the inception report. The following key questions will guide the evaluation process: Evaluation CriteriaEvaluation Questions Relevance Key Evaluation Question-1: To what extent is the project’s design and theory of change appropriate to the priorities and needs of Government and other relevant stakeholders, and aligned with national policies and international frameworks in the area of labour migration? Sub-questions 1. To what extent is the project’s theory of change and design relevant to strengthening the government’s institutional capacity, supporting the needs of migrant workers and addressing labour market needs? 2. To what extent is the project aligned with relevant national frameworks and policies? 3. What opportunities are there to better align the project with emerging migration trends, institutional needs, and policy priorities? Coherence Key Evaluation Question-2: To what extent is the project internally coherent in its design and implementation, and externally coherent with other national, and partner-supported migration initiatives and systems? Sub-questions 4. (External coherence) To what extent did the project create synergy/linkages with other interventions in the country, national labour migration initiatives and international migration cooperation frameworks? 5. (Internal coherence) To what extent are the project’s objectives, outcomes, outputs, and activities logically aligned and mutually reinforcing? How coherent is the project’s theory of change in linking project interventions to intended labour migration outcomes? Effectiveness Key Evaluation Question-3: To what extent has the project achieved its intended outputs and contributed to strengthening institutional capacity, rights-based labour migration, international cooperation, and evidence-based policymaking? Sub-questions 6. To what extent has the project achieved its intended outputs and progress toward outcomes across all six result areas until the end of project duration? (the evaluator is expected to provide detailed analysis of 1) planned activities and results and 2) achievement of results.) 7. To what extent have project activities responded to the needs of target groups and final beneficiaries? 8. What factors (including ownership) have enabled or hindered effectiveness? Efficiency Key Evaluation Question-4: To what extent have project resources been used efficiently to deliver timely, high-quality results, and how well have management, coordination, and adaptive mechanisms supported implementation? Sub-questions 9. To what extent were resources (financial, human, time) used efficiently to deliver timely and high-quality results, ensuring value for money? 10. To what extent and in what ways has ownership - or the lack of it - by the partners/stakeholders influenced the efficiency of the project (e.g. delays, uptake, coordination)? 11. To what extent did MEAL systems, including monitoring recommendations raised by IOM and the donor informed efficient decision-making and course correction? How well did the project respond to external shocks or contextual shifts? Likely Impact Key Evaluation Question-5: To what extent is the project likely to contribute to long-term improvements in labour migration governance? Sub-questions 12. What evidence exists of emerging changes in institutional capacity and private sector stakeholders, quality and consistency of migration procedures, human rights compliance and gender sensitivity, data-driven policymaking? 13. To what extent is the project likely to contribute to longer-term improved labour migration governance, stronger international cooperation, enhanced protection of migrant workers, especially vulnerable groups? 14. What broader effects (positive or negative/intended or unintended) are emerging from the project’s interventions at institutional or system level? 15. What factors are likely to influence the project’s longer-term impact? Sustainability Key Evaluation Question-6: To what extent are the results of the project likely to be sustained through institutional ownership, integration into systems and supportive legal and policy frameworks? Sub-questions 16. To what extent are results owned by stakeholders and embedded in institutional systems? 17. What systems, capacities, and resources are in place to sustain results after project completion? To what extent do legal frameworks and governance structures support continuity of project benefits? 18. What are the main risks to sustainability (institutional, financial, political, operational) and the likelihood of continuation of project benefits after donor funding? Crosscutting Issues Key Evaluation Question-7: To what extent has the project effectively integrated human rights, gender equality, and vulnerability considerations into its design, implementation, and results? Sub-questions 19. To what extent has the project integrated human rights, gender equality, disability inclusion, and Leave No One Behind principles in its design, implementation, and monitoring? 20. To what extent are project activities accessible and responsive to the needs of vulnerable groups? What barriers exist to equitable participation and benefit among stakeholders and beneficiaries? Evaluation Approach and Methodology The evaluation should be transparent, inclusive, participatory and utilization-focused. The overall methodology should be implemented following a theory of change approach, framed by the UN/OECD DAC[1] evaluation criteria drawing upon mixed methods (quantitative and qualitative) data to capture direct project results as well as (likely) contributions. In line with good practice in evaluating this type of complex system change-focused intervention, the overall methodology should be based on three concrete pillars: 1. the project’s theory of change; 2. an evaluation matrix grouping key evaluation questions and sub-questions by broad OECD/DAC criterion allowing analysis of programme results at different levels of its results chain 3. a data collection toolkit for the evaluation describing the quantitative and qualitative primary and secondary data collection tools that will be deployed to collect and analyse data to answer the evaluation questions. The evaluation process should be participatory, engaging government officials, implementing and development partners, project staff, key stakeholders and a wide cross-section of staff and beneficiaries while ensuring inclusion of elements of gender equality. The main analytical framework for the evaluation is provided by the Project’s theory of change, which should be used to organize the evaluation questions according to the Project’s expected results at each level of its results chain. In doing so, the evaluation should use a broad Contribution Analysis (CA) approach to causal inference with a view to understanding the influence of relevant contextual factors, and alternative and additional drivers or obstacles to change at the regional and national levels that may have influenced the Project’s direct and indirect, intended and unintended results. The evaluation should also seek to apply additional evaluation techniques that can further strengthen the plausibility of links between the results of the different strands of work on various intended Project outcomes at the policy, community and individual beneficiary levels as well as telling the story of how and why both intended and unintended change has or has not happened as a result of the intervention. The methodological prism may involve contribution analysis (effectiveness), process tracing (case studies), outcome harvesting/most significant change (unplanned/emerging results), landscape analysis (relevance and coherence), quantitative analysis (indicator achievement and funding analysis), document review (relevance and efficiency), light foresight (forward-looking aspects) and techniques linked to participatory evaluation[2]. In line with UN evaluation practice, the scope of the evaluation should cover six core UN/OECD DAC evaluation criteria: relevance, coherence, effectiveness, efficiency, likely impact, sustainability, and crosscutting issues. In proposing how to conduct the evaluation, the evaluation team should use an evaluation matrix to operationalize the theory of change and its agreed framework of direct and indirect results into a set of measurable categories of evaluative analysis following the results chain of the intervention. The evaluation matrix should properly address gender equality (GE) and human rights (HR) dimensions, including age, disability and vulnerability. The evaluation questions above present a set of preliminary questions that the evaluation team should address in their proposed approach, following the revised UN/OECD DAC criteria. A final, more detailed evaluation matrix will be developed during the inception phase on the basis of document review and initial consultation with key Project stakeholders. On the basis of the questions included above and the information present elsewhere in this Terms of Reference, the evaluation team should deploy a set of data collection methods and tools (that includes gender disaggregation) and allow for rigorous triangulation. These methods and tools will allow leveraging existing secondary data as well as collecting new primary data to be gathered during the field visit, which together will be able to answer the initial questions listed above. The combination of primary and secondary tools or the number of separate ‘lines of evidence’ should be at least five and be designed – as with the rest of the evaluation - with triangulation and complementary assessment of the sub-questions in the matrix in mind. The evaluator is requested to propose a set of mixed methods data collection/analysis methodologies and techniques to answer the evaluation questions.[3] This will be refined in the inception phase. The following lines of evidence should be considered: * Document, literature and monitoring systems review: Evaluation Manager will provide access to all relevant documentation, data collected, and analysis. Further documents may be requested by the evaluation team. The Project team will share information and provide guided walk-throughs of the Project and project management methods, platforms and tools. This should include a review of; * Project document and description of the action * Result Framework/MEAL Framework and Plan * Work Plan * Donor/Progress Reports * Monitoring Reports * Project Steering Committee meeting minutes * Studies relating to the country context and situation * Financial documentation and reports. * Background documents and other documentation. * Analysis of deliverables and financial reports: Comprehensive access to deliverables, financial reports, and reporting dashboards will be provided alongside documentation. * Structured, semi-structured and/or in-depth interviews or Key Informant interviews (KIIs): The team will provide a stakeholder list, including a wide range of stakeholders from the donor, national government and local administrations, as well as project partners, IOM representatives and the project team, and selected provincial stakeholders. All interviews should be undertaken with full confidentiality and anonymity. (The final evaluation report should not assign specific comments of individuals) * Focus groups discussions with capacity building participants. * Quantitative surveys: Surveys should have a clearly defined scope and seek to answer specific questions about the Project outcomes/objectives. * Secondary data analysis. * Direct observations. * Case studies/deep dives for different types of activities. The Evaluator will ensure triangulation of the various data source. Data and evidence will be triangulated with multiple sources to address evaluation questions. The final methodological approach including interview schedule and data to be used in the evaluation should be clearly outlined in the inception report and fully discussed and agreed between IOM, stakeholders and the Evaluation Team. Data collection tools should be gender sensitive, ensure that the data collection is disaggregated by sex and take into account the broader cross-cutting issues as presented below and elsewhere in the ToR. Cross-cutting As noted above, the promotion and protection of Human Rights (HR) & Gender Equality (GE) and Disability Issues (DI) and LNOB are central principles to the mandate of the UN, and all UN agencies must work to fundamentally enhance and contribute to their realization by addressing underlying causes of human rights violations, including discrimination against women and girls, and utilizing processes that are in line with and support these principles. Those UN interventions that do not consider these principles risk reinforcing patterns of discrimination and exclusion or leaving them unchanged. It is therefore important that evaluations commissioned by IOM take these aspects into account. The methodology and techniques to be used in the evaluation should be described in detail in the Inception Report and the Final Evaluation Report, and should contain, at minimum, information on the instruments used for data collection and analysis, whether these be documents, interviews, questionnaires or participatory techniques following high level of research ethics and impartiality. Final decisions about the specific design and methods for the evaluation will be made through consultation among IOM, the Evaluation Team and key stakeholders about what is appropriate and feasible to meet the evaluation purpose and objectives as well as answering the evaluation questions, given limitations of budget, time and data. Ethics, Norms, and Standards for Evaluation The evaluation team must follow the IOM Data Protection Principles, [https://iomint.sharepoint.com/sites/dmsportal/SitePages/ControlNo.aspx?controlNo=IN/00138] UN Evaluation Group (UNEG) norms [https://www.iom.int/sites/g/files/tmzbdl486/files/about-iom/evaluation/UNEG-Norms-Standards-for-Evaluation-2016.pdf] and standards, and relevant UNEG ethical conduct guidelines [https://evaluation.iom.int/sites/g/files/tmzbdl151/files/documents/2020 Ethical Guidelines for Evaluation.pdf] while carrying out the final evaluation. The evaluation of the project is to be carried out according to ethical principles and standards established by the UNEG. * Anonymity and confidentiality. The evaluation must respect the rights of individuals who provide information, ensuring their anonymity and confidentiality. * Responsibility. The report must mention any dispute or difference of opinion that may have arisen between the Evaluator and Project Team in connection with the findings and/or recommendations. The Evaluator must corroborate all assertions and disagreements. * Integrity. The Evaluator will be responsible for highlighting issues not specifically mentioned in the ToR if this is needed to obtain a more complete analysis of the intervention. * Independence. The Evaluator should ensure its independence from the intervention under review and must not be associated with its management or any element thereof. * Incidents. If problems arise during the interviews, or at any other stage of the evaluation, they must be reported immediately to IOM. If this is not done, the existence of such problems may in no case be used to justify the failure to obtain the results stipulated by IOM in this Terms of Reference. * Validation of information. The Evaluator will be responsible for ensuring the accuracy of the information collected while preparing the reports and will be ultimately responsible for the information presented in the evaluation report. * Intellectual property. In handling information sources, the Evaluator shall respect the intellectual property rights of the institutions and communities that are under review. * Delivery of reports/deliverables. If delivery of the reports/deliverables is delayed, or in the event that the quality of the reports delivered is lower than of the quality desired by IOM, the Evaluator will not be entitled for any payment regarding that specific report/deliverable, even person/days for submission of the report/deliverable has been invested. Human Rights, Gender Equality, Vulnerable Groups and Disability Issues The methodology used in the final evaluation, including data collection and analysis methods should be human rights and gender-sensitive to the greatest extent possible, with evaluation data and findings disaggregated by sex, ethnicity, age, etc. Detailed analysis on disaggregated data will be undertaken as part of the final evaluation from which findings are consolidated to make recommendations and identify lessons learned for enhanced gender responsive and rights-based approach of the project. These evaluation approach and methodology should consider different types of groups in the project intervention – women, youth, minorities, and vulnerable groups. The promotion and protection of Human Rights (HR) and Gender Equality (GE) vulnerable groups and Persons with Disabilities are central principles to the mandate of the UN, and all UN agencies must work to fundamentally enhance and contribute to their realization by addressing underlying causes of human rights violations, including discrimination against women and girls, and utilizing processes that are in line with and support of these principles. Those UN interventions that do not consider these principles risk reinforcing patterns of discrimination and exclusion or leaving them unchanged. It is, therefore, important and required that evaluations commissioned by IOM take these aspects into account. * Guidance on Integrating Disability Inclusion (DI) in Evaluations[1] * Integrating Human Rights and Gender Equality in Evaluations[2] Concretely, evaluation team members are requested to incorporate the following key principles from the UNEG guidance for integrating human rights, gender equality, and disability inclusion into their work: Inclusion. Evaluating HR, GE, and DI requires paying attention to which groups benefit and which groups contribute to the intervention under review. Groups need to be disaggregated by relevant criteria: disadvantaged and advantaged groups depending on their gender or status (women/men, age, location, etc.), duty-bearers of various types, and rights-holders of various types, in order to assess whether benefits and contributions were fairly distributed by the intervention being evaluated. In terms of HR & GE, it is important to note that women and men, boys and girls who belong to advantaged groups are not exempt from being denied their human rights or equal rights. The same applies to persons with disabilities. Therefore, the concept of inclusion must assess criteria beyond advantage. Likewise, it is not unusual that some groups may be negatively affected by an intervention. An evaluation must acknowledge who these stakeholders are and how they are affected, and shed light on how to minimize the negative effects. Participation. Evaluating HR, GE and DI must be participatory. Stakeholders of the intervention have a right to be consulted and participate in decisions about what will be evaluated and how the evaluation will be done. In addition, the evaluation will assess whether the stakeholders have been able to participate in the design, implementation and monitoring of the intervention. It is important to measure stakeholder group participation in the process as well as how they benefit from results. Fair Power Relations. Approaches to human rights, gender equality and disability inclusion issues seek, inter alia, to balance power relations between or within advantaged and disadvantaged groups. The nature of the relationship between implementers and stakeholders in an intervention can support or undermine this change. When evaluators assess the degree to which power relations changed as a result of an intervention, they must have a full understanding of the context, and conduct the evaluation in a way that supports the empowerment of disadvantaged groups. In addition, evaluators should be aware of their own position of power, which can influence the responses to queries through their interactions with stakeholders. There is a need to be sensitive to these dynamics.

Responsibilities

Conduct a comprehensive final evaluation of the Labour Migration Programme in Central Asia using OECD-DAC criteria to assess effectiveness, efficiency, and sustainability. Provide evidence-based recommendations and lessons learned to inform subsequent project phases and IOM's strategic positioning.