Evaluator - UNITED NATIONS : Job Details

Result of Service

Inception Report (after 2 months) Data collection and draft Evaluation Report (2 months after inception report) Final evaluation report (2 months after draft report)

Work Location

remote, home-based

Expected duration

6 months

Duties and Responsibilities

1. Introduction As a United Nations (UN) Secretariat entity, the United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA) is required to periodically conduct evaluations of its programmes /subprogrammes, projects, thematic areas, regional or country operations, etc. In line with this requirement, UN OCHA is commissioning an independent evaluation to assess its work on accountability to affected people (AAP) in humanitarian action. This document outlines the purpose, objectives, scope, suggested methods, and key questions, as well as the operational modalities for an evaluation team to conduct the exercise under the management of OCHA Evaluation and Oversight Section (EOS). The evaluation is expected to start in the fourth quarter of 2024 and be completed within 6-9 months' timeframe. 2. Context An accountable humanitarian system, where decision-making power is in the hands of those affected by the crisis, is central to effective humanitarian action. Accountability to Affected People is a primary responsibility and a fundamental way for humanitarians to ensure that the expressed priorities of affected people are at the center of humanitarian action. An accountable system protects and preserves the rights and dignity of people affected by crises, ensures that humanitarian action remains relevant and effective, leaves no one behind and upholds humanitarian principles.This vision was captured through the development of the IASC Principals' statement on AAP in 2022 and the IASC Collective AAP Framework. The former states that collective humanitarian action must be instructed by affected people to guide our actions and to measure how well we provide protection and assistance against their diverse needs, feedback and perceptions, throughout the humanitarian response. This requires collective leadership and aid agencies to prioritize participation and equal partnerships and mechanisms for collective information sharing, feedback, and action, combined with integrated planning and monitoring using key metrics. OCHA's priorities to advance AAP are outlined in the OCHA 2023-2026 Strategic Plan. Being accountable to affected people is a theme that runs throughout the plan; it is most evident among the transformational priorities that seek to ensure a coherent humanitarian response that is people-centered, context-specific, contributes to community resilience and promotes concrete protection outcomes and is participatory, inclusive, gender-responsive and principled humanitarian action that leaves no one behind. OCHA supports the collective humanitarian response to be accountable by providing adequate information that can be shared with communities; by supporting meaningful participation and leadership of affected people, generating and ensuring finances based on community priorities and promoting appropriate community feedback systems are in place. OCHA has been recognized as a strong advocate for accountability to affected people, and collective accountability has been made a strategic pillar of several Humanitarian Response Plans. Yet, the common understanding of AAP and the operationalization of its global-level policies, including commitments to AAP, at the country level remains challenging for OCHA, a problem identified as well for the broader humanitarian community. Feedback from communities continues to show that aid received does not necessarily correspond to what is most required or is available when most needed and not enough aid recipients are aware of how to communicate with aid providers. The level of representation or involvement of affected people in decision making about aid remains limited. OCHA's role in delivering on AAP commitments corresponds to OCHA's role more generally within the humanitarian system and its key areas of work of coordination, information management, humanitarian financing, and policy and advocacy. OCHA aims to provide sustained operational support and guidance to strengthen the participation of communities and the use of their expressed preferences in decision-making. The following details some of OCHA's expected activities: Coordination: OCHA's role is to support Humanitarian Coordinators, Humanitarian Country Teams (HCT), Inter-Cluster Coordination Groups (ICCGs) and other relevant coordination bodies to ensure international humanitarian action is people-driven, community-led, inclusive and accountable. This includes specific support to humanitarian leadership, establishing and maintaining dedicated working groups and related services when needed, linking them to humanitarian coordination architecture through ICCGs and HCTs, and incorporating feedback from affected people in needs analysis and response planning. Information Management: OCHA's role is to support the integration of feedback and perceptions from people of all genders, ages, disability and other diversities affected by crises in the analysis, planning and monitoring of humanitarian responses. OCHA supports the integration of people's expressed preferences, priority needs and levels of satisfaction in the response and supports the adjustment of the response based on how they align with or differ from the analysis. OCHA's role is as well to promote and support data responsibility (the safe, ethical and effective management of personal and non-personal data for operational responses); ensuring that data is treated in a principled manner, kept confidential and used solely for humanitarian purposes Policy & Advocacy: OCHA's role is to advocate for an accountable, effective and impact-oriented response that puts people and their preferences, needs and priorities at the centre. This includes ensuring policy and advocacy are grounded in a strong evidence base of people's own expressed needs and preferences. OCHA coordinates agencies and organisations to engage communities in discussions and consultations, undertake joined-up public information campaigns, including around harmful information such as misinformation, disinformation and hate speech (MDH), and establish and maintain collective and inter-agency complaints and feedback mechanisms. Humanitarian Financing: OCHA's role is to support the strengthening and advancement of AAP by integrating relevant activities and principles in all aspects of OCHA-managed pooled funds, from project design, implementation and monitoring to evaluation. In addition, targeted financing is made available to dedicated AAP projects, including those prioritizing information provision, participatory and inclusive approaches and collective feedback/complaints systems. OCHA also advocates for donors to allocate funding for humanitarian responses in accordance with AAP principles, and in a manner that promotes collective accountability. 3. Purpose and scope of the evaluation 3.1. Purpose Accountability to affected people is a central element of humanitarian action, OCHA's Strategic Plan and the six transformational priorities outlined within. This evaluation aims to assess OCHA's efforts to advance accountability to affected people in collective humanitarian actions. It will therefore assess how successfully OCHA's integrates AAP across its core functions, both in country operations and at the regional and global levels. The evaluation will identify areas of strengths and those that need reinforcement (formative evaluation). Additionally, it will provide recommendations to OCHA's leadership on optimal alignment of the organization's capacity with its role and commitments to AAP. 3.2. Scope The evaluation will focus on OCHA's planning, implementation, and the monitoring of its efforts toward an accountable humanitarian system across i) coordination, ii) humanitarian financing, iii) information management and iv) policy and advocacy. The evaluation's temporal scope will be the period since the previous strategic plan's elapsed and encompass the years 2022 to 2024. In addition, the evaluation will provide a short descriptive (not evaluative) overview of OCHA's actions in support of AAP, starting with the Grand Bargain (2016). This evaluation will not assess in-country activities that are subject to other reviews or learning processes: AAP activities implemented by partner organizations and financed by the Central Emergency Response Fund (CERF) and Country-Based Pooled Funds (CBPF) that are subject of an ongoing review. OCHA's Flagship Initiative has a stand-alone monitoring, evaluation, and learning plan, including a developmental evaluation of the Flagship Initiative's work. EOS will establish a close collaboration between the CERF/CBPF review and the Flagship Initiative's learning efforts to harness potential synergies and avoid duplication/burden on staff. Results from the pooled funds' review and Flagship Initiative's learning will be included in this evaluation where relevant. 3.3. Stakeholders Internal (OCHA) and external stakeholders OCHA have an interest in this evaluation. Some will be asked to play a more active role than others, notably OCHA staff closely involved with activities relating to accountability to affected people across global, regional and country-based positions. The primary intended users of the evaluation include OCHA's senior leadership and management as well as heads of offices with country office leadership and staff directly involved in accountability to affected people. External stakeholders include Humanitarian Coordinators, humanitarian organizations (UN, NGO/Civil Society Organizations), and people affected by humanitarian crisis. See Annex 3 for a detailed analysis of stakeholders. 4. Evaluation approach, methods and ethical consideration 4.1. Evaluation criteria and questions The evaluation will use the criteria of relevance, effectiveness, coherence, and efficiency. Relevance refers to the extent to which OCHAs AAP strategies, objectives and activities align with OCHA's mandate, leverage its comparative advantage within the humanitarian system, and address the expressed priorities and preferences of people affected by crisis; effectiveness refers to concrete outcomes and the extent to which OCHA's actions are achieving or expected to achieve their stated or implicit objectives; coherence refers to the extent to which OCHA's AAP actions align with other activities in OCHA and those of other humanitarian actors; efficiency refers to the extent to which organizational resources are appropriately allocated and optimally used to achieve the desired objectives. In probing these criteria, the evaluation will identify good practices, lessons learned and opportunities for improvement. Key areas of inquiry will be identified in the initial stages of the evaluation through consultations with OCHA's senior leadership, thematic leads and sections that are involved in AAP activities, including country and regional office staff. The following evaluation questions are tentative areas of inquiry and will be finalized during the inception phase: Relevance 1. To what extent do OCHA's AAP related strategies, objectives and activities reflect its mandate and align with the evolving priorities within the humanitarian system, and the expectations of crisis-affected people? 2. How well is OCHA's role and comparative advantage on AAP articulated, disseminated and understood, both internally and externally? 2.1. How does OCHA understand the relationship between AAP and the core work? 2.2. Are humanitarian actors and partners of OCHA satisfied with OCHA's role in AAP? 3. How well and through what mechanisms are OCHA's strategic ambitions translated into operational realities/practicalities for OCHA? How well are commitments to AAP built into OCHA's systems and processes/actions or work plans? Effectiveness 4. How and to what extent has OCHA's work on AAP been adapted effectively and in a timely manner in the areas of coordination, humanitarian financing, information management, policy, and advocacy? 5. How effective is OCHA's decision-making process in developing and delivering a corporate approach to AAP? Does it take account of OCHA's strengths and areas of expertise? 6. What factors contribute to and hinder the achievement of OCHA's work on AAP? 7. How and where have good practices in AAP developed? 8. Are current resources for AAP adequately aligned with OCHA's role and priorities in AAP and priority areas of accountability? 9. How well and through what mechanisms are OCHA's strategic ambitions translated into operational realities/practicalities? What concrete results/changes has OCHA's work on AAP delivered, especially at country-level? Coherence 10. How well do OCHA's efforts in AAP relate to and align, complement and collaborate with other priority thematic areas cross-cutting priorities (for example, disability, inclusion, gender, localization, PSEA)? 10.1. What approaches have been employed to maximize resources and results for AAP? Efficiency 11. How efficiently are available resources for AAP used? 12. How well do OCHA's training and support systems equip staff in country and global offices with the necessary skills to develop and implement AAP objectives? 4.2. Evaluation approach The evaluation will be an appreciative, largely qualitative inquiry focused on learning. The evaluation will employ non-experimental mixed methods (qualitative, quantitative, participatory) and a number of data collection tools that are justified and aligned with the evaluation questions. Data will be derived from secondary sources, including a literature review, desk review of relevant documents including existing survey data. Primary data will be collected via key informant interviews, focus group discussions and surveys, for example the OCHA Partner Survey 2025 or an OCHA staff survey. Examples of good practice might be included as case studies. Perspectives from all stakeholders will be solicited with a particular emphasis on gathering insights from OCHA staff. Data collection will occur in-person and remotely/online. All data used will be triangulated for validation. The evaluation will pay attention to impartiality and reduction of bias by relying on different primary and secondary data sources and stakeholders and adopting systematic triangulation to generate evaluation findings including an in-depth review of available evidence to enhance the evaluation team's understanding of the wider context around AAP including existing guidance, good practices and recurring issues; interviews and surveys covering a wide range of stakeholders; and direct observations in different locations. The detailed methods, including standardized data collection instruments (surveys, interview guides, etc.), and a coherent approach to the analysis of primary and secondary data, will be developed during the inception phase of the evaluation. Visit to OCHA country offices: It is expected that the evaluation team will visit at least two country offices. The objective of the visit is to experience and understand the nuances of delivering on achieving OCHA's commitments on AAP in at least two different contexts. The selection of the focus countries will be finalized during the inception phase, taking into consideration a range of criteria like the extent of integration of AAP into OCHA's core functions, type of context, and relation to other initiatives and evaluations/reviews or audits. Other criteria might be developed during the inception phase. Annex 4 provides more details. Participation of people affected by crisis in this evaluation: OCHA contributes to principled and effective humanitarian response through its core functions of coordination, advocacy and communication, humanitarian financing, policy and information management; OCHA does not deliver aid to affected people. Direct engagement with affected people occurs primarily during inter-agency needs assessments and discussions, within collective feedback mechanisms, participatory approaches, and as part of its humanitarian financing activities. For this evaluation, affected people will be invited to participate in the evaluation in countries the evaluation team will visit. The focus of this inter-action will be to gather perceptions of the level of response-wide accountability and suggestions about accountable needs assessments and to brainstorm with affected people about good practices and their preferences going forward. This consultation will be aligned with the results of an analysis of already available secondary data from communities, for example collected by REACH, Ground Truth Solutions, other researchers and via collective feedback mechanism. The engagement with affected people will be advised by the AAP working group in the country and guided by the concept of free, meaningful and active participation, and in line with UNEG Ethical Guidelines for Evaluation. Given the scope of the evaluation and the diversity within groups of people affected by crisis, the evaluation will not be able to achieve a representative sample. However, extra care will be exercised to sample purposively and, for reasons of transparency, an anonymized profile of those who participated will be included in the evaluation report. The evaluation process will include (1) an inception phase after which the evaluation team will submit an inception report, (2) a data collection phase which includes visits to OCHA country and headquarters offices, remote interviews and surveys, and (3) a data analysis and reporting phase at the end of which the team will submit all deliverables including the evaluation report. 4.3. Ethical considerations The evaluation will be guided by the UNEG Norms and Standards and the UNEG ethical guidance for evaluation to ensure the quality of evaluation process. The evaluation team is expected to consider ethical considerations throughout the entire evaluation process. Due diligence will be given to effectively integrating good ethical practices and paying due attention to robust ethical considerations in the conduct of the evaluation, as stipulated in the UNEG Norms & Standards for Evaluation. This includes, but is not limited to ensuring informed consent, protecting privacy, confidentiality and anonymity of participants, ensuring cultural sensitivity, respecting the autonomy of participants, and complying with precautionary measures put in place by OCHA. It is of utmost importance that the 'do no harm' principle consistently guides evaluation efforts across the board, including as it applies to those involved in the OCHA's work as well as affected people. OCHA will ensure that the consultants have not been directly involved in the implementation of AAP activities in OCHA, and that they have no vested interest, nor have any other potential or perceived conflict of interest. All members of the evaluation team will abide by the UNEG Ethical Guidelines for Evaluation 2020. In addition to signing a pledge of ethical conduct in evaluation, each member of the evaluation team will also commit to signing a statement of confidentiality. Each member of the evaluation team will commit to conducting mandatory, self-paced UN training courses, including for example BSAFE, I Know Gender, Information Security Awareness, Prevention of Sexual Exploitation and Abuse by UN Personnel, United to Respect and others. Most courses are available at least in English, French, Spanish and Arabic. 4.4. Quality assurance OCHA's guide on quality assurance provides a standard checklist for the evaluation process. This will be systematically applied throughout the evaluation process. The quality assurance process does not interfere with the views or independence of the evaluation team but ensures that the report provides credible evidence and analysis in a clear and convincing way and draws its conclusions on that basis. The evaluation team will be required to ensure the quality of data (reliability, consistency and accuracy) throughout the data collection, synthesis, analysis and reporting phases. OCHA expects that all deliverables from the evaluation team are subject to a thorough quality assurance review by the team leader prior to submission of the deliverables to OCHA's evaluation and oversight section. 5. Governance and organization of the evaluation 5.1. Phases and deliverables The evaluation team will be responsible for the following deliverables: 1) Inception report 2) Public evaluation report including agreed annexes/deliverables 3) Associate communication materials (PowerPoint presentations, one-page evaluation briefs, evaluation poster, etc.) 5.2 The following timetable will be used (Annex 1 presents a more detailed timeline): Preparation 3 months - Preparation of concept note - Preparation of TOR - Consultation with Senior Leadership - Recruitment of evaluation team - Constitution of internal reference group Inception 2 months - Briefing evaluation team - Document review - Stakeholder briefings and interviews - Inception Report Data collection & Analysis 2 months - Desk Review - Country visits for data collection - Country debriefings - Global, remote data collection - Debriefing, validation and recommendations co-creation workshops with stakeholders Reporting 2 months - Draft evaluation report - Validation and recommendations co-creation workshops (if needed) - Final evaluation report - Other summative products Dissemination and follow-up - Publication of report - Management response - Briefings (OCHA, other stakeholders) - Dissemination of evaluation results 5.3. Roles & Responsibilities This role is part of a team of three, together with a Senior Evaluator/Team Leader and a Junior Evaluator/research assistant. The deputy team leader/evaluator position requires a minimum of 7 years' experience in in evaluation and humanitarian action. Knowledge of OCHA and accountability to affected people is essential. Experience in other relevant technical areas (IM, PSEAH, DEI) are an asset. The primary responsibility of the deputy team leader/evaluator is to a. Supports the team leader b. Develop and implement surveys c. Collect primary data in collaboration and as agreed with the Team leader; including visit to two country offices d. Contribute to the analysis of primary and secondary data e. Record an audit trail of the evaluation, including a complete list of references and all primary data collection. f. Carry out other tasks as required. The role and responsibility of the other roles are advertised as follows: The team leader position requires a minimum of 15 years' experience in evaluation and humanitarian action. Knowledge of OCHA at corporate level and accountability to affected people is essential. Analytical skills are essential. Expertise in needs assessment and information management is an asset. Experience in other relevant technical areas (IM, PSEAH, DEI, localization, gender) are an asset. The primary responsibilities of the team leader/senior evaluator will be a. Setting out the approach for the evaluation, including data collection and analysis framework/methods b. Guiding and managing the evaluation team during all phases of the evaluation c. Overseeing, organizing and contributing to the primary data collection and analysis, including visit to two country offices d. Overseeing and contributing to document review of secondary data (documents/data) e. Lead briefings, validation and recommendations co-creation sessions/workshops, develop recommendations for the evaluation report f. Producing the inception report, draft and final report by bringing together the input from team members and overseeing responses to feedback from stakeholders. g. Exchange and review progress with OCHA's evaluation manager on a regular basis The junior evaluator/research assistant position requires a minimum of 4 years' experience in data collection and analysis, primarily with qualitative data. Development and management of surveys as well as data visualization skills are an asset. The primary responsibility of the junior evaluator/research assistant is to a. Gather and review relevant secondary data, summarize in line with the evaluation framework b. Support the evaluation team in data collection where needed, enter and clean data for analysis c. Support the evaluator with the development and implementation of the surveys d. Provide administrative and project management support to the evaluation team. OCHA plays the following role in the evaluation: The OCHA internal reference group is composed of individuals of various branches that are familiar with OCHAs work on AAP and OCHA's functioning. The reference group will meet regularly, review key deliverables and serve as a sounding board for the evaluation team. The evaluation manager, under the guidance of the Chief of Evaluation and Oversight, will oversee the day-to-day management of the evaluation, function as the key focal point of the evaluation within OCHA and for the evaluation team and chair the reference group. The evaluation manager is responsible for drafting the TOR, selecting and contracting the evaluation team members, preparing and managing the budget, constituting and chairing the reference group, organizing briefings, workshops and country visits, constituting a document library and initial stakeholder lists, participating in country visits and debriefings/validation workshops, conducting first-level quality assurance of the evaluation products, soliciting OCHA's feedback on evaluation products, maintaining an accurate record and audit trail of the evaluation. We would encourage the evaluation team to consider evaluation manager participation in analysis workshops, though this decision rests with the evaluation team leader. The evaluation manager will develop the evaluation's communication products (evaluation brief). The Chief of Evaluation and Oversight will approve all final evaluation products. 5.4. Security considerations The evaluation team is expected to travel to two country offices, including those with hazardous contexts. Based on the contract status of an Individual Consultant, OCHA will ensure, in collaboration with the team members, the organization of travel and a daily subsistence allowance (DSA). Evaluation team members will be received by the OCHA country office, participate in security briefings to be familiar with the security situation during the country visit. The evaluation team will follow OCHA's security rules and instructions during a country visit. Security considerations will vary depending upon the nature of the context in the countries that will be selected for a visit. The consultant is responsible to ensure their insurance coverage during the contract period. 5.5. Communication All evaluation products will be produced in English. As part of the international standards for evaluations, OCHA requires that all evaluations are made publicly available. The evaluation will be accompanied by a communication plan identifying opportunities for engagement during the evaluation and outlining when and how frequent key stakeholders will be informed about the progress of the evaluation. The plan will also guide the process for successfully sharing the results of the evaluation with relevant stakeholders. 5.6. Budget The evaluation is financed by OCHA's Evaluation and Oversight Section.

Qualifications/special skills

Advanced university degree (Master's degree or equivalent) in social science, evaluation, humanitarian affairs, public administration, international development, economics, political science, or related areas is required. A first-level university degree, combined with an additional two years of relevant professional experience, may be considered in place of a Master's degree. A minimum of 7 years of combined professional experience in both humanitarian action and the evaluation of humanitarian action is required. Experience in management, implementation, or evaluation of accountability to affected people in humanitarian action is desirable. Expertise in needs assessment and information management, or other technical areas (PSEAH, DEI, localization) is desirable.

Languages

Fluency in English (both oral and written) is required. Knowledge of French or Spanish is desirable.

Additional Information

Travel: The consultant is expected to visit two OCHA country offices (6 days each). The selection of country offices will be decided during the inception phase. Communication: The consultant shall obtain OCHA's written approval prior to issuing any press releases or making any public statements concerning the evaluation. All decisions with regards to products or documents and other materials, which bear a direct relation to, or are produced, prepared, collected during the conduct of the evaluation, shall be made by the OCHA. The consultant shall not publish any confidential information or draft deliverables. The consultant is expected to declare any conflict of interest and sign a non-disclosure agreement as part of the contract. Other: (i) The applicant might point out in their application if they are applying as a team together with the senior evaluator/team leader (see JO 244858) and the junior evaluator/research assistant (see JO 245294) (ii) The applicant is expected to conduct several mandatory UN trainings (before and during the length of the contract)

No Fee

THE UNITED NATIONS DOES NOT CHARGE A FEE AT ANY STAGE OF THE RECRUITMENT PROCESS (APPLICATION, INTERVIEW MEETING, PROCESSING, OR TRAINING). THE UNITED NATIONS DOES NOT CONCERN ITSELF WITH INFORMATION ON APPLICANTS' BANK ACCOUNTS.