Project Number and Organization: Armenian Caritas / EA 04/2021/Armenia

Project Title: “Livelihood and Shelter Support to Conflict-Affected Population in Armenia”

 

Terms of Reference for Final External Evaluation

BACKGROUND

More than 90,000 people were displaced when war broke out between Armenia and Azerbaijan in Nagorno-Karabakh (NK) on September 27, 2020. A Russian-brokered peace agreement went into effect on November 10, 2020 stipulating that Armenia and Azerbaijan would maintain control of the territories they held respectively at the end of the conflict, and Armenia would transfer three additional districts of NK to Azerbaijan—leaving more than 10,000 families, mainly from areas newly under control of Azerbaijan, displaced in Armenia without permanent shelter nor access to livelihoods, and largely dependent on the generosity of host families and humanitarian assistance.

In March 2021, Armenian Caritas (AC) in partnership with Catholic Relief Services (CRS), led an assessment in Gegharkunik and Shirak regions to deepen the understanding of the updated needs of families impacted from the conflict, including displaced, host and local families.

  • Displaced: families from Nagorno-Karabakh
  • Host: families accommodating displaced families
  • Local: families from the local communities in Armenia who were directly or indirectly impacted by the conflict (e.g., direct: families with an injured, deceased, or missing family member; indirect: families who may have contributed to the war effort in some capacity and/or suffer consequences from the conflict, psychological or other).

The assessment was designed to be (1) rapid, participatory, and engaging—giving a voice to the perspectives, ideas and experiences of key stakeholders (including conflict-affected families, local government officials and community leaders); (2) evidence-based and (3) useful, with the ultimate goal of informing program design.

CRS and AC employed a combination of methodologies, including a desk review of secondary literature, analysis of program data, 11 semi-structured interviews with local leaders and government officials as well as 61 semi-structured interviews with displaced families (29), host families (13) and local families (19).

The analysis of the results of the assessment showed that families expressed a distinct need for longer-term, sustainable support—specifically, shelter and livelihoods. AC found that there were no actors providing longer-term shelter or livelihood support to individuals or families who had been impacted by the conflict.

Thus, based on conflict-affected families’ expressed needs and the persistent gaps in shelter and livelihoods, as well as organizational capacity, AC launched the project “Shelter and Livleihood Support to Conflict-Affected Population in Armenia” and made a decision to target Shirak and Gegharkunik regions.

 

DESCRIPTION OF RESPONSE

Armenian Caritas provided:

  1. Shelter assistance in the form of minor repairs to 500 vulnerable conflict-affected households living in substandard shelters (including displaced, host and local families)
  2. Light livelihoods support 400 displaced families in the form of productive assets so they may begin to resume their previous livelihoods and earn income.
  3. Mapping and referral activities to 600 families to create linkages and ensure coordination among actors in the field providing services/resources (iNGOs, NGOs, CSOs, local/regional government etc.)

Across all subgroups and regions, displaced, host and local families all expressed shelter needs and all interviewed families in Gegharkunik and Shirak said minor repairs would make the greatest impact for improving their living environment. For example, upgraded bathroom conditions in host families would result in improved hygiene/sanitation for the large number of people living in the household. Moreover, roof and wall repairs and/or window replacement would enhance thermal insulation and decrease costly heating bills during winter months. For families with household members recently injured in the conflict, upgrades or adaptations such as ramps and/or handrails would greatly improve accessibility.

These relatively simple minor repairs for displaced, host and local families are beyond their means, and yet completing them would translate into numerous gains, such as improved health and safety, increased accessibility, better protection from the elements, and greater energy efficiency during the winter and summer months.

The estimated average for shelter repairs is 385,000 AMD (644 Euro) per household.

The actual cost per household will be based on individual households’ needs and corresponding repair plans. Host families living in substandard shelters who are accommodating numerous displaced family members may have more substantial repairs than families who may only require disability access improvements. For this reason, the project is using an estimated average cost per household to budget for repairs.

Livelihoods

Contributing to income security amongst the displaced population is a critical entry point for survival and recovery as well as a precondition for stability, reintegration, and socio-economic development in the post-conflict context. Moreover, AC’s experience shows that livelihoods support, especially when paired with shelter support, can provide maximum impact for sustainable recovery. Yet, the project also recognizes that some displaced households may only need livelihoods support, and those who qualify, will be able to access that distinct support. As such, the project included a light rural and urban livelihoods component with a targeted focus on replacing displaced families’ lost productive assets.

The estimated cost per household to restore livelihoods assets is 250,000 AMD (418 Euro).

  • For displaced families in rural contexts, the cost would cover the purchase of a small livestock, and key inputs (e.g., chickens, rabbits, pigs, fencing, feed, crop seed, tools and/or equipment rental for land preparation.)
  • For displaced families in urban contexts, the cost would cover the purchase of small assets such as a sewing machine and materials, hairdressing tools, baking equipment, tools and inputs etc.

This support would enable displaced families to resume previous livelihoods and generate enough income (or produce enough sustenance) to meet their most basic needs and avoid using negative coping mechanisms.

AC used a cash-based modality for the livelihoods component to give beneficiaries maximum flexibility to purchase inputs (as per their plans) that meet their distinct livelihood needs, as per the IA-RP recommended approach.

Mapping and Referrals

To ensure displaced, host and local community members are aware of the resources/services available to them in their respective regions, the project developed a detailed mapping and referral system. Additionally, with the recent launch of several new government cash programs, project staff monitored the extent to which eligible households were aware of the programs and their ability to apply and access funds.

 

LOGICAL FRAMEWORK

Goal: Displaced and other conflict-affected families rebuild their lives

Strategic Objective 1 (SO1): Conflict-affected families live in dignified shelters

Intermediate Result 1.1 (IR1.1): Conflict-affected families make critical improvements to their shelter according to their repair plans

Output 1.1.1.: Conflict-affected families attend orientation sessions on the shelter repair process

Output 1.1.2: Conflict-affected families work with Caritas staff/engineers to develop shelter repair plans tailored to their needs

Output 1.1.3: Conflict-affected families receive shelter assistance

 

Strategic Objective 2 (SO2): Displaced families use livelihoods assets to meet their basic needs

Intermediate Result 2.1 (IR2.1): Displaced families use livelihoods assets to restart their livelihoods

Output 2.1.1: Displaced families develop livelihood plans, consistent with their previous livelihoods prior to displacement

Output 2.1.2: Displaced households receive livelihoods assistance

 

Cross-Cutting Intermediate Result (IR): Conflict affected families are referred to available services

Cross-Cutting Output 1: Mapping of services and programs

Cross-Cutting Output 2: Referrals provided to targeted displaced families and other conflict-affected families

 

Total Programme Budget requested: 793,889 EUR

Fundraised: 838,666 EUR

 

OBJECTIVES

The final evaluation will focus on the following key objectives:

  • Assess the relevance, appropriateness, effectiveness, accountability and impact/sustainability of the programme;
  • Assess the effectiveness of the leadership of Armenian Caritas in coordinating the activities within the project target locations and regions;
  • Identify lessons learned, best practices and recommendations to inform future programme design.

KEY QUESTIONS

Relevance/appropriateness

  • Was programme design based on an impartial assessment of needs? Are needs assessments disaggregated by age, sex and disability? Do they include people’s needs, vulnerabilities and capacities?
  • Did the assistance provided by Armenian Caritas meet the needs of the affected population? Were the persons most in need identified, selected, and supported by the programme?
  • Which parts of the assistance were the most appropriate and why? Which were least appropriate and why? Were activities aligned with the affected population’s needs and priorities?
  • Were recommendations and learning from past reviews and evaluations applied to the response?

Effectiveness

  • Was the response timely?
  • What internal and external factors affected the speed of the response?
  • Was the internal organizational and managerial structure of the project effective?
  • Were there appropriate systems in place to monitor activities, outputs and outcomes of the programme? Did monitoring outcomes inform programme adjustments/revisions?
  • Did the project activities lead towards the achievement of the expected results/indicators as set in the Results Framework?

Accountability

  • To what extent has the affected population been involved in the design or implementation of the programme?
  • Were appropriate systems of downwards accountability (participation, information sharing and feedback/complaints), put in place and used by project participants? Were project participants aware of the feedback/complaints mechanism?
  • Were project participants and communities aware of selection criteria?
  • Were project participants and communities aware of the assistance they should receive?

Coordination

  • How effective was Armenian Caritas in coordinating internally?
  • How effective was Armenian Caritas in coordinating with external stakeholders such as other agencies, organisations, the local and national government?
  • What aspects of coordination could be improved in the future and how?

Impact/Sustainability

  • Has the response strengthened local capacities?
  • What are the intended and unintended, positive and negative effects of the project?
  • What, if any, aspects of the programme will have a longer-term impact?

METHODOLOGY

The final evaluation will rely on two main evaluation stages (1) design phase (2) field phase

Design phase

  • The evaluator(s) will undertake a desk review of programme documentation including planning documents, project proposals, situation reports and quarterly reports;
  • The evaluator(s) will also review other relevant documentation such as minutes of decision-making meetings, assessment and registration forms, MEAL tool;
  • The evaluator(s) will review other monitoring and reporting documents from secondary sources (i.e UNHCR reports).
  • If appropriate, the evaluator(s) will obtain feedback from the CI HD on the most relevant sites to visit.

Field phase

  • After the design phase, the evaluator(s) will conduct fieldwork to collect and analyse data in order to answer the evaluation questions;
  • Data collection methods should be inclusive and utilise a range of methods, including focus group discussions and key informant interviews and with key project stakeholders. The use of surveys and other remote data collection tools should also be explored by the evaluators to maximise data collection;
  • The evaluator(s) should ensure a systematic triangulation of data sources and data collection methods and tools, and seek to validate data through regular exchanges with programme staff where appropriate.
  1. EXPECTED OUTPUTS

The evaluator(s) should produce the following key deliverables:

  • Draft Evaluation Report to be submitted to Armenian Caritas / CI support mechanism (if applicable) and the CI Humanitarian Department
  • Final Evaluation Report inclusive of:

 

  • Executive Summary
  • Background
  • Introduction
  • Context
  • Description of Methodology
  • Main findings
  • Conclusions inclusive of best practices and lessons learned

USE OF THE EVALUATION RESULTS

The intended audience for the evaluation are Armenian Caritas key staff, including senior management, CI MOs who have supported the programme, the Caritas Internationalis Humanitarian Department and the Caritas Confederation. Evaluation findings will be shared with programme participants as appropriate.

REQUIRED COMPETENCIES

The competencies required from the External Evaluator are:

  • Advanced degree in social sciences, political sciences, economics, development or related fields;
  • Experience in leading evaluations, especially in the field of humanitarian response;
  • Ability to use participatory approaches to evaluation;
  • Experience of operational management of humanitarian/development programmes;
  • Good knowledge of the local context;
  • Good analytical skills;
  • Excellent writing skills in a CI Confederation language (EN/FR/ES);
  • Any other appropriate language skills;
  • Understanding of the Catholic Church and Caritas structure and mission.

 

 

CONSULTANT’S PROPOSAL 

Proposals should include:

  • Proposed evaluation methodology (if different from above);
  • Description of deliverables and a timeline;
  • A financial proposal including the cost implication for other evaluators if any;
  • CV(s) of evaluator(s).

 

TIME FRAME

Add as many lines as needed to prepare Time Frame plan

Evaluation phase Activities Dates # of Days
Preparatory
Design
Field phase
 
Reporting
Total

 

 

Please submit your proposals to ch.mkhitaryan@caritas.am and info@caritas.am by June 12, 2022.