Pays: Arménie
Sources: Caritas France – Secours Catholique
Closing Date: 20 juillet 2021
Final evaluation of the project Aramazd 2018-2021:
Socio-economic development in the border regions of Armenia through partnership and cooperation with local actors
Background and short description of the project:
This project is the third phase of a program that was launched in 2012 by Caritas Armenia with the financial support of Caritas Spain and Secours Catholique – Caritas France. The third phase of the program started in 2018 and was planned for a three-year period. The project aims to promote sustainable socio-economic development in the border regions of Armenia through partnership and cooperation with local actors.
At the time of the final evaluation, the third (final) year of the three-year project is underway. A midterm project evaluation and next phase design consultation were carried out in April 2018.
What:
The goal and objectives of the project are:
- Overall objective: Promote sustainable socio-economic development in the border regions of Armenia through partnership and cooperation with local actors (local authorities, civil society organisations, SMEs, young active citizens).
- Specific objective: Contribution to socio-economic development through support to entrepreneurship and rural employment, cooperation and strengthening of local actors in the communities of Shirak, Gegharkunik and Lori regions with a particular focus on youth.
Whom:
The main target group of the project are:
- Vulnerable communities from Gegharkunik, Lori and Shirak Provinces of the Republic of Armenia, suffering from poverty, unemployment and being at risk of migration
- Local authorities, civil society associations, economic actors and active youth
How:
Outcome 1: Effective links are established and strengthened between civil society organisations and local authorities and businesses to improve living conditions and provide work opportunities for young people.
- Project stakeholders are identified;
- Joint initiatives to promote economic activity and youth employment are defined; new projects are initiated and implemented in the communities.
- A network between stakeholders is created and their joint activities are supported.
Outcome 2: Sustainable economic development mechanisms are stimulated in rural border communities:
- Vocational training sector is improved to provide work opportunities and prospects for rural youth through vocational training capacity building and technical support.
- Internship and exchange opportunities are promoted for youth
Outcome 3: Community projects are implemented
- Economic activities in tourism, agriculture and agribusiness are developed through investment and innovative approaches.
- Opportunities are created to improve community services, stabilise the socio-economic situation in the villages and raise awareness of funding models to make this community development approach successful.
Who:
Organisations implementing the project: Caritas Armenia staff,
In partnership with :
- Ministries of Territorial Administration and Development, Social Affairs and Labor, Education and Science,
- Local Authorities
- Civil Society Organizations
- Small and Medium Entrepreneurs
- Young active citizens.
Where:
Bordering settlements of Gegharkunik, Lori and Shirak provinces of Armenia.
When:
October, 2018 – December, 2021
Aims and objectives of the final evaluation of the project
Objectives of the final evaluation:
(1) Evaluate and analyse to what extent the project objectives and results have been achieved and how sustainable the results achieved have been, and what impact the project has had on the main target group of the project;
(2) Evaluate and analyse project successes and problem areas, analyse the reasons for achieving or not achieving the planned results and draw key lessons learnt
(3) Check if the recommendations of the previous evaluation have been taken into account and make recommendations for future project planning for this project (type of activities, target groups, further stakeholders to be considered). Recommendations drawn by the evaluator will feed into the preparation in the next phase of the program.
Detailed Objectives of the final evaluation:
- Evaluation of the project achievement based on its relevance, effectiveness, efficiency, sustainability, and impact according to the questions below:
- Relevance: Do the activities proposed answer the needs assessed in the regions covered, with respect and in collaboration with other NGOs implementing activities, and without duplication? To what extent does the project respond to the main needs of the target group of the project (men, and women) and satisfactorily address their main problems? Could other problems be addressed in order to ensure an increased relevance of the project without affecting its coherence (eg. NGO staff burnout)? Is the project strategy (involving its three main components), well balanced to achieve the project overall and specific objectives set? Did the project accurately adjust to new needs that arose with the spread of COVID-19, the breakout of the war in Nagorno Karabakh and the negative financial consequences of those events for the country (inflation etc…)?
- Effectiveness:
(a) Project structure: To what extent is the project strategy logical, the indicators are SMART, and the project’s internal monitoring and evaluation system allows the achievement of milestones and indicators to be monitored and allows project activities to be reviewed as necessary?
(b) Project achievements: To what extent have the objectives and outcomes of the project been achieved? Are there any unintended results (positive or negative)? What are the main factors contributing to or hindering the achievement of the project objectives?
(c) Partner’s capacities: Does the partner organization have the necessary experience, knowledge, skills, capacity in the area of project implementation, project management, human resource management, monitoring and evaluation? Do they have direct access to the target group and communities? Are systematic approaches (forward planning of activities, use of eligibility criteria for participants, openness to all genders etc.) applied when identifying beneficiaries of project activities (e.g. participants, training)? To what extent do project management and coordination contribute to project effectiveness
- Efficiency (profitability): Would it have been possible to achieve the project results with fewer resources? Would it have been possible to achieve more results with the same resources?
- Impact: What is the impact of the project on the different target groups? What are the most important changes in beneficiaries’ lives, attitudes or behaviour that have been achieved by the project? Is there any positive change in the situation of the target group representatives as a result of the partnership between civil society representatives, state authorities and other stakeholders developed under the project? Assess the added value of the project among other similar initiatives in the area of coverage, whether it is a good complement to the activities of other stakeholders or not
- Sustainability: To what extent will the project direct achievements, as well as the beneficial indirect changes achieved by the project in relation to the project target group, be sustained after the end of the project? What preconditions have been established to ensure the sustainability of the project outcomes?
Identification of key lessons learnt
What were the main successes and problem areas throughout the realisation of the project? What were the reasons for achieving or not achieving the planned results, and were steps appropriately and systematically taken to readjust the project considering the change of context?
Identify and document key lessons learned and best practices from the project that could be used by the project management, on the continuation of this project or others.
Recommendation for future projects
Identify gaps and make recommendations to improve project design (project strategy, SMART indicators, achievable objectives and targets), project coordination, and project management;
Provide concrete recommendations on how to optimize the project in the future, to increase its impact and possibly, make it more relevant to the new needs to address in the area of coverage. These recommendations could involve:
- Target groups of beneficiaries. Shirak, Gegharkunik and Lori are inhabited by long-term labour migrants, internally displaced people and war-affected people whose needs are addressed through two other projects involving the same donors. Could some linkages and synergies be found between those projects?
- The type of activities developed and crosscutting sectors. Eg: sustainable energy and ecology, social transformation, gender, human rights…
- Relevant stakeholders in the area of coverage that could be involved in the project implementation.
- Evaluate the SCCF and Caritas Spain Added Value
- Has there been any added value to the project thanks to the partnership with SCCF and Caritas Spain (other than financial contribution)? What could SCCF and/or Caritas Spain do to better support Caritas Armenia in the project?
- How efficient and effective was the partnership and the coordination between Caritas Armenia, the SCCF and Caritas Spain (financial support aside)?
- Was there experience sharing between SCCF, Caritas Spain and Caritas Armenia’s teams during activity implementation?
- Was there a 2-way consultation taking place between SCCF, Caritas Spain and Caritas Armenia?
Scope of work and timetable
The estimated period of the final evaluation is 1,5 month, September/October 2021. The following is a table of the anticipated works and a timetable for the evaluation team:
Lead time
Type of activity
- First two weeks (September 2021)
Desk study: review of the project documentation: project proposal, project logical framework, project budget; Code of Ethics and Safeguarding; latest narrative and financial report of the project for the period 2020-2021; latest final evaluation report; partners’ social media activities; latest SCCF monitoring reports.
On-line interviews/written surveys among project managers and coordinators and other staff to assess project coordination and management, and to collect best practices and lessons learnt
- Two weeks (End of September)
Fieldwork/ Observation of at least one event in each location (as suggested by the project manager from Caritas Armenia)
- Beginning of October
Meeting with managers of Caritas Armenia, Caritas France and Caritas Spain (online or offline) to discuss preliminary evaluation findings, lessons learnt and good practices
- By the 6th of October COB
Preparation of a preliminary version of the report in English, to be sent to managers of Caritas Armenia, Caritas France and Caritas Spain by the 6th of October
- By the 12th of October 2021 COB
Comments from to managers of Caritas Armenia, Caritas France and Caritas Spain
- By the 17th of October 2021
The final version of the report is sent to managers of Caritas Armenia, Caritas France and Caritas Spain.
Methodology
The evaluation team should use various interactive methods during the collection of information and all necessary data to answer all the questions of the Terms of Reference. The list of those to be interviewed during the final evaluation should include project staff, representatives of national and local authorities with whom the project works, key stakeholders of the project and representatives of the target group. The methods to be used during the final evaluation should be described in detail in the application form for the final evaluation of the project. Project staff involved in interviews should include both project managers and other key project staff working directly with representatives of the target group and with representatives of national and local authorities in the four project countries. In order to save time and budget, it is possible to conduct interviews and a general meeting with the managers to confirm the preliminary results of the study and identify lessons learnt/best practices online.
The evaluation team may suggest other methods to collect information from project stakeholders and beneficiaries to implement Terms of Reference.
Expected results (outputs) and timetable for delivery:
- Detailed methodology for the Terms of Reference with indicative questionnaires for interviews/surveys and evaluation work plan – by 8 September 2021;
- Preliminary version of the report in English – by 6 October 2021
- The final version of the report in English is due by 17 October 2021.
Requirements for the report:
The report should be written using the format 15-30. The first page of the report should contain the recommendations; the next 3-5 pages should contain the Executive Summary and the next 25-30 pages should contain the full report. All questionnaires used during the evaluation should be attached to the report. The report should be well structured; the conclusions and recommendations in the report should be well argued.
Qualification requirements for consultants (experts) carrying out the final evaluation:
May only apply to this call registered independent consultant or consulting firm that comply with the following cumulative criteria:
- With a university degree, master’s degree (at least) in the social sciences: sociology, law, political science or other related sciences);
- Proven experience (at least 5 years) in conducting project/programme evaluations and skills in conducting impact evaluations on the target group, using an interactive methodology, conducting interviews, developing questionnaires, etc;
- Proven experience in civil society/NGO development and/or international development;
- Experience in socio-economic development is highly preferable;
- Fluent Armenian language is a must to carry out interviews
- Proven analytical and reporting skills in English;
- Knowledge of the regions where the final project evaluation will take place (Gegharkunik, Lori and Shirak provinces of Armenia) is a strong asset.
In case an evaluator meets most of the requirements but does not have knowledge of the region or does not meet any of the other requirements mentioned above, he or she can include in the team of evaluators another consultant or consultants who meet the remaining requirements.
Budget and financial plan for the final evaluation:
The applicants may use a free format for the elaboration of the budget, with the only requirement that the honorariums of the evaluators and the budget for the evaluation are indicated separately in the budget. All expenditure necessary and justified for the evaluation and indicated in the budget will be reimbursed at a later stage, once the relevant financial documents supporting the expenditure have been submitted. When working on the budget, applicants should take into account the fact that fieldwork requires knowledge of Armenian language or costs for interpreters.
The payment will be made as follows:
- 1st instalment – 40% (after signing the agreement); and
- Final instalment – 60% (after approval of final deliverables)
Comment postuler
Candidates planning to apply for the final evaluation of the project may submit an application form in free format, but with a mandatory indication that these candidates meet the eligibility requirements, attaching a CV and reports on similar missions, a detailed description of the methodology, an updated work plan, and a budget for the evaluation. If the final evaluation will be carried out by a team of experts, their role in the Terms of Reference and their knowledge and experience in carrying out project and/or programme evaluations, and added value should be described.
Applications for the final evaluation must be submitted by 20 July 2021 at the latest to missioncourteduree@secours-catholique.org