TERMS OF REFERENCE ENDLINE EVALUATION FOR OROMIA COFFEE AND DAIRY DEVELOPMENT PROJECT IN ETHIOPIA
Solidaridad Network is an international solution-oriented civil society organization working through eight regional expertise centres to transform markets to make them more sustainable and inclusive. Our eight regional expertise centres include Asia, Eastern and Central Africa, Europe, Latin America, North America, Southern Africa, South America, West Africa and our global Secretariat in the Netherlands. Solidaridad Eastern and Central Africa, one of the regional centres, headquartered in Nairobi, Kenya with country offices and programmes in Democratic Republic of Congo, Ethiopia, Tanzania and Uganda. We bring together supply chain players and engage them with innovative solutions to improve production, ensuring the transition to a sustainable and inclusive economy that maximizes the benefit for all. We facilitate strengthening of local capacity in developing countries, support the creation of enabling environments for economies to thrive, and improve market access.
Through Comic Relief funding Solidaridad in partnership with Oromia Coffee Farmers’ Cooperative Union, Bureau of Finance and Economic Development (“BoFED”) and other partners in the coffee sector (Coffee Unions and private plantation’s) are implementing Oromia Coffee & Dairy Development Project (“OCDDP”) project. The main objective of the project is to improve coffee production and climate resilience of the selected 10,000 coffee farmers and their families (3,000 being women) through increasing the income (and profit) of coffee production, promoting gender and diversification (entrepreneurial) activities for women and promoting livestock. By including diversity of export (coffee) and dairy production, risks and income will spread. The project’s zones of interest are Gujii, Borena and W/Aris zones of South Oromia. Under this funding 3 projects are currently running in Ethiopia while the key intervention areas for the OCDDP project includes delivery of training to farmers, providing technical support to the partners, hardware establishments and monitoring data and progress of activities.
2. OCCD PROJECT
At the project inception period, coffee farmers produced an average of 0.50 MT of parchment coffee per hectare with an average production area of 0.68 Ha per farmer. This was deemed to be below the existing potential and a significant production increase could be achieved with adoption of new technology packages, developed by research and Solidaridad for improving production of smallholder coffee farmers. Because of low production, coffee farmers’ income was relatively low (400-900 EUR per year on average including livestock and other crops) and farmers faced at least 2 to 3 months per year of serious food and nutrition insecurity.
The main objective of the project is to improve coffee production and climate resilience of the selected coffee farmers and their families through increasing the income (and profit) of coffee production, promoting gender and diversification (entrepreneurial) activities for women and promoting livestock. By including diversity of export (coffee) and dairy production, risks and income will spread.
3. SCOPE OF THE ENDLINE EVALUATION
This endline evaluation is expected to assess the effectiveness of the implementation strategy of the project in a direct intervention set up. This will include the implementation modalities and participation by the other consortium partners. The evaluation will include a review of the project design and assumptions made at the beginning of the programme development process. It will assess whether results have been achieved, the implementation strategy has been optimum and recommend areas for improvement and learning. The endline evaluation will also assess whether project resources were efficiently utilized to produce planned results. Lastly, the sustainability of the project will be looked at to ensure lasting impact.
The overall objectives of the endline evaluation are to:
a) Assess the OCDDP’s achievements against the outcomes and output indicators set at the start of the project.
b) Capture lessons learnt and provide information on the project approaches identifying what has worked and what has not worked for learning.
The evaluation has the following objectives:
● To assess to what extent the project achieved its formulated results (especially at output and outcome level) with households, smallholder farmers, and institutions (Producer organizations/cooperatives) benefiting from the project.
● To assess wider relevance & contribution of the project to food security and livelihoods in Ethiopia (with a focus on the project implementation zones).
● To assess the relevance and contribution of the project to the coffee, food crops and dairy sectors: adoption of innovations and technologies by smallholder farmers and groups that were both direct and indirect direct beneficiaries.
● To also provide a full discussion of the research questions concerning relevance, additionally and sustainability.
● To assess the appropriateness and effectiveness of strategies, approaches and modalities used in the project to realize the intended results.
● To assess the integration of innovation themes of gender (inclusion of women and youth) and climate change.
● To identify major bottlenecks and lessons learnt and document them to take into consideration for possible follow-up projects. Also, assess risks of pulling out at this moment, especially for the sustainability and durability of the interventions and investments.
● Performance of the direct Implementation strategy: did this strategy work? What is the strategy’s strengths and it’s weaknesses thereof?
The endline evaluation will systematically assess what has been achieved, what has not, and why, at the outcome level of the project and in the two impact pathways of the project (the supply-side pathway and the private sector development pathway) and its 3 themes. The project activities focus on the supply-side pathway and the private sector development pathway and within these activities can be grouped into: (1) Hardware/Organization investments; (2) Training; and (3) learning for sustainability. This will cover the entire implementation period (i.e. 01 July 2016 to 30 April 2021)
4. EVALUATION RESEARCH QUESTIONS
The evaluation team should consider but not limit itself to the following research areas and questions:
Relevance of the program strategy (design and focus of the programme):
● To what extent are the program objectives meeting the needs of small scale coffee farmers and their families?
● Does the program address the key bottlenecks to food security, nutrition, and improved livelihoods of the target group?
● Are the underlying assumptions and context to the program still correct?
● Are the program results framework, indicators and targets suitable (i.e. SMART) monitor and support implementation of the targeted results? If not, what are the suggested changes?
Effectiveness (project progress):
● To what extent has progress been made towards the planned outputs and outcomes, as defined in the program results framework? What recommendations for improvement can be made?
● Were there any unexpected positive or negative side effects and how were these dealt with by the programme management?
● Are the chosen program interventions and activities effective in delivering the desired outcomes?
● To what extent was the program implementation effective (Management arrangements, work planning, finance and co-finance, project-level monitoring and evaluation systems, Stakeholder Engagement, reporting, communications)
● What factors have contributed to or hindered the achievement of intended outputs and outcomes?
● To what extent have partners in the OCDDP project played their expected roles towards the achievement of the program goal?
● Has the consortium partnership been appropriate and effective?
Outcomes and Impact:
● How did the program contribute to lasting change to the lives of beneficiaries with regard to: - Increased local production of food crops and coffee: if there would be no sustainable agricultural practices and production systems, how would the local production situation look?
● Increased availability of nutritious and affordable local food crops and milk for farmers and for local markets; if there would be no business development and market linkages, what would the local markets look like?
● Has progress so far has led to, or could in the future catalyze beneficial development effects (i.e. income generation, gender equality and women’s empowerment, improved governance etc.) that should be included in the project results framework and monitored on an annual basis for the remainder of the project?
● Was the implementation strategy adopted and inputs identified realistic, appropriate and adequate for the achievement of the results?
● Did the actual or expected results (outputs and outcomes) justify the costs incurred? Were resources effectively utilized?
● What are the factors contributing to implementation efficiency?
● Are there any remaining barriers to achieving the project objectives by the end of the project period?
● How successful aspects of the project are further built on to expand the benefits?
● What is the likelihood of continuation and sustainability of programme outcomes and benefits after completion?
● What are the key factors that require attention to improve prospects of sustainability of outcomes and the potential for scaling up?
● What are the important challenges the programme should overcome? In addition, to what extent are these adequately addressed?
● What are the key lessons learnt based on the experiences of project implementation?
● What are the recommendations for similar support in future?
● Scaling: What are the project’s best practices & results to scale up? What are the lessons learnt to be considered in the design of future similar projects? What are the challenges & opportunities for scaling up the good practices of OCDDP within the current Ethiopian institutional and sector context?
Organizational and institutional set-up:
● How has the set-up of the project consortium and teams contributed to the success (failure) of the project? What lessons can be learnt?
In these research areas, the issues of social inclusion (gender, women’s empowerment and youth involvement) and climate change should be taken into consideration and recommendations for improvements in future programs given. The Consultants may choose to categorize the questions differently, as long as these questions are addressed.
This evaluation requires a mixed method approach that allows methodological triangulation to increase the validity and credibility of data. Participatory methods shall be used to collect both qualitative and quantitative data. These methods shall include the following, but not limited to:
● Household survey using survey questionnaire (a representative sample of the total beneficiary population can be used)
● Literature review of existing documents and review of context and poverty trends including the project proposal and other documents, annual and quarterly reports, monitoring and evaluation reports
● Field observation of the targeted project areas
● Key informant interviews (KII) with project management team, local authorities and local development institutions
● Reflection and feedback sessions with the project management team.
The final methodology to be adopted shall be discussed and agreed with consultants/firm during the inception meeting.
6. EXPRESSION OF INTEREST REQUIREMENTS
An expression of interest (EOI) for the endline evaluation should include: outlining the proposed methodology and process of data collection and analysis that specifies:
● Proposed methodology for implementation of the study, including sample sizes.
● Data collection and analysis approach.
● A detailed work plan for the assignment.
● A brief overview of the team.
● A budget overview and narrative.
The full EOI should be a maximum of 15 pages.
7. REQUIREMENT AND SELECTION CRITERIA OF ASSIGNMENT
The consultancy is open to both individual consultants and consultancy firms. Evaluations will be undertaken in accordance with the evaluation criteria (Annex A). The requirements are as follows:**
▪ International track record and demonstrated experience regarding evaluation of agriculture related projects in Africa.
▪ Three Reference letters from donors or other organizations who commissioned project evaluations, indicating how they appreciated the quality of project evaluation work done by the consultant/firm.
▪ Written engagement stating that the consultancy and its team members do not have any direct or indirect interest in the results of the project evaluation and possible future new projects.**
Selection criteria for the evaluation team:
▪ The team leader should have demonstrated experience & expertise in project evaluation in Africa, as well as experience in agricultural value chain development in Ethiopia. Experience in private sector inclusive agribusiness programs is important. Essential to have experience in the area of smallholder coffee, food crops, dairy sector development, agribusiness development, and proven experience in project evaluations.
▪ At least one of the team members should have experience in assessing gender and youth inclusion.
▪ In case of an individual consultant, s/he should be able to Amharic and Oromifa or if it is a firm, at least one member of the team should be able to speak Amharic and Oromifa.
8. ASSIGNMENT TIMELINE
Interested parties should submit a work plan and budget proposal.
Budget proposals should include logistics costs (transport, communications, meals and accommodation).
5th March 2021, 5:00pm (EAT)
Decision of selected consultant - only selected candidates will be contacted.
8th March 2021
Inception report and meeting (virtual or in-person depending on covid19 situation).
12th March 2021, 5:00pm (EAT)
Draft report submitted
9th April 2021, 5:00pm (EAT)
Validation meeting and final report (virtual or in-person depending on covid19 situation).
16th April 2021, 2:00pm (EAT)
How to apply:
1. APPLICATION PROCEDURE
The consultant is required to submit a letter of interest containing a statement on candidate’s experience with similar assignments, curriculum vitae, financial proposal and work plan by 5th March 2021 at 5:00pm (EAT).
Subject reference: “OCDDP– Endline Evaluation”
Submissions are by email to email@example.com while addressed to:
Attn: Managing Director
Solidaridad Eastern and Central Africa Expertise Centre
Kilimani Business Centre, Kirichwa Road,
P.O. Box 42234 - 00100 GPO
ANNEX A: EVALUATION CRITERIA
The consultancy should deliver a technical and financial proposal that will be assessed as follows:
Technical proposal Max 90 points
CV or profile
● Experience with smallholder coffee, food crops and dairy value chain development in Ethiopia, including private sector/market system development (access to markets)
● Experience in Ethiopia
● Experience in gender and youth inclusion
● Experience in program evaluation, especially of value chain development programs and with qualitative data collection
Understanding the assignment, proposed approach and methodology
● Feedback on ToR
● Proposed approach & methodology
● Clarity on roles and responsibilities in team
● Clarity & adequacy of work schedule
● Experience in program evaluation
Note: Minimum requirement for technical proposal to be eligible: 70 points out of 90.*
Financial Proposal: Maximum 10 points for the lowest eligible bid. Others get proportionally less.