Background
and Purpose
Maternal and
Child Health
The Government of
Rwanda, through the Ministry of Health, is committed to achieving MDG goals and
beyond related to maternal and child health. In Rwanda, the maternal mortality
ratio is 210 deaths per 100,000 live births (DHS 2014/2015). Of all pregnant women,
99% received antenatal care (ANC) in 2014/2015 according to the Demographic and
H
ealth Survey. ANC was typically given by a nurse of medical assistant. Despite
the large percentage of women receiving ANC, only 44% met the standard of at
least 4 ANC visits. Of those having 4 ANC visits 44.3% were in a city and 43.9
% in rural environments. The proportion of women delivering at a health
facility in Rwanda is 91% (DHS 2014/2015), up from 69% in 2010. After birth
only 41.6% of women in Rwanda had a postnatal check-up with in the first two
days after birth in (DHS 2014/2015).
In the health
sector, concerted efforts have been made to address the intervention gaps that
lead to maternal, new-born and child mortality. The Government has high
commitment to child survival, and introduced task-shifting in the health system
(the introduction of 45,000 Community Health Workers in 15,000 villages
countrywide) and scaled up high-impact interventions (such as immunizations).
RapidSMS is one of the community approaches to addresses the delay in accessing
maternal, new-born and child health care.
RapidSMS
In order to improve
effective follow up of mothers and new-borns through the health system, an
innovative mobile technology tracking system “RapidSMS” has been developed to
track the continuum of care life cycle (first 1,000 days of a child’s life,
from conception to 2 years ). RapidSMS was initially introduced by the
Government of Rwanda as a pilot project in one district in 2009, as part of
Rwanda’s e-Health and m-Health initiatives, with technical and financial
support from UNICEF. The system has then been adopted and scaled up to all 30
districts, and is now operated by 45,000 CHWs in 15,000 villages across the
country. The system improves tracking of pregnant women (pregnancy
confirmation, emergency response through Red Alert messages and response
scenarios) and new-born health. RapidSMS is an open source software system at
the central level, which offers the opportunity to register pregnancies at
village level to monitor their conditions using real time data. The system uses
reminders, notifications and feedback messages to mothers through CHWs.
Providers at central and health facility level are able to fast track
individual cases including high risk pregnancies, high risk newborns, status of
child nutrition, major child killer diseases (malaria, pneumonia &
diarrhoea), un-immunized children in the community, and maternal, newborn and
child deaths.
To accelerate the
reduction of maternal and child deaths, in 2012, the RapidSMS system was
upgraded to track the full cycle of the first 1,000 days of a child’s life in
Rwanda. The system now covers and tracks the entire maternal and child
continuum of care, from pregnancy, birth, postnatal and new-born care services;
immunization to maternal and child emergencies; community-based nutrition
(CBN); hygiene and sanitation; and addressing childhood killer diseases through
community case management. In the past year, the new version of Rwanda’s
RapidSMS system has been further technologically upgraded and enhanced to
effectively store, manage and display data on the maternal, new-born and child
continuum of care from 1,500 villages countrywide.
Purpose
of the evaluation
To support the
Government of Rwanda, through the Ministry of Health and UNICEF to conduct an
evaluation of the ongoing RapidSMS/MNCH project. The aim is to document,
analyze and disseminate the latest RapidSMS project experience for future use
in programming. The evaluation will look at programmatic achievements and constraints,
to explore and understand the reasons behind those. The findings and
recommendations will be used for developing new policy documents and technical
guidance and to improve RapidSMS implementation. This is a formative
evaluation: the specific recommendations will be used in designing, planning
and enhancing implementation of innovative RapidSMS project.
Justification
In order to conduct
an evaluation of the RapidSMS initiative, there is a need to hire an evaluation
consultant/institution. The RapidSMS has been implemented since 2009 in Rwanda
and is being scaled up to all 30 districts in the country. An evaluation is
required to document programmatic achievements and constraints, to develop new
technical guidance to improve RapidSMS implementation. Given the expected
workload of documenting and evaluating the initiative at national and local
level, the volume of technical deliveries, and the required technical
expertise, the tasks are beyond the current capacity of UNICEF Rwanda’s
in-house personnel
Objectives
/ Study Aim
In order to inform
the scale up phase of the project and to develop lessons for the replication of
its successes beyond the project’s lifetime, an evaluation of the work done to
date is required to assess successes, shortcomings and the replicability to
nationwide scale and its contribution to evidence-based policy change.
The overall
objective of this evaluation is thus to understand whether the intended
objectives of the project have been achieved, in line with the plan.
The evaluation will
seek to elucidate the impact[1] of RapidSMS-MCH on maternal and child health in
30 districts in Rwanda. Little analysis has been performed regarding the impact
of Rapid SMS-MCH on maternal and child health in Rwanda, so this study will
fill a knowledge gap and inform the development of future iterations of Rapid
SMS-MCH in Rwanda and other countries.
Specifically, the
evaluation will determine to what extent the intervention has been able to meet
its objective to create capacity, tools and structures to respond to the high
levels of newborn and maternal morbidity and mortality in Rwanda.
The findings of the
evaluation will be used by MoH and Districts Hospitals/Health Centres and by
the project partners – UNICEF, and donors – in their different capacities and
functions, to develop future plans and interventions and to inform policies and
strategies and improve performance of the project.
Scope,
Focus and Evaluation Criteria
Geographically, the
scope of the evaluation should cover the areas of implementation of all
districts of Rwanda, as well as the national level, to ascertain its sphere of
influence on the overall MNCH programme in Rwanda.
The time period
covered by this evaluation will be since the onset of the initiative (2009).
However, this evaluation may find some aspects of the programme or
data/information will be difficult to collect/analyse for some districts for
the prescribed time period. In addition, the absence of programme theory and
baseline data may pose challenges in establishing a possible causality. Those
elements are considered limitations for this evaluation.
The evaluation
should focus on and include the following beneficiaries and stakeholders in the
process:
Final
beneficiaries:newborn
babies, mothers and other caregivers and communities including volunteers;
Service
providers:health
care professionals whose capacity has been built (including doctors, midwives,
community health nurses and sub district health professionals) and CHWs;
Sub-national
decision-making level:District
and health facility authorities;
National
decision-making level:national
authorities and key stakeholders (Ministry of Health, RBC Development Partners
– KIOCA, USAID, JHPIEGO, UN System- UNICEF, WHO, UNFPA);
· National Professional
Societies and Academia:Rwanda Paediatric Society, Midwifery
Society, School of Public Health, Teaching Hospitals etc.
The evaluation will
be guided by OECD/DAC evaluation criteria of relevance, effectiveness,
efficiency, and sustainability. It will also look at criteria of interest to
the Ministry of Health and UNICEF, including coherence and Human Rights-Based
Approach to Programming and equity. The criteria should be analysed from the
perspective of the following objectives:
Objective 1
is to assess the programme
impact
- To what
extent did the programme contribute to reducing maternal, newborn and
child mortality? - What
was the role of partnerships at national and district level in achieving
the programme results? Has the joint partnership of Government of Korea,
General Electric (GE), UNICEF and Government of Rwanda through this
project contributed to elevating the issue of maternal and newborn health
on the national agenda?
Objective 2
is to assess the programme
relevance
- To what
extent has RapidSMS contributed to national developmental targets, health
included?
Objective 3
is to assess the programme
effectiveness
- Overall,
have the interventions reached the intended number of beneficiaries
according to the stated objectives and timeframe?
National
decision-making level:
- To what
extent has the project contributed to the improvement of the maternal,
newborn and child health continuum of care? To what extend has the project
contributed increased access and utilization of those services?
Sub-national
decision-making level:
- To what
extent was capacity of national health management teams, district health
management teams and health facilities built for planning, implementing
and monitoring maternal-newborn health services?
Service providers’
level:
- To what
extent and how did the intervention improve service providers’ knowledge,
skills and practices on essential maternal and newborn care?
Final beneficiaries’
level:
- To what
extent do mothers and families perceive overall change in the health conditions
of mothers and newborn babies as a result of RapidSMS? - To what
extent do beneficiaries report to have been reached by RapidSMS?
Objective 4
is to assess the programme
efficiency
- Were
the allocated resources used efficiently to achieve the project
objectives? Are the available resources adequate to meet project needs? - How
could efficiency be improved?
Objective 5
is to assess the programme
sustainability
National level:
- To what
extent has the Government of Rwanda prioritized RapidSMS in the
government’s policies, strategies and budgets? How could sustainability be
strengthened?
Sub-national
decision-making level:
- To what
extent do District administration, District Hospitals/Health Centres, and
District Health Units demonstrate ownership and capacity to consolidate
the achievements and the expansion of RapidSMS for maternal, newborn and
child health interventions?
Service providers’
level:
- To what
extent has the capacity building component of the intervention developed
stronger institutions and processes, as opposed to only developing
capacity at individual level? - To what
extent has commitment and motivation of CHW supervisors and CHWs been
built to continue implementation of RapidSMS?
Objective 6
is to assess programme
coordination
- What
were the overall programme coordination mechanisms? How can programme
coordination be improved? - To what
extent is the project facilitating synergies and avoiding duplications
with other interventions and strategies promoted by development partners?
Objective 7
is to assess the application of a human rights-based approach
(HRBA) and equity focus
in programming
- To what
extent does the project apply the HRBA and equity approach (i.e. focus on
most deprived areas, areas with high prevalence of critical newborn and
under-5 mortality, low income families)? - Is data
sufficiently disaggregated to identify excluded groups?
Objective 9
is to assess the results-based
approach
in programme management
- Were
the objectives and indicators SMART? - How
often were outcomes and outputs measured? - To what
extent was data used in decision-making, e.g. adjusting the planned
results, shifting the focus?
The evaluation needs
to adhere to the Government of Rwanda’s evaluation standards. Further key
policies and performance standards to be referenced in evaluating the programme
are described in the United Nations Evaluation Group (UNEG) “Standards for
Evaluation in the UN System” and in UNICEF Evaluation “Policies and Principles”.
The basics of human rights-based approach and results-based approach to
programming are described in the UNICEF Programme Policy and Procedure Manual.
6
Methodological Approach & Process,Expected Output
Type of
Study:The
evaluation is expected to be a mixed-method (quantitative and qualitative),
retrospective longitudinal study, analysing the trends in maternal and newborn
access to care as well as maternal and child health before and after
implementation of Rapid SMS-MCH.
Data Source:On the quantitative aspect,
relevant maternal and child health data will be abstracted from HMIS for the
period before and after Rapid SMS-MCH implementation. Patient level data will
be collected from the Rapid SMS-MCH database for high performing districts,
defined as having a reporting rate greater than 85%. In addition to analysing
available aggregated RapidSMS and MNCH quantitative information, the consultant
will further be expected to collect data from the field, as well as any other
secondary sources of relevant information. The qualitative component will draw
on the understanding and perception of the main stakeholders involved in the
project.
The evaluation will
not attempt to quantitatively measure the behavioural change that occurred (due
to lack of baseline information on this sphere) but will use facility-level
information about newborn and maternal health indicators and RapidSMS data to
determine improvements, as well as qualitative information from a large pool of
stakeholders.
Suggested
quantitative data analysis
Pregnancy Cycle:Descriptive analysis of
demographic data will be performed to describe mothers enrolled in Rapid
SMS-MCH. Descriptive analyses will be used to describe trends in antenatal care
visits in proportions since 2007. Trends in antenatal care visits will be
compared pre and post Rapid SMS-MCH introduction to determine the percentage
change in antenatal care visits in the two years post-Rapid SMS-MCH
introduction to the baseline (two years before).
Birth:Descriptive analyses will be
used to describe trends in delivery locations, births attended by a skilled
health worker and breastfeeding within 1 hour in proportions since 2007 and
birth weights in absolute numbers. Trends in delivery locations, births
attended by a skilled health worker and breastfeeding within 1 hour will be
compared pre and post Rapid SMS-MCH introduction to determine the percentage
change in delivery location, births attended by a skilled health worker and
breastfeeding within 1 hour in the two years post-Rapid SMS-MCH introduction to
the baseline (two years before). Additionally the proportion of women cared for
at a health facility or by a skilled health worker after being identified
through Rapid SMS-MCH as previously giving birth at home or having had another
previous complicated birth will be determined.
Postnatal:Descriptive analysis will be
used to determine trends in the number of mother and child visits to health
facilities with postnatal complications. Averages will be compared to pre-Rapid
SMS-MCH period baseline. Using HMIS data, trends in maternal and newborn deaths
for the selected districts before and after Rapid SMS-MCH will be compared to
determine the percentage change.
The evaluation
methodology will be guided by the norms and standards of the United Nations
Evaluation Group (UNEG), and the UNEG guidelines on integrating Human Rights
(HR), Gender Equity (GE) in Evaluation. In order to be responsive to HR and GE
aspects, special consideration will be given to gender, sex, distance from
service locations and wealth when stakeholders and beneficiaries’ view are
sought in data collection. In the design phase of the evaluation framework,
careful considerations will be given to such inclusion aspects. In the analysis
phase, appropriate disaggregation will be attempted to shed light on HR and GE
elements.
The evaluation
sampling strategy will be further defined for the key indicators with support
from the consultancy institution. A two-stage cluster design will be used to
select sample clusters and households to estimate the indicators stated above
at district level.
One possible
approach could be to capture evidence of evolution towards mortality reduction,
through the counter factual strategy by comparing matured districts and new
districts; equivalent to a quasi-experimental design.
The international
lead evaluator or institution will work with the Rwanda MoH and UNICEF (and
other partners when needed) to finalize the design and conduct the evaluation
under the leadership of the steering committee. The evaluation team will work
with the Lead Evaluator to provide assistance for the situation analysis in
line with the country context and quantitative assessment of the intervention
by collecting and using the service delivery data. The evaluation team will
share the responsibilities for field visit, data compilation, data analysis and
drafting of the report. The evaluation team will further work with the steering
committee and other stakeholders to coordinate the work, conduct interviews,
conduct the data collection and analysis, and disseminate the findings of the
evaluation.
The evaluation
process and methodology will include three phases:
Phase 1.
Inception:
- Evaluation
Plan development – draft work plan to be submitted to the Steering
Committee for approval. - Reconstruction
of theory of change, and establishment of an evaluation framework in
participatory manner. - In-depth
desk review of available documents related to MNCH and RapidSMS, data on
MNCH from RapidSMS and HMIS, facility-based data from the project
districts (before project implementation for comparative analysis) and
during the project implementation), National/District reviews and other
literature related to Rwanda’s RapidSMS/MNCH programming; - Preliminary
discussions with the Rwanda MoH/MCCH Division Head Coordinator, members of
National Maternal Child and Community Health Technical Working Group,
steering committee and UNICEF, to facilitate a common in-depth
understanding of the conceptual framework, refining the evaluation
questions and adjusting data collection methods, tools and sources; - Drafting
of Inception
report (deliverable 1),including the details of the
methodology to be used, an Evaluation Matrix for each finally agreed
evaluation question and a detailed analysis plan, to be presented to and
approved by the members of steering committee. The proposed methodology
needs to be sufficient to capture all the indicators agreed for this
evaluation purpose.
Phase 2.
Data collection:
- In
alignment with the agreed methodology, the consultant firm will carry out
a mix of focus group discussions and in-depth interviews with key health
care providers, health managers at National, District & Health
facility level, community leaders, community volunteers, mother support
groups, caregivers, MoH officials, Maternal Child and Community Health
Technical Working Group members, donors, members of the UN System and
development partners. The consultant will submit a report with the key
information and findingscollected through interviews (deliverable 2); - Field
visits to Project Districts and review –to the extent possible– CHWs and
health facilities documents and records of newborn case management from
2012 to June 2015. The field
reportwill constitute deliverable 3.This data
will be compared with the facility based data from the period before the
project; therefore data also needs to be collected for above mentioned
period to allow for the comparative analysis.
Phase 3.
Analysis and reporting phase:
- Following
the completion of the fact-finding and analysis phase, the evaluation team
will make a presentation
of the key findings (deliverable 4)to MoH, Maternal Child
and Community Health Technical Working Group and other key stakeholders
working on MNCH. Once these are discussed and validated by the Maternal Child
and Community Health Technical Working Group, a draft final reportin
English should be shared with key partners for final review and
validation. The final evaluation report, three case studiesof good
practices (deliverable 5),
and anacademic article (deliverable 6)are the end products,
subject to approval by the Steering Committee. 7 Major Tasks,
Deliverables &Timeframe
Tasks
Desk review
of available documents related to RapidSMS/MNCH, data on MNCH from RapidSMS
system & HMIS, National/District reviews and other literature related to
Rwanda’s MNCH programming
Expected
Deliverables
Feedback
meeting on findings from desk review
Timeframe
Week 1
Tasks
Reconstruct
theory of change, and establish an evaluation framework in participatory manner
Expected
Deliverables
Theory of
change and evaluation framework
Timeframe
Week 2
Tasks
Design of
the data collection and relative tools and preparation of inception report.
Expected
Deliverables
Inception report
(deliverable 1)including
work plan, methodological approach, instruments to be used, interview and field
visit protocols, annotated outline of final report[2], to be presented and
approved by the Steering Committee.
Timeframe
Week 3 &
4(1st
payment, 80%)
Tasks
Obtaining,
cleaning and analysing quantitative data
Expected
Deliverables
Quantitative
data analysis progress report
Timeframe
Week 5 &
6
Tasks
National
level stakeholders meeting and interviews: MoH/MCCH, UN System, Maternal and
Child Health Technical Working Group (MCCHTWG)
Expected
Deliverables
Brief report of the
in-depth interviews (deliverable 2)
Timeframe
Week 6 &
7
Tasks
Field work
(selected districts) including interviews with DHU, DH & HC, service
providers, beneficiaries, sampled facilities visit and interviews with primary
beneficiaries
Expected
Deliverables
Field visit,
observation and interview report (deliverable 3)
Timeframe
Week 7-9
Tasks
Analysis of
findings and draft report preparation
Expected
Deliverables
PPT presentation
(deliverable 4), or presentation in other format on the key findings;
preliminary draft of the analytical report, draft case studies of good
practices and two-page executive summary[3]),
Timeframe
Week 10 & 11
Tasks
Debriefing and
validation of findings with the National MCCHTWG and other stakeholders
Expected Deliverables
Meeting report
confirming the validation of findings, including feedback/recommendations from
stakeholders.
Timeframe
Week 12
Tasks
Incorporate comments
from key stakeholders and finalization of the evaluation report
Expected
Deliverables
Final evaluation
report and case studies of good practices(deliverable 5)as end product, subject to
approval by the Steering Committee
Timeframe
Week 13
Tasks
Writing and
submission of article to peer-reviewed publication
Expected
Deliverables
Academic article
(deliverable 6)
Timeframe
Week 14 (2nd payment, 20%)
The final report
should be in line with agreed Government and UNICEF evaluation standards and
very focused on practical and implementable recommendations. The report
template should include:
- Title
page and opening pages - Executive
summary - Programme
description - Role of
UNICEF, Government of Korea and other stakeholders in programme
implementation - Purpose
of Evaluation - Evaluation
criteria - Objectives
- Evaluation
design - Methodology,
including sampling strategy and methodological limitations - Stakeholder
participation - Ethical
issues - Major
findings - Analysis
of results - Case
studies of good practices - Key
Constraints - General
Conclusions - Recommendations
- Lessons
learned - Annexes
TOR, tools of data collection used
The report should be
provided in both hard copy and electronic version in English. Complete data
sets (database, filled out questionnaires, records of interviews and focus
group discussions etc.) should also be provided to MoH and UNICEF at the end of
the evaluation.
Potential
uses of the evaluation findings:This
study will serve (1) to inform policy makers on the impact of Rapid SMS-MCH on
maternal and child health (2) to make policy makers and developers aware of
areas in which Rapid SMS-MCH can be strengthened to support maternal and child
health, and (3) to inform external stakeholders on the impact of Rapid SMS-MCH
on maternal and child health.
Dissemination
of Results:Findings
of the evaluation will be summarized and discussed with the Steering Committee
Members. Findings will also be made available to individual health care
providers, health care facilities, and other relevant organizations through
scientific meetings, presentations, and publications.
8
Stakeholder Participation
Key stakeholders
include the members of the Steering Committee (The Ministry of Health,
Government of Korea and UNICEF), which will function as a Reference Group for
the evaluation and assume the following responsibilities for the evaluation:
- Plan
and design the evaluation through consultation with the main parties
involved and final approval of evaluation’s terms of reference - Provide
technical inputs to the design of the evaluation - Provide
guidelines to evaluators and monitor the evaluation implementation - Review
the evaluators’ inception report (including proposals for desk review of
documents, evaluation instruments, country visits, annotated outline of
the report) - Review
preliminary findings for validation of facts and analyses, and help
generate recommendations - Approve
the preliminary report - Review
and approve the final report, verify the evaluators’ findings and propose
management response - Ensure
that the evaluation findings are used for future planning and RapidSMS/MCH
programmatic interventions as well as advocacy purposes.
The variety of
stakeholders in the Steering Committee will ensure that different opinions are
represented and objectivity is achieved.
The Ministry of
Health will be responsible for selection of the institution to conduct the
evaluation, and the selection will be endorsed by the Steering Committee to
enrich transparency of the process and ensure neutrality/impartiality.
The evaluation will
be managed by UNICEF. The management of the evaluation will involve drafting
the terms of reference, initiating evaluation selection process, liaison
between the evaluation team and other members of the Steering Committee, as
well as quality assurance of the report.
The Ministry of
Health, with UNICEF Rwanda support, will be responsible for providing relevant
information at country level, providing access to relevant reports/statistics,
providing inputs for data analysis, organizing field visits, logistical
support, and organizing meeting with different stakeholders.
9
Existing information sources
The following
information sources are available:
- RapidSMS
Project Proposal - HMIS
- Relevant
MCH / RapidSMS Policy documents (reliable, good quality) - Narrative
reports by implementing partners (quality varies) - Field
trip reports (quality varies) - UNICEF
Annual Work Plans (reliable, good quality) - RapidSMS
CO Funding Proposals (partially valid, quality varies)
10
Ethical Consideration/confidentiality
Adequate measures
should be taken to ensure that the process responds to quality and ethical
requirements as per UNICEF Evaluation Standards. As per United Nations
Evaluation Group (UNEG) Standard and Norms, the consultants should be sensitive
to beliefs, manners and customs and act with integrity and honesty in
relationships with all stakeholders. Furthermore, consultants should protect
the anonymity and confidentiality of individual information. Consultants should
respect the confidentiality of the information which is being handled during
the assignment. Consultants are allowed to use documents and information
provided only for the tasks related to the terms of reference of this
evaluation. Data will be stored in a secure location, kept confidential with
access restricted to principal investigators. The study data will be used only
for the purpose of this study.
11
Evaluation team composition / qualifications and requirements
Evaluation
team composition
The selected
evaluation institution will be responsible for the creation of an evaluation
team. The minimum request is that the team consists of at least two experts
(one expert in quantitative research and impact evaluation, and a further
expert team members for qualitative research). The exact division of work will
be decided by the institution, but in general, the team leader will be
responsible for discussions, negotiations, final decisions, shape of the
evaluation, while further team members will be tasked with more technical
issues (revision of technical reports, in-depth interviews with service
providers, decision makers, parents, revision of existing research reports
etc.).
The qualifications
and skill areas required include:
Technical expert
& team leader:
- Extensive
quantitative research and impact evaluation expertise and experience - Academic
background in health / strong knowledge of epidemiological approaches - Familiarity
with technical aspects related to Maternal, Child and Newborn Health /
RapidSMS - Knowledgeable
on institutional issues related to the provision of global public goods
(including funding, administration, the role of the UN system,
partnerships, sustainability of activities) - Knowledge
of the areas of intervention
Qualitative research
expert:
- Extensive
qualitative evaluation expertise and experience, including data collection
skills; demonstrated skills in similar evaluations - Knowledge
of technical aspects of similar programmes - Knowledge
of the areas of intervention
All members of the
team:
- Language
proficiency: excellent writing skills in English - Advanced
university degree in related field or social science - Work
experience in different countries globally: at least 8 years of field
experience for team leader and research expert; at least 3 years of field
experience for all other team members. Experience in working with UN
agencies (desired) - Experience
in evaluations/research: knowledgeable on UN evaluation policy,
recommended by UNICEF regional or global evaluation advisors or other
senior managers, skilled in performing structured interviews and
facilitating focus group discussions - Analytical
skills: Demonstrated analytical skills related to the use of quantitative
and qualitative data for decision-making - Process
management skills: Demonstrated skills and experience in conducting and
presenting evaluations - Good
communication and advocacy skills: Ability to communicate with various
stakeholders, and to express ideas and concepts concisely and clearly in
written and oral form
Evaluators should be
sensitive to beliefs and act with integrity and respect to all stakeholders.
Evaluators should protect the anonymity and confidentiality of individual
interviewees.
Evaluation
and selection criteria of the consultancy institution:
A two stage
procedure shall be utilized in evaluating proposals, with evaluation of the technical
proposal being completed prior to any financial proposal being compared. A
70/30 assessment model for the technical and financial proposal respectively
will be adapted. Cumulative weighted average methodology will then apply in
determining the best value for money proposal.
Applications shall
therefore contain the following required documentation:
a. Technical
Proposal:Consultant
institution should prepare a proposal on the basis of the tasks and
deliverables (as per the ToR). The proposal should include approach and
methodology with detailed breakdown of inception phase, proposed scope and data
collection methodology and approach that will be used by the consultant. The
proposal shall also include a brief explanation of the data analysis and report
writing and possible dissemination plan. Draft work plan and timeline for the
evaluation should be included. The Technical Proposal shall also include
updated CVs and copies of 2 reports of previous evaluations (ideally
health-related) conducted by the consultants.
b. Financial
Proposal:Companies
that meet technical threshold of 70% will be contacted to provide financial
proposals.
12 Supervision
The evaluation will
be supervised by UNICEF CSD-Health Unit and PME jointly with MoH. The Steering
Committee will provide technical inputs to the design of the evaluation,
provide guidance to the evaluators, and monitor the evaluation implementation
process.
13
Terms and conditions
Procedures
and logistics
Evaluators are
expected to use their own hired vehicles, equipment, including computers.
UNICEF will be under no operational obligation to pay for operational costs
related to this consultancy, all costs required to operationalise this
consultancy shall be borne by the hired institutional firm and should be included
into the proposed financial proposal.
Terms of
payment
The payment will be
in three (3)
instalments as follows:
· 30% of the total
payment upon completion of the desk review, submission of inception report with
work plan and methodology, theory of change and research instruments and
protocols.
· 40% of the total
payment upon completion quantitative and qualitative data collection and
analysis, including field visits and submission of the draft final report and
PPT of the evaluation;
· The remaining 30%
will be paid upon completion of all deliverables, as per the above schedule
(validated final report of the evaluation; a set of Power Point slides (25-30
slides) with key salient features of the evaluation; and four selected topics
for publication in the form of academic articles for submission to
peer-reviewed journals).
All the deliverables
need to meet UNICEF requirement and quality standards. Payment will only made
for work satisfactorily completed and accepted by UNICEF. UNICEF reserves the
right to withhold all or a portion of payment if performance is unsatisfactory,
if work/outputs is incomplete, not delivered or for failure to meet deadlines.
All materials
developed by the firm will remain the copyright of MoH and UNICEF, who will be
free to adapt and modify the materials for future use.
[1] It may not be
technically possible to establish the Rapid SMS attribution in the country’s
reduction of child and maternal mortality /to establish causality for this
short duration of intervention at impact level. Rather, “impact” in this
context means that the evaluation will analyse and document evidence of
potential evolution towards the country’s mortality reduction. For example,
using counterfactual strategy by making a comparison between matured districts
where Rapid SMS was implemented from the very beginning and those districts
where intervention started relatively recently.
[2] See “UNICEF
Evaluation Report Standards”.
[3] See UNICEF
Technical Notes Series No 3 “Writing a Good Executive Summary”.
14.
How to apply
Qualified
institutions are requested to submit a full proposal, consisting of two parts
(technical and financial, which can be downloaded from our website) to Rwasupply@unicef.org.
Deadline for
submission is 23 October 2015 at 5:00pm.
