End-term evaluation of RNW Media’s Next Generation programme
Terms of Reference
RNW Media is looking for an external evaluator to conduct the end-term evaluation of our Next Generation programme, consisting of the SRHR (Sexual and Reproductive Health and Rights) programme and the SCIG (Social Cohesion and Inclusive Governance) programme, implemented in eight countries : These terms of reference contain:
• A brief description of our work (section 2);
• The suggested approach for the end-term evaluation (section 3);
• Available information resources and data collection (section 4);
• Governance and accountability (section 5);
• Profile of the evaluator(s) (section 6);
• Key deliverables and timeline (section 7);
• Information on submitting questions and proposals (section 8);
• Remuneration (section 9).
About RNW Media
RNW Media builds digital communities for social change. We envision a world where young people in restrictive settings confidently claim their human rights, assume their place in society, and shape a better future. We operate at the intersection of media, internet technology and international development. Our focus is on young people, between 15-30 years of age. We bring young people together in digital communities across a range of platforms—with websites, WhatsApp, Facebook, Twitter, Instagram and other channels. Through our platforms, young people access information that enables them to make informed decisions. In this alternative digital civic space, they engage in constructive dialogue from diverse viewpoints, focus on their aspirations, realise their rights and contribute to making their societies more inclusive.
The Next Generation programme, funded by the Dutch Ministry of Foreign Affairs, empowers young people to reach their aspirations, deploying digital media as a means to social (system, norms and behavioural) change. It builds young people’s (organisations’ and movements’) capacity to operate, supply and moderate digital information platforms, and build large digital communities where young people can freely and safely access trustworthy information, post blogs, deliberate and dialogue with each other, undertake research and advocacy and articulate alternative discourses. This allows them to make their own informed decisions, to challenge restrictive norms, to mobilise and engage their peers and to undertake initiatives to reshape the society around them – making it more inclusive and coherent, raise their voices and claim their gender, human and citizen’s rights.
The Next Generation programme uses a positive, aspirational approach in the MENA region, Sub-Saharan Africa and China on two themes. The first theme focuses on realising sexual and reproductive health and rights (SRHR). We focus on unmet needs relating to reproductive health and access to safe abortion, intimate partner violence, gender-based violence and LGBT rights. The second theme is Social Cohesion and Inclusive Governance and surfaces (social, cultural, political, economic, legal) needs and aspirations as well as barriers to participation such as discrimination, polarisation, corruption, violence and lack of accountability.
RNW Media has a rapidly growing online reach. In 2019 we had 10.9 million people active on our social media communities and there were 1.3 billion content views across our platforms. A testament to our ability to engage young people in discussion: there were 15.5 million social interactions.
About the ‘Citizens’ Voice’ programme
Citizens’ Voice (CV) started in 2015 and promotes inclusive governance and social cohesion with young people. The main funders of CV are the Dutch Ministry of Foreign Affairs and the EU. There are Citizens’ Voice digital platforms in DRC, Burundi, Mali, Yemen, Libya and China, respectively. Citizens’ Voice takes an aspirational approach to content and uses strategic moderation to support the transition of polarised discussions into constructive dialogue that fosters diversity of opinion. The platforms offer inclusive, safe spaces for a broad diversity of young people from across political, ethnic, gender, social, cultural or religious divides in restrictive settings. On our platforms young people can access and engage with pluralistic, reliable information and perspectives on sociocultural norms and values, youth needs and aspirations. In our digital communities they can discuss what is important to them, collectively envision the future, and connect to influential actors. For more information, see: www.rnw.org/what-we-do/citizens-voice.
About the ‘Love Matters’ and wider SRHR programme
Love Matters (LM) commenced in 2012 and supports safe, healthy and pleasurable love, sex and relationships for young people (15-30 years old). Love Matters is currently active in India, China, Egypt, Kenya, Nigeria, DRC and Mexico. In 2019, we started additional SRHR programmes in Yemen, Libya, Burundi and Mali. The SRHR programme is primarily funded by the Dutch Ministry of Foreign Affairs and Amplify Change. As an innovative online platform providing SRHR information, we operate at the intersection of (digital) media and popular culture to put the control in young people’s hands. Our platforms use a pleasure-positive and taboo-breaking approach to attract young people. They challenge prejudices and deliver accessible evidence- and rights-based SRHR information driven by young people’s interests and needs. Young people are then better able to make informed decisions and improve their access to appropriate services and commodities. For more information, see: rnw.org/what-we-do/love-matters
3. Approach and Design
Purpose of the evaluation
RNW Media wishes to conduct an end-term evaluation for the SGIC and SRHR programmes to assess programme implementation against six OECD DAC criteria: relevance, coherence, effectiveness, efficiency, impact and sustainability. Moreover, the goal of the evaluation is to capture the lessons learned with the aim to improve the implementation of future programmes.
• What is the relevance of the Theories of Change (ToCs) and results achieved in relation to the evolving context of the respective countries and needs of young people? Have we adapted fast enough to the rapidly changing (digital) environment?
• To what extent have our ToC objectives been achieved?
• What effects do our platforms have on the platform users (knowledge, attitudes and behaviour) and which factors were involved in achieving these effects?
• To what extent have we contributed to an increased response from decision makers to young people’s needs?
• To what extent have we contributed to increased public support for young people’s needs?
• To what extent has the relationship between inputs and outputs been timely, cost-effective and to expected standards?
• To what extent has this programme added value without duplicating other efforts in the same field?
• What has been the added value of partnerships with other stakeholders in the field in relation to the achievement of programmatic outcomes?
• To what extent has this programme contributed to social change (systems, norms, values)? Which factors contributed to this?
• To what extent have we contributed to young people’s ability to make informed decisions?
• To what extent are project teams self-sufficient and able to independently move forward after the end of this programme? We would particularly like the evaluation to focus on the Love Matters franchise and the Citizens’ Voice replication model.
The above evaluation questions will be further refined and elaborated jointly between the consultant(s) and RNW Media team.
Design and Methodology
We wish to conduct an in-depth evaluation of three selected project countries, supplemented by evaluation snapshots of five other project countries. Suggested countries for in-depth evaluation are Burundi and Yemen (SGIC + SRHR), and Egypt (SRHR). Final country selection will be done in consultation with the evaluator(s).
As part of the application, the consultant is invited to suggest evaluation methods proven to be well suited for the evaluation of digital campaigns and advocacy programmes, and able to capture and answer the specified evaluation questions. Final specification of evaluation design, methodology and evaluation plan will be established in close consultation with the PMEL specialists at RNW Media and the in-country teams.
4. Available information sources & data collection for the end-term evaluation
Available information sources
The consultant will be provided with relevant documents and data for the evaluation, including but not limited to:
• Grant proposal of both programmes;
• Theories of Change for both programmes;
• Measurement framework for both programmes;
• Annual and quarterly reports;
• Baseline and midterm reports.
The remaining, supplemental data should be collected in the field. Due to COVID-19, data collection through field visits may be complex or not possible at all. We welcome suggestions from the evaluator on how to optimise field data collection, given this situation. The evaluator is also expected to conduct surveys and interviews with beneficiaries and relevant stakeholders, such as RNW Media country teams, CSOs/NGOs and governmental institutions.
5. Governance and accountability
The evaluator(s) will report to the evaluation steering committee, who will:
• Further specify the strategic focus of the evaluation in cooperation with the evaluator(s);
• Make any necessary decisions related to the evaluation along the way;
• Provide feedback on (draft) deliverables of the evaluator(s);
• Approve the content and quality of the final deliverable.
Members of the evaluation project team will serve as a main point of contact for the evaluator(s), who will:
• Serve as a first point of contact and sounding board for the evaluator(s);
• Facilitate the evaluator(s), for example by linking them to other RNW Media staff;
• Monitoring progress of the evaluation and escalate to the steering committee if required;
• Pre-review any of the deliverables;
• Advise the steering committee.
6. Profile of the evaluator(s)
The following criteria will be used to shortlist potential evaluators:
• Demonstrable experience in evaluating digital media and/or advocacy programmes;
• Demonstrable experience in mixed-method research methodology;
• Demonstrable experience in managing a multi-country evaluation;
• Demonstrable experience in translating findings into actionable learnings;
• Fluency in English and French and/or Arabic.
• Thematic knowledge of SRHR and/or social cohesion and inclusive governance;
• Experience with interviewing young people in restrictive settings;
• Experience in digital marketing research and data-driven interventions;
• Relevant contextual knowledge of our project countries.
7. Key deliverables and timeline
The evaluator(s) should submit the following key deliverables:
1. An inception report in English outlining the approach, proposed methodology, an initial desk review due before the field work takes place, detailed plan of action which contains a detailed description of evaluation design and methodology, (such as the design of the methodology), including a clear division of tasks and responsibilities between the evaluator(s) and RNW Media, required information and support from RNW Media.
2. A bi-weekly progress update in which the evaluator(s) briefly summarise the work that has been done so far and how the evaluation progresses in relation to the project planning.
3. Draft evaluation report (in English) – subject to RNW Media personnel feedback.
4. A final report (in English).
5. Presentation of the report findings to RNW Media staff.
Selection and contracting of the evaluator(s): September (2020)
Start of the inception phase: October (2020)
Start of the evaluation: November (2020)
Submission of the final report: March (2021)
8. Submitting questions and proposals
Any questions about this ToR can be directed to:
Bojan Pavlovic (PMEL manager): [email protected]
Proposals should be submitted by the 24th of September to Bojan Pavlovic (PMEL manager).
Please apply HERE.
Proposals should contain at least:
• Your understanding of and reflections on the assignment;
• A description of your proposed approach;
• Any suggestions for changes in or additions to the proposed methodology;
• A detailed budget (not exceeding EUR 50,000);
• An up-to-date CV of the evaluator(s);
• A short overview of how the evaluator / evaluation team meets the required profile;
• An example piece of work from a similar assignment;
• References from other clients are desirable.
We kindly invite interested parties to include a tentative more detailed timeline in their proposal, which should at least specify most important milestones (such as end of the inception phase, submission of the draft report, et cetera). A detailed planning at activity level can be developed as part of the inception phase in cooperation with RNW Media.
The maximum budget for the evaluation is EUR 50,000, including VAT, consultant’s fee, relevant taxes, office, communication and translation-related costs associated with the study, travel costs and any other additional costs. Remuneration is based on the submission of the final deliverables.
The consultant will be in charge of logistical arrangements for the data collection and conduct of the study overall.