Mid-term evaluation of RNW Media’s Next Generation programme

  1. Introduction

RNW Media is looking for an external evaluator to conduct a mid-term evaluation of our Next Generation programme, consisting of the Citizens’ Voice and Love Matters programmes in four countries of the programmes’ implementation: These terms of reference contain:

  • A brief description of our work (section 2);
  • The suggested approach for the mid-term evaluation (section 3);
  • Available information resources and data collection (section 4);
  • Governance and accountability (section 5);
  • Profile of the evaluator(s) (section 6);
  • Key deliverables and timeline (section 7);
  • Information on submitting questions and proposals (section 8);
  • Remuneration (section 9).
  1. Background

 

About RNW Media

At RNW Media, we support young people who want to speak freely and make their own informed choices about their futures. We work with them to develop online platforms[1] that provide young people with information in areas where this information is often restricted or taboo. These online communities are safe places where they can engage with each other and have online conversations about what really matters to them – both personally and politically.

The Next Generation programme, funded by the Dutch Ministry of Foreign Affairs, empowers young people to reach their aspirations, deploying digital media as a means to social (system, norms and behavioural) change. It builds young people’s (organisations’ and movements’) capacity to operate, supply and moderate digital information platforms, and build large digital communities where young people can freely and safely access trustworthy information, post blogs, deliberate and dialogue with each other, undertake research and advocacy and articulate alternative discourses. This allows them to make their own informed decisions, to challenge restrictive norms, to mobilise and engage their peers and to undertake initiatives to reshape the society around them – making it more inclusive and coherent, raise their voices and claim their gender, human and citizen’s rights.

The Next Generation programme uses a positive, aspirational approach in two regions and China on two themes. The first theme focuses on realizing sexual and reproductive health and rights (SRHR) through the Love Matters programme. We focus on unmet needs relating to reproductive health and access to safe abortion, intimate partner violence, gender-based violence and LGBT rights. The second theme is called Social Cohesion and Inclusive Governance and surfaces (social, cultural, political, economic, legal) needs and aspirations as well as barriers to participation such as discrimination, polarization, corruption, violence and lack of accountability.

RNW Media has a rapidly growing online reach. In 2018[2] we had 10.5 million people active on our social media communities. And there were 610 million content views across our platforms. A testament to our ability to engage young people in discussion: there were 19.7 million social interactions.

About the ‘Citizens’ Voice’ programme

Citizens’ Voice (CV) started in 2015 and promotes inclusive governance and social cohesion with young people. The main funders of CV are the Dutch Ministry of Foreign Affairs and the EU. There are seven Citizens’ Voice digital platforms, in DRC, Burundi, Mali, Yemen, Libya, Syria and China, respectively. Citizens’ Voice takes an aspirational approach to content and uses strategic moderation to support transition of polarised discussions into constructive dialogue that fosters diversity of opinion. The platforms offer inclusive, safe spaces for a broad diversity of young people from across political, ethnic, gender, social, cultural or religious divides in fragile or repressive settings. On our platforms young people can access and engage with pluralistic, reliable information and perspectives on sociocultural norms and values, youth needs and aspirations.  In our digital communities they can discuss what is important to them, collectively envision the future, and connect to influential actors. For more information, see: www.rnw.org/what-we-do/citizens-voice.

About the ‘Love Matters’ programme             

Love Matters (LM) commenced in 2012 and supports safe, healthy and pleasurable love, sex and relationships for young people (15-30 years old). Love Matters currently works in India, China, Egypt, Kenya, Nigeria, Rwanda, DRC and Mexico, and is mainly funded by the Dutch Ministry of Foreign Affairs and Amplify Change. As an innovative online platform providing SRHR information, we operate at the intersection of (digital) media and popular culture to put the control in young people’s hands. Our platforms use a pleasure-positive and taboo-breaking approach to attract young people. They challenge prejudices and deliver accessible evidence- and rights-based SRHR information driven by young people’s interests and needs. Young people are then better able to make informed decisions and improve their access to appropriate services and commodities. For more information, see: www.rnw.org/what-we-do/love-matters.

 

  1. Approach and Design

 

Purpose of the evaluation

RNW Media wishes to conduct a mid-term evaluation for the CV and LM programmes to assess programme performance up to date against three OECD DAC criteria: effectiveness, relevance, impact. Moreover, the goal of the evaluation is to capture the lessons learned with the aim to improve the implementation of the programmes.

Effectiveness:

  • To what extent have our Theory of Change (ToC) objectives been achieved?
  • What effects do our platforms have on the (knowledge and attitudes and behaviour of) platform users and which factors were involved in achieving these?
  • To what extent do we contribute to (building an evidence base for) advocacy?
  • How do we contribute to an inclusive media landscape in our project countries?

Relevance:

  • What is the relevance of the ToCs and results achieved in relation to the evolving context of the respective countries and needs of young people? Do we adapt fast enough to the rapidly changing (digital) environment?
  • How can we optimise our approach to achieve increased knowledge, changing attitudes and any other relevant outcomes?

Impact:

  • To what extent do we contribute to social change (systems, norms, values)? Which factors contributed to this?
  • To what extent have the programmes contributed to young people’s ability to make informed decisions?

Programmatic successes and lessons learned up to date:

  • Which successes and lessons learned have been encountered by the both programmes, if any? Why did they occur?
  • What is the feasibility of replicating the programmatic successes and lessons learned in the future programmes of RNW Media?

 

The above evaluation questions will be further refined and elaborated jointly between the consultant(s) and RNW Media team.

Design and Methodology

We wish to conduct an in-depth evaluation of four project countries (two for Citizens’ Voice, two for Love Matters), supplemented by a brief status update on other project countries. Suggested countries for in-depth evaluation are Burundi and Yemen for Citizens’ Voice, and Egypt and China for Love Matters. Final country selection will be done in consultation with the evaluator(s).

As part of the application, the consultant is invited to suggest evaluation methods proven to be well suited for the evaluation of digital campaigns and advocacy programmes, and allowing to capture and answer the specified evaluation questions. Final specification of evaluation design, methodology and evaluation plan will be established in close consultation with the PMEL specialists at RNW Media and country teams.

  1. Available information sources & data collection for the mid-term review

 

Available information sources

The consultant will be provided with relevant documents and data for the MTR, including but not limited to:

  • Grant proposal of both programmes;
  • Theories of Change for both programmes;
  • Measurement framework for both programmes;
  • Annual and quarterly reports;
  • Baseline reports.

Data Collection

The remaining, supplemental data should be collected during selected field visits. Data collection in the field is mandatory as well as thorough study of programme documents and reports. The consultant is also expected to conduct interviews with beneficiaries and relevant stakeholders, such as RNW Media country teams, CSOs/NGOs and governmental institutions.

 

  1. Governance and accountability

 

The evaluator(s) will report to the evaluation steering committee, who will:

  • Further specify the strategic focus of the evaluation in cooperation with the evaluator(s);
  • Make any necessary decisions related to the evaluation along the way;
  • Provide feedback on (draft) deliverables of the evaluator(s);
  • Approve on the content and quality of the final deliverable.

Members of the evaluation project team will serve as a main point of contact for the evaluator(s), who will:

  • Serve as a first point of contact and sounding board for the evaluator;
  • Facilitate the evaluator, for example by linking him/her to other RNW staff;
  • Monitoring progress of the evaluation and escalate to the steering committee if required;
  • Pre-review any of the deliverables;
  • Advise the steering committee.
  1. Profile of the evaluator(s)

 

The following criteria will be used to shortlist potential evaluators:

Required:

  • Demonstrable experience in evaluating digital media or advocacy programmes;
  • Demonstrable experience in mixed-method research methodology;
  • Demonstrable experience in managing a multi-country evaluation;
  • Demonstrable experience in translating findings into actionable learnings;
  • Fluency in English and French.

Desired:

  • Thematic knowledge of SRHR, e-health/m-health, and/or social cohesion and inclusive governance;
  • Experience with interviewing young people in restrictive settings;
  • Experience in digital marketing research and data-driven interventions;
  • Relevant contextual knowledge of our project countries (in particular China).
  1. Key deliverables and timeline

Key deliverables

The evaluator should submit the following key deliverables:

  1. An inception report in English outlining the evaluator’s approach, proposed methodology, an initial desk review due before the field work takes place, detailed plan of action which contains a detailed description of evaluation design and methodology, (such as the design of the methodology), including a clear division of tasks and responsibilities between the evaluator and RNW Media, required information and support from RNW Media.
  2. A bi-weekly progress update in which the evaluator briefly summarizes the work that has been done so far and how the evaluation progresses in relation to the project planning.
  3. Draft evaluation report (in English) – subject for RNW Media personnel feedback – based on the following structure:
  1. Title page;
  2. Table of contents;
  3. List of any acronyms, tables and charts;
  4. Executive summary: Key findings, conclusions and recommendations should be part of the summary (3 pages);
  5. Introduction, including brief description of activities, objectives, and main questions and rationale of evaluation (2 pages);
  6. Methodology, including process, data collection methods, field visits, and encountered (2 pages);
  7. Findings against the OECD DAC criteria (15 pages);
  8. Lessons learnt (10 pages)
  9. Conclusions and Recommendations (4 pages)
  10. Annexes
  11. List of interviewed persons
  12. Sources of information used

III. Evidence, such as transcriptions of interviews etc.

  1. References/citations
  1. A final report (in English);
  2. Presentation of the report findings to RNW Media staff.

 

Suggested timeline

Contracting of the evaluator:                          Start of April (2019)

Start of the inception phase:                           Mid-April (2019)

Start of the evaluation:                                   End of April (2019)

Submission of the final report:                        Mid-August (2019)

 

  1. Submitting questions and proposals

Any questions about this ToR can be directed to:

Grazina Raguckaja (PMEL manager): [email protected]

 

Proposals should be submitted by 7th of April through:

Grazina Raguckaja (PMEL manager): [email protected]

 

Proposals should contain at least:

  • Your understanding of and reflections on the assignment;
  • A description of your proposed approach;
  • Any suggestions for changes in or additions to the proposed methodology;
  • A detailed budget (not exceeding EUR 50,000);
  • An up-to-date CV of the evaluator(s);
  • A short overview of how the evaluator / evaluation team meets the required profile;
  • An example piece of work from a similar assignment;
  • References from other clients are desirable.

We kindly invite interested parties to include a tentative more detailed timeline in their proposal, which should at least specify most important milestones (such as end of the inception phase, submission of the draft report, et cetera). A detailed planning at activity level can be developed as part of the inception phase in cooperation with RNW Media.

  1. Remuneration

The maximum budget for the MTR is EUR 50,000, including consultant’s fee, relevant taxes, costs for field visits, office, communication and translation-related costs associated with the study, travel costs and any other additional costs. Remuneration is based on the submission of the final deliverables.

The consultant will be in charge of travel and logistical arrangements for the data collection and conduct of the study overall.

[1] This includes our own websites and several social media platforms, depending on the country.

[2] Draft figures, final per 7/4/2019.