Develop and Implement M&E Plan

In the initial phase, although a basic M&E needs to be structured and guided by tools and a coordination system, it tends to remain flexible and simple to allow for the rapidly changing context. As the emergency stabilizes, M&E should start to become more rigorous and be guided by better defined tools.

One way of doing this is to develop a formal M&E Plan. An M&E plan is a document that defines what data will be collected, and when and how it will be collected. It establishes the system that will guide measurement of the immediate, intermediate and long-term effects of the intervention.

In the resolution phase, it becomes easier to consolidate data from information collected during the initial phase or from rapid needs assessments done to inform activities (see Unit 2). As part of the M&E plan, objectives and indicators are defined (see Unit 6) to track progress. More information on the development of an M&E plan.

The M&E plan should be developed in conjunction with the SBCC strategy, but remain a separate document. Generally, it is designed by staff with research or evaluation experience in partnership with program personnel. A complete M&E plan generally contains the seven following elements:

• Identification of the program goal and communication objectives
• Definition of indicators
• Definition of data collection methods and timeline
• Identification of roles and responsibilities
• Creation of a data analysis and reporting plan
• Creation of a dissemination plan

The program goal defines the desired outcome resulting from the SBCC program or strategy. In cases of emergencies and outbreaks, the goal is often to stop the emergency.

The communication objectives are the desired changes in behaviors or factors that affect behaviors that can contribute to achieving the program goal. Examples of factors that affect behavior include knowledge, attitudes and beliefs (e.g., self-efficacy, perceived susceptibility and perceived severity). Communication objectives should be established on the basis of the rapid needs assessment (see Unit 2: Rapid Needs Assessment) and the data collected in the initial phase of the emergency, and they should be SMART (see Unit 6: Developing Communication Objectives and Indicators). SMART objectives support the development of well-defined indicators that can be measured.

Indicators are the tools that measure any change and progress toward the behavioral communication objectives as a result of the intervention. Indicators can be used both to monitor and to evaluate the intervention, and they can be of four kinds: input, output, outcome and impact indicators. Process and output indicators are monitoring indicators that measure who the intervention is reaching and how, while outcome and impact indicators are evaluation indicators that provide information on the effects of the intervention. All four types of indicators should be included in the M&E plan. In order to develop these indicators, it is essential to have a clear operational definition.

More information on program goal, objectives and indicators, and how to formulate each, can be found in Unit 6: Developing Communication Objectives and Indicators.

Example Indicators for Each Phase of Emergency Response

Data collection methods and timelines refer to how and when information will be gathered to measure each established indicator. The methods used will depend on the type of indicator being measured and on the source of the required information. Where available, multiple data sources can be used to collect information on indicators and construct a more complete picture of how the communication response is faring. A range of different data sources can be accessed depending on the information being sought. Examples of data sources include, but are not limited to: program activity monitoring tools, health service statistics, facility logs, referral cards, focus group discussions with representatives from the intended audience groups, community-based or population-based surveys, in-depth interviews, media consumption studies, media ratings, listener-viewer discussion groups, meetings and discussions with emergency management personnel, surveillance officers, health workers and social mobilizers.

Qualitative data sources can be used to gather necessary information on the context and to help explain quantitative data found in other data sources. Unit 2 includes Worksheet 2.1: Gathering Existing Data Sources to Inform the Needs Assessment, which can be used as a starting place for teams developing lists of relevant data sources.

Once a decision has been made on the data that needs to be collected and from which sources, it is necessary to determine the frequency at which it will be collected. This will be influenced by the accessibility of the data sources, the availability of resources and the timeline of the intervention. In an emergency, monitoring data should be collected and shared with partners as frequently as possible to identify and address new issues promptly, report on progress in a timely manner and shape the overall emergency response. Information about data sources and timing of data collection for each indicator should be inserted in a table to be included in the M&E plan. The table can be printed out and shared with all partners working on the communication response so that all are informed of the data required and when it needs to be collected.

Roles and responsibilities should be agreed upon from the early planning stages. Roles and responsibilities identify who is responsible for collecting data for which indicator. It is likely to involve a mix of M&E personnel, research staff and program staff. During an emergency, when multiple partners work together, determining clear roles and responsibilities becomes even more important to ensure effective collaboration and the sharing of accurate information in a coordinated and timely fashion to monitor and inform the emergency response. Importantly, assigning roles and responsibilities should be agreed upon with partners and stakeholders who are involved in the communication response.

The data analysis and reporting plan provides details on what data will be analyzed and how the results will be presented. This defines the methods that will be used and who will be responsible for performing the analysis. During an emergency certain indicators will need to be reported regularly and frequently, such as the number of cases affected by the emergency in different geographical areas. Key information can be presented in table format with statistics for each relevant indicator and shared among all partners in the emergency coordination cell. Graphs and charts can be used to present data in a visual way and quickly see trends over time. Qualitative information about possible rumors, misinformation and at-risk behaviors is also important for shaping the communication response, and careful attention must be given to preparing this information in culturally sensitive ways.

The dissemination plan describes how and to whom the data will be disseminated. Questions to consider when establishing a dissemination plan include:

• How often will the monitoring data be shared with partners, stakeholders and audiences?
• How will the M&E data be used to inform staff and stakeholders about the progress and success of the communication response?
• How will the M&E data be used to inform the necessary adjustments to stay on track of the program goal?
• How will the M&E data be used to improve program effectiveness?

The M&E plan should include a plan both for internal dissemination among partners and for external dissemination among other stakeholders and donors. The internal dissemination plan is particularly important during an emergency as this enables monitoring of how the situation is evolving and supports collaboration and coordination of activities among partners. Especially in the initial phases of an emergency, internal dissemination of M&E information among response partners should occur at least on a weekly basis.

Some information may need to be shared more frequently, and both the frequency and in what form it is shared should be agreed upon in partnership with relevant stakeholders from the onset. As the emergency stabilizes, the frequency of information sharing may reduce; however, it should still happen on a regular basis as this can also act as an early warning system should the situation change. Post-emergency, evaluation data will be disseminated internally and externally among stakeholders and donors to highlight successes, lessons learned and promising practices.

For both internal and external dissemination, involving affected communities through a participatory approach is recommended. Keeping communities informed of progress and providing them with feedback on the successes, challenges and areas of improvement of the emergency response has several important functions, including:

• Increasing accountability and building trust among communities
• Promoting community ownership of activities and long-term changes
• Supporting community engagement
• Aiding the healing process and supporting communities to rebuild post-emergency

Once you have developed your M&E objectives, you can use the checklist in Worksheet 9.1 to assess whether they are SMART and to identify how to improve them.