Step 9: Monitor and Evaluate

Designing an SBCC Intervention for FBP Behavior Change

All SBCC programs, including those that focus on FBPs, must include a monitoring and evaluation (M&E) component. While M&E is introduced in Step 9 of this I-kit, it is important to remember that throughout the SBCC design process, you made key decisions that are a key part of M&E. Specifically:

Step 1: Situation Analysis/Performance Needs Assessment – You identified what were the key behavioral problems that needed to be addressed and subsequently measured in your evaluation.
Step 2: Identify the Core Problem – You identified the core problem that needed to be addressed.
Step 3: Define Key Audience Segments – You identified which cadre of FBPs you would focus on in order to change behavior.
Step 4: Develop Communication Objectives – You determined which specific motivational factors you would address and developed SMART objectives to measure them.
Step 5: Determine the Key Promise and Support Points – You developed a promise telling your audience what they would receive by changing their behavior and supported this with evidence.
Step 6: Define and Prioritize Communication Approach – You determined the communication channels you would use throughout implementation, and those you would subsequently track throughout implementation.
Step 7: Match Communication Approach to Identified Barrier – You matched the communication channels to your SMART objectives.
Step 8: Develop Implementation Plan – You developed the overall implementation plan to inform both your monitoring and evaluation activities.

fbp-3 Your M&E efforts help you to compare the effects of your SBCC intervention with your program objectives and identify factors that helped or limited the program’s success. Motivation cannot be observed or measured directly and as a result, monitoring and evaluation must measure the key factors of motivation. For FBPs these are defined as: self-efficacy, perceived place in social hierarchy/status, social and gender norms, rewards, and work environment.

Developing a monitoring and evaluation plan to measure your program’s success is important. However, before developing a Monitoring and Evaluation plan for SBCC, it is important to understand the difference between Monitoring and Evaluation and the indicators they measure.

Monitoring

Monitoring tracks and measures program activities. It helps you quantify what has been done, when it has been done, how it has been done and who has been reached. Monitoring also help you identify any problems so that adjustments can be made. The indicators tracked by monitoring are called Process Indicators.

Process Indicators

Process indicators measure the extent to which SBCC activities were implemented as planned. Examples include: the number of community dialogues held, the number of job aids distributed, the number of support supervision visits conducted, the number of peer group sessions conducted, and the number of SMS messages sent to FBPs.

C-Change created guidelines on how to develop an SBCC monitoring plan (see pg. 24).

Examples of performance monitoring and routine support supervision tools:

Situation Behavior Impact (SBI) – An interactive performance monitoring and coaching technique that can be used by FBP supervisors to monitor FBP job performance.
PSI’s IPC Toolkit – Guidelines and resources to monitor IPC activities including routine monitoring for providers.
PSI’s Provider Behavior Change Toolkit on Coaching and Feedback – Tools to provide structured routine support supervision and feedback to FBPs.

Evaluation

fbp-4 Evaluation is data collected at discrete points in time to systematically investigate whether an SBCC program has brought about the desired change in an intended population or community. Evaluation enables the SBCC program to determine whether the communication strategy and activities were effective.

Evaluation requires a comparison of variables and the measurement of changes in them over time. It measures what has happened among the intended audiences as a result of program activities and allows SBCC practitioners to answer questions like:

Were the barriers to improved FBP motivation reduced by our efforts?
Did these changes improve our program success?

Evaluation indicators for SBCC typically include Output, Outcome and sometimes Impact Indicators.

Output Indicators

These indicators will measure:

Changes in the key factors of FBP motivation as defined by: levels of self-efficacy, perception in changes of social and gender norms, perception of status, and perceived changes in rewards for work and the work environment.
The extent to which these changes correlate with exposure to SBCC activities.

Example: The proportion of FBPs who feel more supported by their colleagues as a result of activities to promote emotional intelligence and peer recognition.

Outcome Indicators

Outcome indicators measure:

Changes in audiences’ behavior.
The extent to which these changes correlate with program exposure.

Example: The proportion of FBPs who participated in positive deviance groups who now provide quality HIV testing and counseling to female sex workers.

Impact Indicators

Impact indicators measure changes in health outcomes.

Example: Percent decrease in malaria cases among children under 5; percent decrease in maternal mortality; percent decrease in HIV incidence

While effective SBCC programs have the potential to contribute to health impact it may not be possible to attribute this impact entirely to SBCC. As a result, while impact indicators are defined above, most SBCC programs – including those that target FBPs – track process, output and outcome indicators.

To increase the utility of M&E data, indicators should be disaggregated to facilitate more in-depth analysis of program performance. It is recommended that indicators are also disaggregated by gender, experience level, geographic location and type of provider, etc.

Because the SBCC component of your program may be part of a larger health systems strengthening or FBP performance improvement plan, if M&E plans already exist, add appropriate outcome or impact indicators and provide input into the existing M&E plan.

C-Change has more guidelines on developing an SBCC Evaluation Plan and indicators.

Designing an SBCC Intervention for FBP Behavior Change