Step 9: Monitor and Evaluate

Designing an SBCC Intervention for CHW Behavior Change

All SBCC programs, including those that focus on CHWs, must include a monitoring and evaluation (M&E) component. While M&E is introduced in Step 9 of this I-kit, it is important to remember that throughout the SBCC design process, you made key decisions that are a key part of M&E. Specifically:

Step 1: Situation Analysis/Performance Needs Assessment – You identified what were the key behavioral problems that needed to be addressed and subsequently measured in your evaluation.
Step 2: Identify the Core Problem – You identified the core problem that needed to be addressed.
Step 3: Define Key Audience Segments – You identified which cadre of CHWs you would focus on in order to change behavior.
Step 4: Develop Communication Objectives – You determined which specific motivational factors you would address and developed SMART objectives to measure them.
Step 5: Determine the Key Promise and Support Points – You developed a promise telling your audience what they would receive by changing their behavior and supported this with evidence.
Step 6: Define and Prioritize Communication Approach – You determined the communication channels you would use throughout implementation, and those you would subsequently track throughout implementation.
Step 7: Match Communication Approach to Identified Barrier – You matched the communication channels to your SMART objectives.
Step 8: Develop Implementation Plan – You developed the overall implementation plan to inform both your monitoring and evaluation activities.

chw-1 Your M&E efforts help you to compare the effects of your SBCC intervention with your program objectives and identify factors that helped or limited the program’s success. Motivation cannot be observed or measured directly and as a result, monitoring and evaluation must measure the key factors of motivation. For CHWs these are defined as: connectedness, social status, social and gender norms, incentives and personal rewards, and personal attitudes and beliefs.

Developing a monitoring and evaluation plan to measure your program’s success is important. However, before developing a Monitoring and Evaluation plan for SBCC, it is important to understand the difference between Monitoring and Evaluation and the indicators they measure.

Monitoring

Monitoring tracks and measures program activities. It helps you quantify what has been done, when it has been done, how it has been done and who has been reached. Monitoring also help you identify any problems so that adjustments can be made. The indicators tracked by monitoring are called Process Indicators.

Process Indicators

Process indicators measure the extent to which SBCC activities were implemented as planned. Examples include: the number of community events conducted, the number of SMS messages sent to CHWs, the number of leaders met and the number of support supervision visits conducted.

C-Change created guidelines on how to develop an SBCC monitoring plan (see pg. 24).

Examples of performance monitoring and routine support supervision tools:

Situation Behavior Impact (SBI) – An interactive performance monitoring and coaching technique that can be used by CHW supervisors to monitor CHW job performance.
PSI’s IPC Toolkit – Guidelines and resources to monitor IPC activities including routine monitoring for providers and CHWs.
PSI’s Provider Behavior Change Toolkit on Coaching and Feedback – Tools to provide structured routine support supervision and feedback to health workers.

Evaluation

chw-2 Evaluation is data collected at discrete points in time to systematically investigate whether an SBCC program has brought about the desired change in an intended population or community. Evaluation enables the SBCC program to determine whether the communication strategy and activities were effective.

Evaluation requires a comparison of variables and the measurement of changes in them over time. It measures what has happened among the intended audiences as a result of program activities and allows SBCC practitioners to answer questions like:

Were the barriers to improved CHW motivation reduced by our efforts?
Did these changes improve our program success?

Evaluation indicators for SBCC typically include Output, Outcome and sometimes Impact Indicators.

Output Indicators

These indicators will measure:

Changes in the key factors of CHWs motivation as defined by: connectedness, social status and support, perception in changes of social and gender norms and perceived changes in personal needs being met.
The extent to which these changes correlate with exposure to SBCC activities.

Example: The proportion of CHWs who now feel an improved sense of connectedness to the community or health system as a result of community mobilization activities to promote the importance of CHWs’ work.

Outcome Indicators

Outcome indicators measure:

Changes in audiences’ behavior.
The extent to which these changes correlate with program exposure.

Example: The proportion of CHWs who participated in training, support supervision and coaching who now provide quality family planning counseling to young people ages 15-24.

Impact Indicators

Impact indicators measure changes in health outcomes.

Examples: The number of youth accessing modern contraceptives in the CHW’s village; Percent decrease in malaria cases among children under 5; Percent decrease in HIV incidence

While effective SBCC programs have the potential to contribute to health impact it may not be possible to attribute this impact entirely to SBCC. As a result, while impact indicators are defined above, most SBCC programs – including those that target CHWs – track process, output and outcome indicators.

To increase the utility of M&E data, indicators should be disaggregated to facilitate more in-depth analysis of program performance. It is recommended that indicators are also disaggregated by gender, experience level, geographic location and type of provider, etc.

Because the SBCC component of your program may be part of a larger health systems strengthening or CHW performance improvement plan, if M&E plans already exist, add appropriate outcome or impact indicators and provide input into the existing M&E plan.

C-Change has more guidelines on developing an SBCC Evaluation Plan and indicators.

Designing an SBCC Intervention for CHW Behavior Change