In this example you will learn about how the evaluation of the Pfizer-funded, Population Services International-implemented Mobilize Against Malaria (Kenya) Initiative used facility-based cross-sectional health facility surveys. These surveys were used to evaluate whether facility-based public providers were adhering to national guidelines for rapid diagnostic testing (specifically whether positive or negative test results were being followed in the treatment of children under 5) as a result of Mobilize Against Malaria's activities.
Use this example and the links included to think about how to collect pre and post-intervention cross-sectional health facilities to evaluate provider SBCC activities.
Case Study
Population Services for Health (PSI) implemented a five-year, $15 million dollar Pfizer-funded initiative designed to improve the way artemether-lumefantrine (AL) was provided to patients in Kenya, Ghana, and Senegal clinics. In Kenya, consortium partners designed and implemented interventions to enhance in-service training, provided job aids, and facilitated community awareness activities encouraging prompt care seeking for fever among children under 5.
Study design: An evaluation of the intervention using pre- and post-intervention cross sectional health facility surveys was conducted in Bondo district in Kenya. The surveys included: audit of government health facilities, health worker structured interviews and exit interviews with caretakers of sick children below five years of age. The outcome indicators were the proportions of febrile children who had AL prescribed, AL dispensed, and four different dispensing and counseling tasks performed.
Objectives:
- Increase awareness among the community of the need that children with fever attend clinics promptly, under the banner ‘haraka upesi’ the Kiswahili meaning of prompt action
- Improve the performance of health workers at facilities, focusing on reinforcement of AL treatment for febrile children and adherence to recommended dispensing and counseling tasks in accordance with national guidelines
Sampling: All health workers attending febrile children in the survey were interviewed. All caregivers of children under 5 years with fever or a history of fever who were not referred for further hospitalization were interviewed after leaving the facility. Data was collected over two consecutive days.
Data collection and analysis:
Outcome 1: Proportion of febrile children having AL prescribed
Outcome 2: Proportion of febrile children having AL prescribed and dispensed
Outcome 3: Proportion of febrile children with AL prescribed and dispensed who had weight measured, first dose administered at the facility, advice provided on dosing schedule and advice provided on what to do in case of vomiting
Evaluation conclusions: Not all service providers were exposed to all aspects of the intervention, leading evaluators to conclude insufficient intervention coverage was achieved. Modest improvements were observed in providing AL and counseling of patients. Evaluators cautioned that in-service training and provision of job aids are likely to be insufficient to change provider behavior, calling for other features like supportive supervision.
Health worker performance in the management of paediatric fevers following in-service training and exposure to job aids in KenyaReplicate the scenario and learn
A five-year USAID-funded project involving a consortium of service delivery and SBCC partners includes improving RDT guideline compliance among service providers as key a deliverable. In-depth interviews with health workers and focus group discussions in communities have found a significant proportion of service providers do not trust RDT results and often treat febrile but malaria negative patients with ACT. This overprescription was attributed in part to patient expectations influencing service providers' judgement. Consortium partners designed and implemented a number of peer-group trainings, posted diagnosis and treatment guidelines in health facilities, and sent performance feedback and motivational SMS messages to providers. Consortium partners are asked to design an evaluation that will measure the proportion of patients with non-severe, non-malarial illness who were correctly prescribed recommended antimalarials. RDT uptake, provider adherence to results, and antibiotic prescription practices are also to be measured.
Quantitative measurement tool
Questions to answer: What proportion of febrile children under the age of 5 receive an RDT for fever? What proportion of service providers adhere to RDT test results according to national guidelines? What are caretakers' perception of the quality of services offered? What are service providers' perceptions of their facility's quality of care? What are service providers' perceived barriers to RDT protocol adherence?
Data collection method: Pre and post intervention cross sectional health facility-level service provider surveys
Conduct structured interviews with service providers at public health facilities in an intervention health district at baseline and end line. Conduct exit interviews of caregivers of children under 5 who visit health facilities in the health district chosen. Ensure pre-testing of all interview materials. Base interview on literature review of other interviews that have been used and national policies and guidelines. Data collection tools should include:
- Heath facility assessments of AL stock and availability of training materials
- Structured interviews that collect demographic information, exposure to trainings and guidelines
- Conduct exit interviews with caretakers sick children below 5 years of age. Ensure pre-testing of all interview materials with key audiences for comprehension, acceptance.
Informed written consent form must be obtained before interviews begin.
Limitations: This measurement method lacks a control group, making it impossible to examine differences in intervention and non-intervention areas. It may not be generalizable to other areas if based on specific provider needs.
SBCC Implementation Kit Step-by-step guidance on how to conduct SBCC for service providers
Prescriber and patient-oriented behavioural interventions to improve use of malaria rapid diagnostic tests in Tanzania: facility-based cluster randomised trial. A three-arm stratified cluster-randomised trial was conducted in 36 primary healthcare facilities from September 2010 to March 2012 within two rural districts in northeast Tanzania where malaria transmission has been declining. Interventions were guided by formative mixed-methods research and were introduced in phases. Prescribing staff from all facilities received standard Ministry of Health RDT training. Prescribers from facilities in the health worker (HW) and health worker-patient (HWP) arms further participated in small interactive peer-group training sessions with the HWP additionally receiving clinic posters and patient leaflets. Performance feedback and motivational mobile-phone text messaging (SMS) were added to the HW and HWP arms in later phases. The primary outcome was the proportion of patients with a non-severe, non-malarial illness incorrectly prescribed a (recommended) antimalarial. Secondary outcomes investigated RDT uptake, adherence to results, and antibiotic prescribing
Designing and implementing interventions to change clinicians’ practice in the management of uncomplicated malaria: lessons from Cameroon. A literature review combined with formative quantitative and qualitative research were carried out to determine patterns of malaria diagnosis and treatment and to understand how malaria and its treatment are enacted by clinicians. These findings were used, alongside a comprehensive review of previous interventions, to identify possible strategies for changing the behaviour of clinicians when diagnosing and treating uncomplicated malaria. These strategies were discussed with ministry of health representatives and other stakeholders. Two intervention packages - a basic and an enhanced training were outlined, together with logic model to show how each was hypothesized to increase testing for malaria, improve adherence to test results and increase appropriate use of ACT. The basic training targeted clinicians’ knowledge of malaria diagnosis, rapid diagnostic testing and malaria treatment. The enhanced training included additional modules on adapting to change, professionalism and communicating effectively. Modules were delivered using small-group work, card games, drama and role play. Interventions were piloted, adapted and trainers were trained before final implementation.