Evaluation of an audit and feedback intervention to reduce gentamicin prescription errors in newborn treatment (ReGENT) in neonatal inpatient care in Kenya: a controlled interrupted time series study protocol
Implementation Science volume 17, Article number: 32 (2022)
Medication errors are likely common in low- and middle-income countries (LMICs). In neonatal hospital care where the population with severe illness has a high mortality rate, around 14.9% of drug prescriptions have errors in LMICs settings. However, there is scant research on interventions to improve medication safety to mitigate such errors. Our objective is to improve routine neonatal care particularly focusing on effective prescribing practices with the aim of achieving reduced gentamicin medication errors.
We propose to conduct an audit and feedback (A&F) study over 12 months in 20 hospitals with 12 months of baseline data. The medical and nursing leaders on their newborn units had been organised into a network that facilitates evaluating intervention approaches for improving quality of neonatal care in these hospitals and are receiving basic feedback generated from the baseline data. In this study, the network will (1) be expanded to include all hospital pharmacists, (2) include a pharmacist-only professional WhatsApp discussion group for discussing prescription practices, and (3) support all hospitals to facilitate pharmacist-led continuous medical education seminars on prescription practices at hospital level, i.e. default intervention package. A subset of these hospitals (n = 10) will additionally (1) have an additional hospital-specific WhatsApp group for the pharmacists to discuss local performance with their local clinical team, (2) receive detailed A&F prescription error reports delivered through mobile-based dashboard, and (3) receive a PDF infographic summarising prescribing performance circulated to the clinicians through the hospital-specific WhatsApp group, i.e. an extended package.
Using interrupted time series analysis modelling changes in prescribing errors over time, coupled with process fidelity evaluation, and WhatsApp sentiment analysis, we will evaluate the success with which the A&F interventions are delivered, received, and acted upon to reduce prescribing error while exploring the extended package’s success/failure relative to the default intervention package.
If effective, these theory-informed A&F strategies that carefully consider the challenges of LMICs settings will support the improvement of medication prescribing practices with the insights gained adapted for other clinical behavioural targets of a similar nature.
PACTR, PACTR202203869312307. Registered 17th March 2022.
Improving medication safety is a global priority as medication errors arising from prescribing, dispensing, transcribing, administering, and monitoring medicines can cause severe harm and increase healthcare costs [1,2,3]. Most evidence on medication safety in routine healthcare settings is from high-income countries (HICs) . From the limited findings available, medication errors might be substantively higher in low- and middle-income countries (LMICs) [4,5,6], especially in neonatal (i.e. first 28 days of life) hospital care [7, 8] where the population with severe illness has high mortality . While around 14.9% of drug prescriptions have errors in neonatal care settings , there is scant research on interventions to improve medication safety to mitigate such errors in LMICs.
Electronic prescribing (i.e. e-Prescribing with in-built error checking) might improve neonatal care medication practices, but may not be feasible to implement in many public hospitals in LMICs due to resource constraints and level of maturity of electronic health records (EHRs) [10, 11]. Quality improvement (QI) programmes have had some success in improving clinical outcomes (i.e. 35–50% reduction in prescription errors in neonatal care), might be more feasible in many LMIC settings, and could benefit from context-appropriate audit and feedback (A&F) strategies and cycles [12,13,14,15].
Audit requires data. It may come from (1) intermittent record audit, (2) digital data on patients and prescribing, or (3) EHRs and e-prescribing which require low, moderate, and high technological capacities, respectively. Feedback is posited to reduce unsafe prescribing practices especially when it has multiple components (e.g. education), involves key agents (facilitators and champions) such as pharmacists, or addresses individual and team goals [16, 17]. The roles of key agents (facilitators and champions), for example pharmacists, for prescribing practices improvement have only rarely been empirically tested [18, 19]. As we have observed in Kenyan hospital practice, there is little interaction between clinical teams and pharmacists to guide medication prescribing with medications reconstituted and administered by nurses on the ward (e.g. gentamicin); pharmacists typically only get involved for inpatient care when potentially toxic medications (e.g. chemotherapy) are administered, but this is a rare event confined to higher level hospitals. This context provides an opportunity to evaluate including pharmacists as key agents in improving prescribing practices in neonatal care.
Additionally, sources of data are limited in many LMICs on different modes of feedback to individuals or teams. Feedback directly to clinicians on their performance is posited to be most effective , However, healthcare workers (HCWs) are few in number in many LMICs with a finite supply of time and resources to engage with feedback . The design choices in LMICs A&F studies should take account of both the specific characteristics of these contexts and existing knowledge and theory [21, 22]. To be useful, studies need to consider external validity, whether the data or technologies needed to support A&F approaches might be available at scale, what advantages might be gained by leveraging local clinical champions, and the value and practicality of feedback at team or individual levels. Their design also needs to consider that the effect of A&F interventions may diminish over time so they should aim to go beyond a single feedback cycle or short-term intervention period  and, ideally, should move beyond simple before-after designs.
In LMIC, neonatal infections are one of the most common causes of death during the neonatal period , and the increase in global prevalence of antibiotic resistance in neonatal units indicates a need for improved antimicrobial stewardship . Evidence from A&F interventions suitable for the Kenyan context could advance this agenda as well as improving patient safety. We focus on prescription accuracy for Gentamicin since:
Gentamicin is on the World Health Organization (WHO) essential medicines list  and is the first-line drug for treatment for neonatal sepsis in newborns in Kenya and LMICs and is even being used for community-based treatments [25,26,27,28].
Gentamicin is associated with well-known risks of toxicity if doses are too high for too long , while if doses are too low, it is less effective in bacterial killing. Inadequate dosing is therefore important because of the increase in antimicrobial resistance globally [25, 27, 28].
We already know that approximately 14% of the gentamicin prescription provided in the Kenyan hospitals we work with has prescribing errors, i.e. aggregate of dosages that are either too high or too low (which we explain in detail in the “Study setting” sub-section of the “Design and methodology” section), and even higher rates have been reported previously from Kenya [32, 33].
Gentamicin prescribing therefore presents a good case to study whose findings could be applied to other medications prescribed in Kenya and other settings. Penicillin is typically prescribed together with gentamicin for neonatal sepsis in Kenya and elsewhere in line with national and global guidance. It is much less likely to cause patient harm if there are prescription errors and has a much lower prescription error rate in our setting (unpublished data, further detail provided in the “Study setting” sub-section of the “Design and methodology”). For this reason, we focus on gentamicin as a priority in terms of need to intervene [8, 29, 34].
Key particularities about LMICs study settings like Kenya that make them different from HIC studies and are important considerations for A&F intervention design for reducing such prescribing errors include the following:
The lack of electronic prescribing for inpatients, thus, no automated dosage checks or decision support is available in these sites .
Junior clinical personnel do 80% of the admitting/prescribing and rotate thrice monthly  and, until recently, tend to have had limited neonatal training.
There are only one or two and sometimes no paediatricians for these hospitals , so the junior prescribers often have limited supervision, for example on ward rounds, from any specialist.
There are national guidelines that are widely disseminated to clinicians and are approximately the same as the WHO guidance that should govern prescription practices .
Routine therapeutic monitoring of gentamicin or other aminoglycoside drug levels is not available in any site.
Empiric antibiotic treatment is very common with clinicians very rarely having access to diagnostics for sepsis such as blood cultures.
More specifically, we work with 20 first-referral level hospitals organised into a Clinical Information Network (CIN) [11, 13, 39] where the hospitals receive 3-monthly clinical A&F reports on the quality of care they provide for common conditions, which include a summary of prescription error rates for gentamicin . More details are provided later on in the “Study setting” sub-section of the “Design and methodology” section. In this study, we will use a pharmacist-facilitated A&F intervention that is guided by the Clinical Performance Feedback Intervention Theory (CP-FIT) and builds on the principles of previously reported studies [16, 20, 40], in which clinical pharmacists are conceptualised as QI champions, and we anticipate they will work with doctors and nurses in their hospital’s neonatal units to act upon feedback (Table 1).
We will explicitly target feedback variables that are theorised to make feedback effective. These include the following: (1) an important clinical goal targeted by the feedback intervention, (2) using verifiable data collection and analysis methods that enhance accuracy, credibility, and acceptance of feedback, (3) employing understandable feedback displays that reinforce positive healthcare workers (HCWs) intentions and behaviours, and (4) employing feedback that targets HCWs teams’ inherent motivation to improve an important practice (Fig. 1) . To enhance feedback in some of these thematic areas and in some sites, we will use a novel electronic, interactive mobile-friendly dashboard that provides summaries of prescribing performance auto-updated monthly. The inclusion of pharmacists as key agents of A&F design and implementation is informed by CP-FIT theory , with the expectation that they serve a fundamental role of offering clinical leadership in prescribing practices [18, 19].
High-quality contemporaneously collected data on drug prescribing is needed to provide timely A&F. Investment in such systems is justified at scale if they result in better clinical practices, especially in sub-Saharan Africa (SSA)’s public healthcare systems [22, 41]. Therefore, it is important to assess the advantages of such approaches against more basic improvement or A&F strategies with fewer data demands. Conducting head-to-head experiments informed by current empirical and theoretical insights in A&F research is also a global priority [21, 41]. Head-to-head comparisons offer additional advantages. They allow more direct examination of the fidelity of delivery of different approaches, mechanisms of action, potential confounders, and effect modifiers of A&F implementation strategies. They go beyond causal description to help interpret and determine the generalisability of evaluation findings to produce transferable learning [21, 22]. When coupled with data collected over longer time periods, they can help evaluate any decay in effects of interventions .
Therefore, this study will use interrupted time series (ITS) comparison of gentamicin error rates from pre- and post-A&F intervention periods to compare enhanced A&F interventions to the usual routine feedback reports that the CIN hospitals receive (presented in documents sent to hospitals); the evaluation of the relative effect of the enhanced A&F intervention over the basic feedback reports will also include comparison of two versions of enhanced A&F intervention packages including an assessment of the implementation process of both. Effectively, the null hypotheses are that over time, (1) there is no difference in gentamicin prescribing error rates in hospitals receiving enhanced A&F compared with when they have basic A&F interventions, and any time-based changes are due to secular performance trends, and (2) there are no differences in gentamicin prescribing errors between two sets of hospitals receiving two different forms of enhanced feedback.
Given that the primary outcome of this study being measured is the proportion of patients in newborn units receiving an inaccurate gentamicin prescription over time (i.e. incidence rate ratio), our objectives are threefold:
To evaluate if enhancing A&F intervention approaches over and above existing use of feedback reports reduces the prevalence of gentamicin prescribing errors (measured as an incidence rate ratio) in neonatal inpatient hospital care over time
To evaluate if an A&F package incorporating more Clinical Performance Feedback Intervention Theory (CP-FIT) components is more effective in reducing gentamicin prescribing errors in inpatient neonatal care compared to an A&F package incorporating fewer CP-FIT informed components (which is likely to be easier to scale across facilities)
To explore the value of the CP-FIT model as a guiding framework for designing and helping understand the results of a prospective behaviour change implementation strategy employing A&F in Kenyan clinical settings
Design and methodology
The study will have a standard interrupted time-series study (ITS) design with an internal control to evaluate the comparative effectiveness of basic versus enhanced A&F after the introduction of enhanced A&F. The study design will also incorporate a parallel group controlled ITS design to compare the standard enhanced A&F package with a further extended A&F package. A process evaluation will be used to track implementation of both. Facilities will be randomised with an allocation ratio of 1:1 to receive the enhanced and extended pharmacist-delivered A&F intervention with the data-dependent components (package 1), or the enhanced pharmacist-delivered A&F intervention with standard components (package 2), which are explained further below. Participating hospitals and the clerks responsible for collecting de-identified data will be blinded to the initial group assignment, but the researchers administering the interventions and assessing the outcomes will not.
The study will be conducted in partnership with 20 first-referral level hospitals in Kenya purposefully selected to be of at least moderate size and representative of different malaria transmission zones (Table 2). This will involve the patient population admitted to the newborn unit (NBU), a separate unit with a specific clinical and nursing team, where the average age on admission is 0- or 1-day old; most admitted neonates are inborn . These hospitals joined the Clinical Information Network (CIN), a learning health system in Kenya at different calendar time points between 2014 and 2020 [11, 13, 39]. The hospitals receive 3 monthly clinical audit and feedback reports on the quality of care they provide for common conditions, which include a summary of prescription error rates for gentamicin and penicillin . Neonatal team leaders (neonatologists, paediatricians, and nurses) met face to face once or twice annually until 2020 (before the COVID-19 pandemic) to discuss these reports and how to improve clinical care. The pharmacists in these hospitals have not previously been involved in CIN feedback activities except in some hospitals linked to the “Supportive care and antibiotics for severe pneumonia among hospitalized children (SEARCH)” trial  where their role is to support correct use of study drugs used on the paediatric wards.
The primary outcome of this study is the proportion of patients in newborn units receiving an inaccurate gentamicin prescription. The calculation of the correct prescription according to the Kenyan guidelines is age and weight dependent as illustrated in Fig. 2. Gentamicin prescription is reported in milligram units. From the CP-FIT model, this outcome represents the standard of clinical performance against which clinical behaviour would be measured explicitly (goal setting) .
For the planned analysis, this study’s target process outcome is at least a 35% reduction in prescription errors from individual hospital baseline error rates with this target based on published evidence of 35–50% reductions in prescription errors in neonatal care from before to after studies .
All hospitals already receive regular standardised quarterly A&F reports. The quarterly report is shared via email and as a printed copy to the paediatrician and the hospital manager. The email report sent belongs to a hospital team whose members include the data clerks, hospital records officers, the hospital managers, and the clinical team (i.e. the nurses, medical officers, paediatricians) working in the NBU. The quarterly reports, which are described elsewhere , include summaries of the quality of the processes of care, e.g. recording of birth weight and gestational age, and whether or not basic investigations are done and their results documented . The quarterly report also already includes feedback on the correctness of dosing of commonly prescribed medications (e.g. gentamicin and penicillin).
Therefore, there is a baseline A&F intervention already in use in all sites that may be allocated to package 1 or package 2 of the enhanced A&F intervention. If participating hospitals agree, they will be randomised to receive either A&F intervention package 1 or package 2. These packages and their differences are illustrated in Table 3 and Fig. 3. Hospitals receiving package 1 act as the contemporaneous control group for those receiving package 2 to address objective 2. Data from both package 1 and 2 hospitals will be used to address objective 1 of evaluating the comparative effectiveness of enhanced A&F relative to the basic baseline A&F. We will use a control outcome within the ITS design to examine whether improvements in performance (correct prescribing) are more pronounced than improvements that may be linked to underlying secular trends . Such a control outcome should not be affected by the intervention but would be affected by confounding events . The planned control outcome is incorrect penicillin prescription in the same patient population, since 99% of neonates with gentamicin prescription also have a penicillin prescription. In contrast to gentamicin, other first-line antibiotics are considerably less likely to cause patient harm if there are prescription errors (e.g. penicillin), or from reported evidence, typically have lower prescription error rate, or are uncommon in LMICs routine hospital settings like Kenya, thereby giving gentamicin a higher priority in terms of need to intervene [8, 29, 34]. Feedback on both gentamicin and penicillin errors is still being provided throughout this study by the CIN quarterly reports, which will continue throughout the study. Specific ITS analyses will compare effects of package 2 over 1 in the parallel control group study design component.
There is currently little interaction between clinical teams and pharmacists for medication prescription such as gentamicin which are reconstituted and administered in the ward; pharmacists tend to come in when potentially toxic medications such as chemotherapy are administered and have oversight of outpatient department prescriptions. However, using CP-FIT to inform the design of the enhanced A&F intervention encouraged us to consider the potentially pivotal role pharmacists might play, as theorised by CP-FIT, by providing enhanced A&F interventions to the usual inpatient care team.
Details of the feedback components of the interactive digital application platform that is mobile-friendly and auto-updated monthly and used to deliver the enhanced A&F report summaries, together with the proposed PDF infographic, are provided in Additional file 1: Supplementary Table 1 and Supplementary Figs. 1–3.
The feedback visualisations are limited to three to reflect HCWs finite capacity to handle feedback . They aim to minimise complexity while automating active delivery and matching resources available to HCWs; we believe they target relevant elements required to influence clinical behaviour change . Because HCWs have strong beliefs regarding how care should be provided which in turn influences their interactions with feedback, the provision of a WhatsApp avenue is to help facilitate and evaluate if the perceptions, acceptance, and intentions teased from their interaction with the A&F intervention align with the observed clinical behaviours . The WhatsApp group will also include two research paediatricians to encourage discussions and raise questions.
Installation and access to the mobile-based dashboard
A member of the research team will upload the dashboard Android application (i.e. app) onto the Google Play store. Anonymised patient data from the CIN hospitals randomised to package II intervention which are already being backed up by KWTRP will be used to generate aggregate summaries to allow the dashboard to be populated; the dashboard is populated by aggregate summary data only. HCWs in the CIN hospitals randomised to package II intervention will be able to access the hospital-specific gentamicin prescription safety dashboard through their smartphone devices using their individual-specific, site-linked login credentials. The login credentials follow industry standard OAuth 2.0 authentication and security framework to mitigate against unauthorised application access; only authorised users can access the remotely populated dashboard . The user and hospital data are not stored within the Android app minimising the risk of data breach if the application is hacked. However, to minimise risk of the Android app being hacked, software engineering techniques of obfuscating the app have been employed . The dashboard smartphone application will be accessible throughout the study duration. The delivery of the dashboard on a mobile platform is meant to facilitate easier access to the A&F performance summaries. Distribution of the dashboard application will be publicised to the participating HCWs in package II facilities through the local WhatsApp groups mentioned in the Intervention section and during physical CME seminar meetings organised by the pharmacists or CIN research staff. Package I hospital clinicians will not be provided with similar login credentials.
Referral-level hospitals with (1) high-quality gentamicin prescription baseline clinical data for neonatal inpatient care and (2) a standardised A&F routine clinical improvement cycle to track prescribing practices in neonatal inpatient care are eligible for this study. To our knowledge, only facilities that have historically been involved with the CIN (Table 2) satisfy this requirement.
De-identified data from all patients admitted to the selected hospitals’ NBUs who are under the age of 28 days (i.e. neonates) who receive gentamicin drug prescription on admission are eligible for inclusion into the analysis regardless of gestational age as in Kenya prescriptions are based on weight and postnatal age rather than gestation at birth. Only the patient population with full data on age, weight, and the dose of the prescription will be included in the outcome measurement since these data are needed to measure errors. Numbers of admissions where prescribing information is inadequate to calculate dose accuracy (e.g. missing weight) will also be measured and reported. Inadequate gentamicin prescription documentation currently stands at < 1% of all prescriptions documented within CIN in the last 24 months.
In addition to pharmacists participating from all hospitals’ all HCWs rotated into (or posted in), the NBUs of the hospitals receiving the package 2 intervention are eligible for inclusion into the aspect of the study that explores comments in the pharmacists and local hospital WhatsApp group discussions respectively after due informed consent processes. Click-stream data (defined as a detailed log of how participants navigate through the Android application when using it, which typically includes the within-app pages visited, time spent on each within-app page, how they arrived on the within-app page, and where they went next) from mobile dashboard activity of the HCWs receiving enhanced A&F package 2 will also be used in additional analysis with informed consent.
We will apply restricted randomisation to achieve greater equivalence between total population size across arms and the baseline prescription error rate  and allocate half of the participating CIN hospitals to package 1 with the remainder assigned to package 2 (Fig. 2). This study’s application of restricted randomisation will ensure that facilities have similar characteristics based on the number of neonates receiving gentamicin prescription and the historical levels of facility-based gentamicin prescription errors from CIN data. Randomisation will also increase the likelihood that facility-based characteristics are balanced across the control and experimental study arms by minimising selection bias. During the intervention period and the analysis stage, the participating hospitals and clerks capturing prescribing data will be blinded to the random allocation process, but the research staff will not. Sequence generation for random allocation will use the anticlust package in R [47, 48]. Table 4 illustrates the expected randomisation and is generated from the most recent pre-intervention data.
Data collection procedures and management
Methods of collection and cleaning of data in the CIN are reported in detail elsewhere [30, 49]. In summary, clinical data for neonatal admissions to the hospitals within the CIN are captured through structured neonatal admission record (NAR) forms coupled with standard treatment sheets that are approved by the Ministry of Health. The NAR prompts the clinician with a checklist of fields including patient biodata, clinical assessment, admission and discharge diagnoses, and record of outcome (survival or death). The CIN supports one data clerk in each hospital to extract data from paper medical records, nursing charts, treatment charts, and available laboratory reports each day after a newborn’s discharge into the primary data collection tool developed in Research Electronic Data Capture (REDCap). Automated error checking happens at the point of entry by daily review, every week centrally, and both are complemented by regular external data quality assurance reviews . A minimal dataset — which is unsuitable for our planned analyses — is collected for (1) admissions during major holiday breaks, (2) admissions when the data clerk was on leave, and (3) on a random selection of records in hospitals where the workload is very high. This process is explained in detail elsewhere [49, 50].
All data will be stored in secure KEMRI-Wellcome Trust Research Programme servers with specified researchers provided password-protected access. Data held on these servers are backed up in mirror servers also within the KEMRI-Wellcome Trust Research Programme. Data on quality of care provided to the KEMRI-Wellcome Trust as part of this collaborative proposal are made available in de-identified form derived from medical records. The primary data are therefore owned by the hospitals and their counties with the Ministry of Health. Research staff will not have permission to share the data without further written approval from both the KEMRI-Wellcome Trust Data Governance Committee and the Facility, County, or Ministry of Health as appropriate to the data request.
Data analysis and statistical methods
This study involves all healthcare workers assigned to work in the NBUs of the participating hospitals. On average, there is one paediatrician, one medical officer, 3-day nurses, and 2-night nurses in a typical NBU unit in this study (Table 2). The number of clinicians working in a specific NBU varies over time based on hospital-specific staff rotation routines, county health system hiring practices, and whether medical training institutions are in session or not. The number of interns (both nurses and medical officers) remains difficult to assess.
Interrupted time series (ITS) sample size
There are two quantitatively testable hypotheses. We have adopted a sample size calculation approach that uses generalised estimating equations (GEE) specified in a form that is suited for testing both hypotheses (Additional file 2). Given the complex study design, the GEE approach specified in Additional file 2 has been used to estimate study power of the controlled ITS analysis using a simulation technique that arises naturally from the underlying data model and typically assumed by power and sample size equations. Our approach is applicable to our count outcome and ITS design, and it easily accommodates complex design features such as different and multiple treatment interventions and different site-specific cluster effects . Our analyses will apply the treated analysis principle: all data from patients fitting the inclusion criteria who receive an evaluable admission gentamicin prescription will be analysed. Data on gentamicin prescription error rates in CIN hospitals from end of 2020 to end of 2021 will be used as pre-intervention period data and from the following 12 months after the intervention introduction (i.e. from mid-2022) as the post-intervention data.
Inpatient admissions and data collection during the COVID-19 pandemic period were largely unaffected  and will therefore include these admissions in the study, but data collected during major health workforce labour strikes will be omitted. Summary statistics with discontinuity analysis will be reported for the omitted strike period data. For the ITS analysis, based on the levels of erroneous prescribing across practices in the CIN neonatal study sites, we assume a baseline risk of 14% at the pooled CIN level (pooled rate from Table 4), with the intervention posited to reduce it to 9.1% (i.e. 35% reduction). From the GEE specified in detail in Additional file 2, we estimate that with 690 patients per month across the 20 CIN hospitals, our study will have 90% power to detect this 35% reduction of prescription error with a statistical significance of 0.05. Sample size analysis at the individual hospital level revealed that 19/20 of the hospitals did not have sufficient patient numbers per month with gentamicin prescription to facilitate separate within hospital time-series analysis. Currently, the average CIN admissions to NBUs with a gentamicin prescription at admission that is likely to be eligible are 1123 patients per month (with a standard deviation of 47 admissions across the CIN hospitals).
Interrupted time series analysis
We will apply a segmented linear mixed effects model with an autoregressive covariance structure on the proposed interrupted time deries (ITS) design, specified as a “natural experiment” that accounts for the pre-intervention trends in the study outcomes . Comparison of pre-intervention to post-intervention trends of the study outcomes addresses this study’s first objective (i.e. evaluating if enhancing A&F intervention approaches over and above existing use of feedback reports reduces the prevalence of gentamicin prescribing errors in neonatal inpatient hospital care over time). Informed by previous findings , our hypothesised impact model assumes both immediate (level) and month-to-month (slope) changes following the implementation of the intervention. We anticipate observing a slope plus level change (i.e. changes in the trend in the study outcome) of between 35 and 50% reduction in gentamicin prescription errors in neonatal care based on published evidence . Given our negative binomial modelling approach (Additional file 2), the outcome for objective 1 will be reported as an incidence rate ratio.
For the second objective (i.e. To evaluate if more intense relative to less intense theory-informed A&F is effective in reducing gentamicin prescribing errors in inpatient neonatal care), the ITS regression model used to address the first research question also includes a binary covariate term for comparing the differences in study outcome trend due to the study packages, with the less-intense package as the reference category. The significance test of the coefficient for the binary covariate term linked to whether the hospital received the enhanced A&F package will serve as the hypothesis test. We provide further explanation of our analysis approach in methodological supplements in Additional file 2. Given our negative binomial modelling approach (Additional file 2), the outcome for objective 2 will also be reported as an incidence rate ratio.
Additional supplementary analyses as part of the process evaluation
As an assessment of fidelity to the study design and intervention rollout, we will also embed a simple process evaluation to check whether CMEs happened, how frequently the HCWs accessed the mobile dashboard, and the pattern of the WhatsApp messages volume after sharing of the A&F summary reports. A research team member will observe some CME meetings with the aim to visit at least 3 hospitals in each arm, identified as 2 performing less well and one performing well, and take field notes of the way in which CMEs are conducted and discussions that take place during CMEs. No audio recording will be conducted during CMEs, only note-taking by the research team.
These CME observations may be curtailed by future COVID-19 pandemic lockdown measures. While every effort will be made to collect this data in person and in line with the KEMRI and government’s guidance on site visits, where in-person data collection is not possible, 1–2 research team members will join the CMEs virtually, through either video or voice calls to observe and follow along with the CME session while taking notes. A debrief session with the pharmacist will be held before and after the virtual observations.
Click-stream data (defined as a detailed log of how participants navigate through the Android application during when using it, which typically includes the within-app pages visited, time spent on each within-app page, how they arrived on the within-app page, and where they went next) from mobile dashboard activity of the HCWs receiving enhanced A&F package II will also be analysed by the research team. This data will contain the name of the page interface, the time the HCW accessed the within-app page, the amount of time the HCW spent on the page, and the page that the HCW navigated to next. The click-stream data is limited to the intervention Android app activity.
Using messages shared on the study’s pharmacists WhatsApp group (expected to have between 8 and 12 clinicians/nurses per hospital over the 12 months), we will explore group members’ participation as the intervention progresses at the end of the study period. The messages shared will be collated, de-identified, and thematically analysed according to their source, target, timing, and content. The analysis of the shared messages on WhatsApp will explore the reception, comprehension, and acceptance of the feedback by the HCWs (Interaction, Perception, and Acceptance, respectively) and planned behavioural responses that may be attributed to the feedback (Intention and Behaviour) and any barriers to behaviour change . The use of CP-FIT will serve as a starting point for these analyses, but we will be open to identifying issues that are not captured in CP-FIT.
The analyses for objectives 1 and 2 will be conducted using R software version 4.0.2  and the NBZIMM  library. The thematic text analysis of WhatsApp messages will be done using Python software version 3.8  together with NetworkX  and NLTK  libraries.
Time frame/duration of the project
Subject to obtaining scientific and ethics approval, we presume that the study’s first face, which involves baseline data collection 12 months prior to intervention introduction, will wrap up at the end of April 2022. The intervention phase will run 12 months afterwards, with integrated analysis and report writing running concurrently. This study has been designed to be conducted mostly remotely. We do not expect any interruption in data collection and abstraction in case the country goes into lockdown again due to the ongoing COVID-19 pandemic; routine patient data collection from the CIN — the data platform for this study — has remained largely unaffected by previous rounds of lockdowns . The WhatsApp and CMEs interventional components are largely unaffected by lockdown measures and therefore require little or no contingency planning.
The analyses described in this protocol have been approved by the KEMRI’s Scientific and Ethical Review Committee (SERU #4378) and the Oxford Tropical Research Ethics Committee (OXTREC #574-21). Any future study protocol modifications require pre-approval from these committees. All facilities/individuals that agree to take part in efforts to improve neonatal care will be free to withdraw their collaboration at any time with no penalty. Individual patient consent for the de-identified data on gentamicin doses will not be required. However, informed consent from HCWs to collate and analyse their WhatsApp and mobile-dashboard click-stream data will be sought by research team (Additional file 3: Study Tools 3 a, b, and d). Participating clinicians can withdraw at any point. The results of this analysis will be shared with the Kenyan Ministry of Health and will also be submitted to peer-review publications and for presentation at international conferences.
The work proposed engages directly to improve patient care, and thus, we will be applying the results as part of the study (in the form of feedback) and its efforts to improve care on NBU. Emerging results will be shared with the counties and the MoH and key concerns highlighted as part of efforts to ensure high quality, safe care is provided in Kenyan hospitals. The findings of the current study will also be used in the development of the guidelines and policy formulation governing the use of gentamicin.
We also hope the work will develop better scalable and effective quality improvement approaches, better information systems, and improve health workers’ motivation to focus on improved neonatal outcomes. In the past work, improved tools have been adopted and implemented nationally by the national- and county-level ministries of health, and we will work with ministries, the Kenya Paediatric Association, and hospitals to promote sustained use of improved tools after the project.
All of this should enable health workers to deliver more accurate drug prescribing during clinical care. We hope the better practices will be spread by the professional associations and by formal authorities such as MoH in Kenya, while wider lessons may influence NBU care across the region.
Availability of data and materials
The datasets generated and/or analysed during the current study are not publicly available due to the primary data being owned by the hospitals and their counties with the Ministry of Health. The research staff do have permission to share the data without further written approval from both the KEMRI-Wellcome Trust Data Governance Committee and the Facility, County, or Ministry of Health as appropriate to the data request.
Requests for access to primary data from qualitative research by people other than the investigators will be submitted to the KEMRI-Wellcome Trust Research Programme data governance committee as a first step through email@example.com, who will advise on the need for additional ethical review by the KEMRI Research Ethics Committee.
Audit and feedback
Clinical Information Network
Continuous medical education
Clinical Performance Feedback Intervention Theory
Electronic health records
Interrupted time series
Kenya Paediatric Association
Low- and middle-income countries
Ministry of Health
KEMRI’s Scientific and Ethics Review Unit
World Health Organization
Donaldson LJ, et al. Medication without harm: who’s third global patient safety challenge. Lancet. 2017;389(10080):1680–1.
European Medicines Agency. Medication errors. 2015. Available from: https://www.ema.europa.eu/en/human-regulatory/post-authorisation/pharmacovigilance/medication-errors. Cited 2021 12th March.
Panagioti M, et al. Prevalence, severity, and nature of preventable patient harm across medical care settings: systematic review and meta-analysis. BMJ. 2019;366:l4185.
Mekonnen AB, et al. Adverse drug events and medication errors in African hospitals: a systematic review. Drugs Real World Outcomes. 2018;5(1):1–24.
Okello N, et al. Antibiotic prescription practices among prescribers for children under five at public health centers III and IV in Mbarara district. PLoS One. 2020;15(12):e0243868.
Baraki Z, et al. Medication administration error and contributing factors among pediatric inpatient in public hospitals of Tigray, northern Ethiopia. BMC Pediatr. 2018;18(1):1–8.
Alghamdi AA, et al. Prevalence and nature of medication errors and preventable adverse drug events in paediatric and neonatal intensive care settings: a systematic review. Drug Saf. 2019;42(12):1423–36.
Eslami K, et al. Identifying medication errors in neonatal intensive care units: a two-center study. BMC Pediatr. 2019;19(1):1–7.
IGME, U. Levels and trends in child mortality. New York: United Nations Inter-agency Group for Child Mortality Estimation (UN IGME); 2017.
Mohsin-Shaikh S, et al. The impact of electronic prescribing systems on healthcare professionals’ working practices in the hospital setting: a systematic review and narrative synthesis. BMC Health Serv Res. 2019;19(1):1–8.
English M, et al. Programme theory and linked intervention strategy for large-scale change to improve hospital care in a low and middle-income country-a study pre-protocol. Wellcome Open Res. 2020;5:265.
Leis JA, Shojania KG. A primer on PDSA: executing plan–do–study–act cycles in practice, not just in name. BMJ Qual Saf. 2017;26(7):572–7.
Irimu G, et al. Approaching quality improvement at scale: a learning health system approach in Kenya. Arch Dis Child. 2018;103(11):1013–9.
Gachau S, et al. Does audit and feedback improve the adoption of recommended practices? Evidence from a longitudinal observational study of an emerging clinical network in Kenya. BMJ Glob Health. 2017;2(4):e000468.
Ayieko P, et al. Effect of enhancing audit and feedback on uptake of childhood pneumonia treatment policy in hospitals that are part of a clinical network: a cluster randomized trial. Implement Sci. 2019;14(1):1–14.
Avery AJ, et al. A pharmacist-led information technology intervention for medication errors (PINCER): a multicentre, cluster randomised, controlled trial and cost-effectiveness analysis. Lancet. 2012;379(9823):1310–9.
Tuti T, et al. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implement Sci. 2017;12(1):1–20.
Nzinga J, McGivern G, English M. Examining clinical leadership in Kenyan public hospitals through the distributed leadership lens. Health Policy Plann. 2018;33(suppl_2):ii27–34.
Nguyen M-NR, Mosel C, Grzeskowiak LE. Interventions to reduce medication errors in neonatal care: a systematic review. Ther Adv Drug Saf. 2018;9(2):123–55.
Brown B, et al. Clinical Performance Feedback Intervention Theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci. 2019;14(1):1–25.
Grimshaw J, et al. Reinvigorating stagnant science: implementation laboratories and a meta-laboratory to efficiently advance the science of audit and feedback. BMJ Qual Saf. 2019;28(5):416–23.
Ivers NM, et al. No more ‘business as usual’with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9(1):1–8.
Ivers N, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;2012(6):CD000259. https://doi.org/10.1002/14651858.CD000259.pub3.
UNICEF. Levels and trends in child mortality 2020. UNICEF; 2021. p. 18. https://www.unicef.org/reports/levels-and-trends-child-mortality-report-2020. Retrieved on January, 2020.
Downie L, et al. Community-acquired neonatal and infant sepsis in developing countries: efficacy of WHO’s currently recommended antibiotics—systematic review and meta-analysis. Arch Dis Child. 2013;98(2):146–54.
Organization, W.H, WHO model list of essential medicines: 17th list, March 2011. 2011.
World Health Organisation. WHO recommendations on newborn health. 2017. Available from: https://apps.who.int/iris/bitstream/handle/10665/259269/WHO-MCA-17.07-eng.pdf;jsessionid=FB9C2E7AF310716433EB5CC62A34A041?sequence=1. Cited 2021 15th March.
Okomo U, et al. Aetiology of invasive bacterial infection and antimicrobial resistance in neonates in sub-Saharan Africa: a systematic review and meta-analysis in line with the STROBE-NI reporting guidelines. Lancet Infect Dis. 2019;19(11):1219–34.
Musiime GM, et al. Risk of gentamicin toxicity in neonates treated for possible severe bacterial infection in low-and middle-income countries: systematic review. Trop Med Int Health. 2015;20(12):1593–606.
Maina M, et al. Using a common data platform to facilitate audit and feedback on the quality of hospital care provided to sick newborns in Kenya. BMJ Glob Health. 2018;3(5):e001027. https://gh.bmj.com/content/3/5/e001027, https://doi.org/10.1136/bmjgh-2018-001027.
Maina M, et al. Antibiotic use in Kenyan public hospitals: prevalence, appropriateness and link to guideline availability. Int J Infect Dis. 2020;99:10–8.
Aluvaala J, et al. Assessment of neonatal care in clinical training facilities in Kenya. Arch Dis Child. 2015;100(1):42–7.
English M, et al. A randomised, controlled trial of once daily and multi-dose daily gentamicin in young Kenyan infants. Arch Dis Child. 2004;89(7):665–9.
Fuchs A, et al. Reviewing the WHO guidelines for antibiotic use for sepsis in neonates and children. Paediatr Int Child Health. 2018;38(sup1):S3–S15.
Muinga N, et al. Digital health systems in Kenyan public hospitals: a mixed-methods survey. BMC Med Inform Decis Mak. 2020;20(1):1–14.
Ogero M, et al. Examining which clinicians provide admission hospital care in a high mortality setting and their adherence to guidelines: an observational study in 13 hospitals. Arch Dis Child. 2020;105(7):648–54.
English M, et al. The paediatrician workforce and its role in addressing neonatal, child and adolescent healthcare in Kenya. Arch Dis Childh. 2020;105(10):927–31.
McKnight J, et al. Evaluating hospital performance in antibiotic stewardship to guide action at national and local levels in a lower-middle income setting. Glob Health Action. 2019;12(sup1):1761657.
Irimu G, et al. Neonatal mortality in Kenyan hospitals: a multisite, retrospective, cohort study. BMJ Glob Health. 2021;6(5):e004475.
Peek N, et al. Evaluation of a pharmacist-led actionable audit and feedback intervention for improving medication safety in UK primary care: an interrupted time series analysis. PLoS Med. 2020;17(10):e1003286.
Ivers NM, Barrett J. Using report cards and dashboards to drive quality improvement: lessons learnt and lessons still to learn. BMJ Qual Saf. 2018;27(6):417–20.
National Institutes of Health. A study to compare different antibiotics and different modes of fluid treatment for children with severe pneumonia (SEARCH). 2019. Available from: https://clinicaltrials.gov/ct2/show/NCT04041791. Cited 2022 3rd March.
Lopez Bernal J, Cummins S, Gasparrini A. The use of controls in interrupted time series studies of public health interventions. Int J Epidemiol. 2018;47(6):2082–93.
Hardt, D., The OAuth 2.0 authorization framework. 2012, RFC 6749.
Balachandran V, Tan DJ, Thing VL. Control flow obfuscation for android applications. Comput Secur. 2016;61:72–93.
R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2020.
Papenberg M, Klau GW. Using anticlustering to partition data sets into equivalent parts. Psychol Methods. 2020. https://doi.org/10.1037/met0000301.
Tuti T, et al. Innovating to enhance clinical data management using non-commercial and open source solutions across a multi-center network supporting inpatient pediatric care and research in Kenya. J Am Med Inform Assoc. 2016;23(1):184–92.
Tuti T, et al. Improving documentation of clinical care within a Clinical Information Network: an essential initial step in efforts to understand and improve care in Kenyan hospitals. BMJ Glob Health. 2016;1(1):e000028.
Arnold BF, et al. Simulation methods to estimate design power: an overview for applied research. BMC Med Res Methodol. 2011;11(1):1–10.
Wambua S, et al. The indirect impact of COVID-19 pandemic on inpatient admissions in 204 Kenyan hospitals: an interrupted time series analysis. PLoS Glob Public Health. 2021;1(11):e0000029.
Mascha EJ, Sessler DI. Segmented regression and difference-in-difference methods: assessing the impact of systemic changes in health care. Anesth Analg. 2019;129(2):618–33.
Zhang X, et al. Negative binomial mixed models for analyzing longitudinal microbiome data. Front Microbiol. 2018;9:1683.
Van Rossum G. Python programming language. In: USENIX annual technical conference; 2007.
Hagberg A, Swart P, Chult DS. Exploring network structure, dynamics, and function using NetworkX. Los Alamos: Los Alamos National Lab.(LANL); 2008.
Loper E, Bird S. Nltk: the natural language toolkit. arXiv preprint cs/0205028; 2002.
Gude WT, et al. Clinical performance comparators in audit and feedback: a review of theory and evidence. Implement Sci. 2019;14(1):1–14.
Lahitani AR, Permanasari AE, Setiawan NA. Cosine similarity to determine similarity measure: study case in online essay assessment. In: 2016 4th international conference on cyber and IT service management: IEEE; 2016.
Stevens S, et al. Analysing indicators of performance, satisfaction, or safety using empirical logit transformation. BMJ. 2016;352:i1114.
Linden A. Conducting interrupted time-series analysis for single-and multiple-group comparisons. Stata J. 2015;15(2):480–500.
Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348–55.
Irimu G, et al. Tackling health professionals’ strikes: an essential part of health system strengthening in Kenya. BMJ Glob Health. 2018;3(6):e001136.
The Clinical Information Network (CIN) group: The CIN group hospital teams who are tagged to collaborate in the network’s development, data collection, data management, implementation of audit, and feedback interventions and who will participate in this study include the following focal persons:
a. Paediatricians: Juma Vitalis, Nyumbile Bonface, Roselyne Malangachi, Christine Manyasi, Catherine Mutinda, David Kibiwott Kimutai, Rukia Aden, Caren Emadau, Elizabeth Atieno Jowi, Cecilia Muithya, Charles Nzioki, Supa Tunje, Dr. Penina Musyoka, Wagura Mwangi, Agnes Mithamo, Magdalene Kuria, Esther Njiru, Mwangi Ngina, Penina Mwangi, Rachel Inginia, Melab Musabi, Emma Namulala, Grace Ochieng, Lydia Thuranira, Felicitas Makokha, Josephine Ojigo, Beth Maina, Catherine Mutinda, Mary Waiyego, Bernadette Lusweti, Angeline Ithondeka, Julie Barasa, Meshack Liru, Elizabeth Kibaru, Alice Nkirote Nyaribari, Joyce Akuka, and Joyce Wangari
b. Nurses: Amilia Ngoda, Aggrey Nzavaye Emenwa, Patricia Nafula Wesakania, George Lipesa, Jane Mbungu, Marystella Mutenyo, Joyce Mbogho, Joan Baswetty, Ann Jambi, Josephine Aritho, Beatrice Njambi, Felisters Mucheke, Zainab Kioni, Jeniffer, Lucy Kinyua, Margaret Kethi, Alice Oguda, Salome Nashimiyu Situma, Nancy Gachaja, Loise N. Mwangi, Ruth Mwai, irginia Wangari Muruga, Nancy Mburu, Celestine Muteshi, Abigael Bwire, Salome Okisa Muyale, Naomi Situma, Faith Mueni, Hellen Mwaura, Rosemary Mututa, Caroline Lavu, Joyce Oketch, Jane Hore Olum, Orina Nyakina, Faith Njeru, Rebecca Chelimo, Margaret Wanjiku Mwaura, Ann Wambugu, Epharus Njeri Mburu, Linda Awino Tindi, Jane Akumu, Ruth Otieno, and Slessor Osok
c. Health record information officers (HRIOs): Seline Kulubi, Susan Wanjala, Pauline Njeru, Rebbecca Mukami Mbogo, John Ollongo, Samuel Soita, Judith Mirenja, Mary Nguri, Margaret Waweru, Mary Akoth Oruko, Jeska Kuya, Caroline Muthuri, Esther Muthiani, Esther Mwangi, Joseph Nganga, Benjamin Tanui, Alfred Wanjau, Judith Onsongo, Peter Muigai, Arnest Namayi, Elizabeth Kosiom, Dorcas Cherop, Faith Marete, Johanness Simiyu, Collince Danga, Arthur Otieno Oyugi, and Fredrick Keya Okoth
The Clinical Information Network (CIN) group’s monitored email address is CIN@kemri-wellcome.org, and the list can change when new paediatrician(s), nurse(s), or HRIO leave or come into the hospital.
Protocol date: 17th March 2022
Protocol version: version 2.0
This work was primarily supported by a Wellcome Trust Senior Fellowship (#097170) awarded to ME. Additional support was provided by a Wellcome Trust core grant awarded to the KEMRI-Wellcome Trust Research Programme (#092654). The funders had no role in the preparation of this report or the decision to submit for publication.
Ethics approval and consent to participate
Ethical approval was provided by the KEMRI Scientific and Ethical Review Committee (SERU 4378) and Oxford Tropical Research Ethics Committee (OXTREC 574-21). Individual patient consent for the de-identified clinical data was judged to not be required, but consent from participating hospitals and healthcare workers would be sought.
Consent for publication
This protocol is published with the permission of the Director of Kenya Medical Research Institute (KEMRI). At this stage with no protocol containing no data from any individual person, individual consent is not applicable.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Feedback components in the interactive mobile-based dashboard and PDF infographics. Supplementary Figure 1. Score card reporting deviation from explicit targets. Supplementary Figure 2. Peer comparison of Gentamicin prescription error by patient sub-groups. Supplementary Figure 3. Hospital-specific performance trends by age-groups [11, 13, 20, 30, 58, 59].
Informed Consent Forms and Tools.
About this article
Cite this article
Tuti, T., Aluvaala, J., Malla, L. et al. Evaluation of an audit and feedback intervention to reduce gentamicin prescription errors in newborn treatment (ReGENT) in neonatal inpatient care in Kenya: a controlled interrupted time series study protocol. Implementation Sci 17, 32 (2022). https://doi.org/10.1186/s13012-022-01203-w
- Audit and feedback
- Clinical guidelines
- Inappropriate prescribing
- Low- and middle-income settings