Skip to main content

Organizational contextual factors that predict success of a quality improvement collaborative approach to enhance integrated HIV-tuberculosis services: a sub-study of the Scaling up TB/HIV Integration trial

Abstract

Background

A quality improvement (QI) collaborative approach to enhancing integrated HIV-Tuberculosis (TB) services may be effective in scaling up and improving the quality of service delivery. Little is known of the role of organizational contextual factors (OCFs) in influencing the success of QI collaboratives. This study aims to determine which OCFs were associated with improvement in a QI collaborative intervention to enhance integrated HIV-TB services delivery.

Methods

This is a nested sub-study embedded in a cluster-randomized controlled trial. Sixteen nurse supervisors (clusters) overseeing 40 clinics were randomized (1:1) to receive QI training and mentorship, or standard of care support (SOC). In the QI arm, eight nurse supervisors and 20 clinics formed a “collaborative” which aimed to improve HIV-TB process indicators, namely HIV testing, TB screening, isoniazid preventive therapy (IPT) initiations, viral load testing, and antiretroviral therapy for TB patients. OCFs measured at baseline were physical infrastructure, key staff, flexibility of clinic hours, monitoring data for improvement (MDI), and leadership support. Surveys were administered to clinic staff at baseline and month 12 to assess perceptions of supportiveness of contexts for change, and clinic organization for delivering integrated HIV-TB services. Linear mixed modelling was used to test for associations between OCFs and HIV-TB process indicators.

Results

A total of 209 clinic staff participated in the study; 97 (46.4%) and 112 (53.6%) from QI and SOC arms, respectively. There were no differences between the QI and SOC arms scores achieved for physical infrastructure (78.9% vs 64.7%; p = 0.058), key staff (95.8 vs 92; p = 0.270), clinic hours (66.9 vs 65.5; p = 0.900), MDI (63.3 vs 65; p = 0.875, leadership support (46.0 vs 57.4; p = 0.265), and perceptions of supportiveness of contexts for change (76.2 vs 79.7; p = 0.128 and clinic organization for delivering integrated HIV-TB services (74.1 vs 80.1; p = 0.916). IPT initiation was the only indicator that was significantly improved in the parent study. MDI was a significantly associated with increasing IPT initiation rates [beta coefficient (β) = 0.004; p = 0.004].

Discussion

MDI is a practice that should be fostered in public health facilities to increase the likelihood of success of future QI collaboratives to improve HIV-TB service delivery.

Trial registration

Clinicaltrials.gov, NCT02654613. Registered 01 June 2015.

Peer Review reports

Background

Among high burden countries for tuberculosis (TB), South Africa ranks second highest for TB incidence rates, estimated at 615 cases per 100 000 population [1]. Fifty-eight percent of new TB cases are co-infected with HIV and mortality rates among HIV-TB co-infected cases (62 per 100 000 population) are double that of TB mono-infected cases (38 per 100 000 population) [1]. The World Health Organization’s End TB Strategy set ambitious targets to reduce TB incidence and mortality by 90% and 95%, respectively, by 2035 [2]. South Africa has a significant contribution to make in achieving these targets and addressing the HIV-TB burden is a key public health priority [3]. To this end, the South African National Department of Health treatment guidelines, recommend integrated HIV-TB services, care, and treatment as routine care [4]. Recent studies have highlighted gaps in integrated HIV-TB service delivery such as patients missed for screening and diagnosis of HIV and TB [5,6,7]; missed viral load monitoring [8]; and sub-optimal coverage of TB prevention treatment for eligible HIV patients [1].

Missed opportunities to offer HIV-TB services to patients already accessing healthcare point to health systems weaknesses at the frontline of healthcare. Quality improvement (QI) methods offer an ideal solution to improve underlying systems for HIV-TB service delivery [9]. QI collaboratives offer a potentially effective strategy to facilitate scale-up of best practices in HIV-TB service delivery [9]. While there are many adaptions of QI collaboratives, the essential components include (i) different facility teams work together to improve performance on a common health topic, led by a faculty of experts; (ii) sharing of experiences, change ideas, and best practices between clinic teams; and (iii) mentorship of clinic teams to develop and rapidly test change ideas for a given improvement aim [10]. This approach is premised on the principle that group learning accelerates the generation of change ideas and optimally utilizes experts to facilitate learning and inform best practices [10, 11].

First becoming popular in high-income countries before spreading to low- and middle-income countries, QI collaboratives are widely adopted and utilized for improvement in a multitude of health topics since their introduction over 30 years ago [10, 11]. As the strategy proliferated, concerns regarding lack of clear evidence of effectiveness, cost-effectiveness, replicability, and sustainability have been raised [10,11,12,13,14,15].

A systematic review of QI to improve antiretroviral (ART) uptake reported modest improvement with wide variations between QI collaboratives from one setting to the next; median improvement was 22% ranging from 12.8 to 29.8% [16]. Similarly, a review of 29 QI collaboratives, specifically from low- and middle-income countries, found variations in improvement; however, larger improvements were more likely when a training component was added to the QI collaborative strategy as opposed to QI collaborative alone [14]. On its own, QI collaboratives showed no to little improvement in patients’ outcomes (median effect size (MES) less than 2%); however, combined with a training component, both patients’ outcomes (MES of 111.6%) and healthcare provider practice outcomes (MES from 52.4–63.4%) improved [14].

The variation between settings suggests that what works in one setting may not work in other settings [10]. Much of the explanations for the variations is attributed to “organizational context” and the inherent differences and uniqueness of organizations, individuals, and teams from one setting to another [17]. The Promoting Action on Research Implementation in Health Services (PARIHS) framework defines “context” as the environment or setting in which people receive healthcare services, or the environment in which the proposed change is to be implemented [18, 19].

The few studies that investigated contextual factors influencing the QI outcomes, attribute variations to baseline performance (low performing indicators have a larger room for improvement) [13], simplicity of interventions [20], and clinic team characteristics such as leadership, access to resources, and clinical skills [21, 22]. In recent literature, supportiveness of organizational contexts for change is emerging as a key factor for implementing new interventions or changes [23, 24]. Given the use of experts, time away from clinics to attend collaborative meetings, and in-person mentorship activities, QI collaboratives represent a substantial investment in time and resources and have been cited as costly [12]. Understanding which and how contextual factors impact QI collaboratives is important to enhance success and sustainability of this strategy [11, 17].

The Scaling up TB/HIV Integration (SUTHI) trial tested the effectiveness of a QI collaborative approach to enhancing integrated HIV-TB services [9]. This is a sub-study of the SUTHI trial, to determine which organizational contextual factors influenced the QI intervention to improve HIV-TB services so that these factors can be strengthened in future scale-up efforts. A secondary objective was to determine if there were any major differences in organizational contextual factors (OCF) in the QI arm compared to the standard of care arm (comparator group) which may explain the differences in HIV-TB process outcomes observed in the two study arms.

Methods

Study design: The Scaling Up TB HIV trial

The design and rationale for the SUTHI trial are published elsewhere [9]. Briefly, SUTHI was a cluster-randomized trial to determine the effectiveness of QI methods in integrating HIV-TB services on mortality in TB, HIV, and HIV-TB patients [9]. Sixteen nurse supervisors (clusters) and the 40 primary healthcare (PHC) clinics under their oversight, were randomly assigned (1:1) to either a QI intervention (hereafter known as the QI arm) or to standard of care (SOC) support and supervision (hereafter known as the SOC arm). Eight nurse supervisors and their 20 clinics were assigned to the QI arm and eight nurse supervisors and their 20 clinics were assigned to the SOC arm. The study was implemented in the Ugu and King Cetshwayo Districts of KwaZulu-Natal, South Africa from 01 December 2016–31 December 2018. All study clinics were followed up for 18 months.

Study design: Organizational contextual factors nested sub-study

This is a nested sub-study of the SUTHI trial which was designed to collect data on OCFs that may influence improvement of integrated HIV-TB service delivery and explain why the QI intervention was successful or not. Parallel to the implementation of the parent study, OCFs were assessed at set study time points using surveys administered to consenting clinic staff, and study exit focus group discussions (FGDs) conducted with clinic staff from both study arms.

The intervention: The Breakthrough Series Collaborative

The SUTHI trial adopted a QI intervention structured as a Breakthrough Series Collaborative [25]. Nurse supervisors and clinics in the QI arm formed the QI collaborative. The collaborative met for three 2-day learning sessions timed at 6-month intervals. Learning sessions included coursework on the principles and practice of QI methods and interactive group-based work. Figure 1 illustrates the topics covered at each learning session. Six-month intervals allowed clinic teams time to develop and test changes ideas, and acquire best practices to present to each other. Between learning sessions, a QI nurse mentor, made in-person visits to clinics and provided QI mentorship, reinforced knowledge from learning sessions, and reviewed clinic data. The Model for Improvement was the methodological framework to identify, develop and test change ideas [26]. Rapid, plan-do-study-act cycles facilitated the development and testing of change ideas at the clinic level. QI mentorship visits were fortnightly for the first 12 months and reduced to once a month for the last six months of the study period.

Fig. 1
figure1

Timing of QI activities and data collection in the SUTHI trial. COACH, Context Assessment in Community Health; PDSA, Plan-Do-Study-Act; LS, learning session; QI, quality improvement; SOC, standard of care; TB, tuberculosis. *The standard of care arm received standard support and supervision for HIV-TB integration

The QI collaborative worked toward a single goal of improving integrated HIV-TB service delivery and focused on eight HIV-TB process indicators, namely: HIV Testing Services (including testing TB patients); TB screening; isoniazid preventative therapy (IPT) for eligible HIV patients; ART for all HIV-TB patients; cotrimoxazole therapy for HIV-TB co-infected patients; retention in care strategies; enhanced treatment adherence strategies including, viral load testing coverage; and a single integrated data management system for both HIV and TB data.

Identification of organizational contextual factors

The PARIHS framework contributed to defining and identifying key OCFs measured in this sub-study [19]. The framework proposes that successful implementation of evidence is a function of three inter-related key elements: (i) the strength of the evidence being implemented, (ii) the supportiveness of the context in which implementation is occurring, and (iii) the facilitation mechanism used to introduce change [19]. In this paper, reference to ‘organizational context’ pertains to the clinic-level where care is provided, and OCFs are the elements of organizational context that facilitate the adoption of changes.

The PARIHS framework identified key elements of a supportive organizational context, namely: physical infrastructure, human resources, leadership support, monitoring and evaluation of performance, and receptiveness of contexts to implement changes [19]. These key elements were adopted and assessed in this sub-study. In addition, we reviewed other studies that measured clinic-level factors and identified flexibility in clinic hours, and clinic-level organization and planning for integrated HIV-TB service delivery, as elements of organizational context that were relevant to this sub-study [27, 28]. In Table 1, we define each the OCFs assessed in this study.

Table 1 Definition and measurement of organizational contextual factors

Data collection tools and surveys

We searched for piloted, validated, and published measures to quantitatively assess the selected OCFs. We adopted tools appropriate for low-and middle-income countries and where no tool was available or appropriate, we designed a tool in-house. In this sub-study, three surveys were used, the Clinic Profile Tool (CPT), The Context Assessment for Community Health (COACH) survey, and the Degrees of integrated Tuberculosis and HIV services survey. Figure 1 illustrates the study time points at which each survey was administered and Table 1 shows who were involved in completing the surveys.

The Clinic Profile Tool

The Institute for Healthcare Improvement (IHI) provided a survey, routinely used in past QI projects, to assess resources at facilities and we amended the survey in collaboration with an IHI QI advisor. Amendments included using words and terms that were familiar to clinic staff in our setting and we added on items pertaining to integration of HIV and TB systems. The CPT contained several sub-scales; however, we only assessed the following: physical infrastructure, key staff availability, flexibility of clinic hours, monitoring data for improvement, and leadership support from the District Health Offices. This survey was completed jointly by a trained study staff member and the clinic facility manager and in some instances direct observation by study staff were used to confirm responses. All responses were binary, that is, either a “yes” or “no” was required. Table 1 shows the scoring method used to assess each OCF. The CPT was administered at baseline only (Fig. 1). Due to limited study resources and time, the CPT was not validated. Additional file 1 contains the full CPT.

Supportiveness of contexts for change

To assess clinic staffs’ perceptions of the supportiveness of contexts to implement changes, we used a validated survey, called the Context Assessment for Community Health (COACH) survey. Developed by Bergstrom et al. (2015), the COACH was designed to measure the extent to which nurses, physicians, midwives, and community health perceived their work environment as receptive and prepared for implementing changes [23]. We extended the administration of the COACH survey to non-clinically trained staff. The survey has eight sub-scales, namely: resources, community engagement, monitoring services for action, knowledge sources, commitment to work, work culture, leadership, and informal payment (Additional file 2). Sub-scale items are phrased as statements to which respondents could agree or disagree on a 5-point Likert-type scale;1 =Strongly Disagree and 5=Strongly Agree. The COACH survey had a Cronbach’s Alpha score of > 0.70, which is an indication that items similar to each other are highly correlated and this is reflective of a reliable tool [23] The COACH survey was administered at baseline and months 6, 12 and 18 of the study (Fig. 1).

Importantly, some sub-scales in the COACH survey overlap with the CPT (Leadership, Resources, and Monitoring data for improvement); however, the defining characteristic is that the COACH measures perceptions of clinic staff and the CPT was a relatively more objective measure where direct observation and verification of data were used.

Degree of integrated tuberculosis and HIV services

The degree to which HIV and TB services are integrated at a clinic level is a function of joint planning and coordination of different clinic teams and systems. Uyei et al. (2016) developed and validated the Degree of Integrated Tuberculosis and HIV Service Delivery tool (Additional file 3), which quantifies the extent to which respondents perceived their clinic processes and systems to be organized and prepared for offering integrated HIV/TB services (Cronbach’s alpha of > 0.70) [28]. The tool measured eight sub-scales, namely, integrated TB and ART service delivery, availability of policies and protocols, integrated TB and pre-ART service delivery, same clinicians for both TB and HIV services, TB infection control, co-operation between TB and ART staff, TB screening, and clinician awareness of patient’s co-infection status. Sub-scale items are phrased as statements to which respondents could agree or disagree on a 5-point Likert-type scale; 1=Strongly Disagree and 5=Strongly Agree. The tool was administered at baseline and months 6, 12, and 18.

HIV and TB process indicators

The parent study collected data on HIV-TB process indicators in both study arms from clinic registers and patient electronic database downloads. Monthly summary data on the number of patients that received a service (numerator) and number of patients who were eligible for a service (denominator) were collected and proportions calculated to monitor improvement for each HIV-TB process indicator.

Focus group discussions with clinic staff

Clinic staff from both arms were recruited to participate in a study exit interview. The exit interviews were conducted as FGDs and designed to assess understanding of integrated HIV-TB service delivery, describe experiences of the QI clinic staff in implementing QI methods and document any improvement efforts of the SOC clinics. The FGDs were an opportunity to collect any insights on OCFs that were missed by the surveys. A purposive sample of clinic staff were recruited based on category of staff, availability and years spent in the clinic (at least 1 year). FGDs were conducted, using a semi-structured interview guide that was developed in-house (Additional file 4).

FGDs were conducted primarily in isiZulu and voice recorded. All participating clinic staff provided signed consent. Voice recordings were transcribed verbatim and then translated to English for analyses. Two study staff read the transcripts separately and extracted themes, including any barriers or facilitators to implementing QI or HIV-TB service integration. Themes were compared and common themes adopted. Direct quotes that supported a theme were highlighted.

Eleven FGDs involving 43 clinic staff were conducted. Six FGDs with an average of three participants each were from the QI arm and five FGDs with an average of four participants were from the SOC arm. In the QI arm, there were 16 female and four male participants and the mean number of years served in the clinic was 5.5 years (min-max: 1–15). In the SOC arm there were 18 female and three male participants and the mean number of years served in the clinic was 6.8 years (min-max:1–16).

Recruitment of clinic staff

Participation in the surveys and FGDs were offered to professional nurses, enrolled nurses, lay counsellors, and data capturers. Written consent and at least 1 year of full-time employment were the minimum criteria. At baseline, we approached clinic staff in both the QI and SOC arms and gauged their interest for participation in the surveys once every 6 months. It was neither practical nor possible to administer the surveys to all clinic staff, hence, we recruited one team member from each staff category. During the study, we attempted to administer the survey to the same team member; however, work demands, time constraints, vacation leave, and absenteeism made this impossible. If the team member was not available, that individual was replaced with another team member from the same staff category in the same clinic. All surveys and FGDs were conducted in private spaces within the clinic.

Data collection and management

Between 01 December 2016 to 1 June 2017, clinic infrastructure data were collected from all 40 study clinics. Surveys were paper-based and devoid of any identifiers that could link responses to a clinic staff member. All completed surveys were faxed to the study offices and electronically captured.

Statistical analysis

The COACH survey and Degrees of Integrated TB and HIV services survey were used to develop a score for supportive contexts for change and the extent to which clinic teams were organized to offer integrated HIV and TB services, respectively. Both surveys were scored as follows : sub-scale means were calculated by adding up all responses and dividing by the number of items in that sub-scale. A total score was calculated by adding all sub-scale means and dividing by the number of sub-scales. A clinic’s score was calculated as the mean of all clinic staff who completed the survey. A cluster mean was calculated as the mean of clinic means in that cluster and finally, the study arm mean was the mean of all cluster means. The highest possible mean for both surveys was five. Means were converted to percentages by dividing by 5 and multiplying by 100. This was done to make survey scores standardised with other scores. If a survey question was missed by the researcher, a score for that question was replaced by the mean of all other items in that sub-scale.

Responses to items in the CPT were ‘Yes’ or ‘No’ responses and coded as a one or zero, respectively. As per Table 1, mean scores for physical infrastructure, staffing availability, flexibility of clinic hours, monitoring data for improvement and leadership support, were calculated for each clinic by adding all items in the sub-scale and dividing by the number of items in that sub-scale. The mean cluster score was the mean of all clinic scores in that cluster. The study arm score was the mean of the cluster score means.

A t-test was used to compare scores between the QI and SOC arms. We compared baseline and month 12 scores for the COACH and Degrees of Integrated TB and HIV services as the QI intervention was at its full strength during this period. Linear mixed modelling was conducted to determine which OCFs best predicted improvements for each HIV-TB process indicator. Each OCF were analysed separately in the model adjusted for study arm, time, and the interaction of study arm and time. The model assumed an exchangeable covariance and time was nested within the cluster for HIV-TB process indicators. The statistical software used was STATA, version 15.1.

Ethics approval

The study was approved by the University of KwaZulu-Natal Biomedical Research Ethics Committee (BF 108/14). All clinic staff who agreed to complete a survey or who participated in FGDs, signed an informed consent form in English or isiZulu.

Results

Across the 40 study clinics, a total of 461 clinic staff were available for this sub-study and 209 (45.3%) completed at least one survey (Fig. 2). Of the 209 clinic staff, 97 (46.4%) and 112 53.6%) were from the QI and SOC arms, respectively (Table 2). In the QI and SOC arm, 51.5% (50/97) and 54.5% (61/112) of respondents were nurses (Table 2). Most respondents (>80%) were female. The mean years of experience was 8.8 years [standard deviation (SD)=4.4] and 8.4 years (SD=5.4) in the QI and SOC arms, respectively.

Fig. 2
figure2

Clinic staff categories that responded to surveys at baseline and months 6, 12, and 18. QI, quality improvement; SOC, standard of care; PHC, primary health care

Table 2 Characteristics of healthcare workers who participated in the study

A comparison between QI clinics and SOC clinics, showed similar access to basic services and staffing (Table 3). The QI arm had more high-volume clinics than the SOC group (14% versus (vs) 11 %). The mean monthly headcount in high-volume clinics were similar in both arms (Table 3).

Table 3 Clinic characteristics of the quality improvement arm and standard of care arm clinics

Differences in Integrated HIV-TB service delivery performance

The parent study evaluated improvement in HIV-TB process indicators in the QI arm at baseline and post QI intervention (defined as months 13-18) [29]. Of the eight HIV-TB process indicators, we were unable to intervene on and analyze cotrimoxazole therapy and retention in care for HIV-TB patients, due to large amounts of missing data and limited study time and funds. An integrated patient electronic database was implemented in both study arms. Supplementary Figures 1 (A-F), shows the proportions achieved at baseline and post-QI intervention in the QI and SOC arms. In the QI group, IPT initiation rates improved by 60.5%, (Supplementary Figure 1D) [29]. In comparison the SOC arm improved by 23.1%. Modest improvements are noted in the QI and SOC for HIV testing services (9.7% versus 2.9%), HIV testing services in TB patients (7.6% versus 9.2%), TB screening (9.0% versus 7.7%) and viral load testing (10.8% versus 15.3%).

Comparison of organizational contextual factors in QI and SOC arms

The mean scores achieved for OCFs measured in the QI and SOC arms are compared in Table 4. There were no OCF scores that were statistically significantly different between the QI and SOC arms. The largest difference in scores was observed in Physical Infrastructure which was 78.9% and 64.7% in the QI and SOC arms respectively; p = 0.058. The QI arm achieved a score of 46% for Leadership support visits versus 57.4% scored in the SOC arm; p = 0.265. The QI and SOC groups scored similarly in monitoring data for improvement (63.3% vs 65%; p = 0.875); however, both groups demonstrated a very wide range in scores, with some clinics scoring 100% in both groups.

Table 4 Comparison of organizational contextual factor (OCF) scores between QI and SOC groups

The QI and SOC arms achieved scores of 77.5% and 79.0%, respectively at baseline, on the COACH survey (Table 4). After 12 months in the study, the QI and SOC arms scored 76.2% versus 79.7%, respectively; p = 0.128. After scoring the Degrees of integrated HIV-TB service delivery survey, the QI and SOC arm scored 77.1% and 76.1% respectively, at baseline. After 12 months in the study, QI and SOC arm, scored 74.1% and 80.1% respectively, p = 0.916.

Organizational contextual factors associated with IPT initiation rates

While improvements were noted in HIV testing, TB screening and viral load monitoring, regression analyses were not possible in these indicators due to the small improvements made and the regression models did not converge. We used IPT initiation rates as the outcome variable in our regression analyses. Table 5 shows the bi-variate linear mixed modelling that tested for associations between each OCF and IPT initiation rates adjusted for time, study group and the interaction between study group and time. MDI was significantly associated with increasing IPT initiation rates (β = 0.04; p = 0.004). All other OCFs showed no statistically significant association with IPT initiation rates. In every bi-variate linear mixed model, the interaction of study group and time was significantly associated with increasing IPT initiation rates, suggesting that exposure to QI over time is predictive of increasing IPT performance irrespective of the influence of the OCF (β = 0.012; p = 0.004).

Table 5 Linear mixed models testing associations between organizational contextual factors and isoniazid preventive therapy

Clinic staff reflections on integrated HIV-TB service delivery and improvement activities

Barriers and facilitators to integrated HIV-TB service delivery extracted from the FGDs were related to (i) Understanding of what constitutes HIV-TB services, (ii) Awareness of gaps in HIV-TB service delivery (iii) Motivation to make improvements.

Understanding of integrated HIV-TB services

Understanding of integrated HIV-TB service delivery was similar in both study arms, with one exception, the mention of IPT to prevent TB. Focus group participants in both arms emphasized testing and screening for both diseases at the same clinic visit, linkage to TB and HIV treatment, and a single file system. Nurses in the QI clinics provided more comprehensive definitions of what it means to offer integrated HIV-TB services.

Coinfected patients should have one file for both TB/HIV. A person infected with HIV only should be screened for TB every visit. A person infected with TB only should be screened for HIV every 3 months. A person with both TB/HIV should be initiated to cotrimoxazole. Those that do not have TB but have HIV should be on INH to be prevented from contracting TB. (QI group, nurse)

Awareness of service delivery gaps

Lack of monitoring and evaluation of the IPT programme emerged as a possible reason for the low baseline IPT initiation rates in the QI clinics. Clinic staff in the QI arm reported being unaware that IPT initiation rates were low until it was highlighted during QI activities and the data was revealed to them. When asked to comment on how QI has improved HIV-TB integration, the QI group (without being prompted about IPT initiation) expressed how the QI highlighted IPT initiation and performance.

...things like IPT, IPT coverage, initiating IPT within 28 days of ART and all of that, you do not realize it is a problem until you start plotting and seeing what is happening. It also has helped to see staff performance (QI group, Professional nurse)

In the SOC clinics, three nurses reported receiving regular feedback from the District Health Offices and facility managers, on service delivery gaps.

Motivation to make improvements

In the QI clinics, interviewees mentioned several facilitators to making improvements in their clinic, including, a sense of shared responsibility for improvement efforts, clarity and transparency of individual roles and responsibilities, healthy competition, and benchmarking with other clinics in the collaborative. According to two nurses the QI trainings were too few learning sessions and limited to a small number of attendees which was a barrier to improvement in some clinics. Transfer of knowledge from learning session attendees to non-attendees was described as vague and incomplete which may have led to some clinic staff feeling ‘distanced’ from the QI intervention.

SOC clinics reported having access to resources for improvement, such as file audit templates, and access to expertise from local non-governmental organizations for data analysis, and development of performance charts. However, a lack of formal training and in-house experience in implementing improvement were barriers mentioned.

Discussion

In the SUTHI trial, IPT initiation rates were dramatically improved in the QI arm compared to the SOC arm (Supplementary Figure 1D). After testing several OCFs for association with improvement in IPT initiation rates, we found that MDI and exposure to QI over time were the only factors significantly associated with increasing IPT initiation rates. Importantly, in this study MDI was a factor that was measured at baseline (before QI intervention implementation). In South Africa, an electronic health information system was designed for the purpose of collecting and analysing patient and process data and evaluating the HIV and TB programme for effectiveness. The practice of MDI is important for data-informed decision-making regarding the direction and effectiveness healthcare services and shows commitment to improving services to communities by clinic teams [30]. The range in MDI scores for the QI and SOC arms shows that all clinics were, to varying degrees, using routine data to monitor and improve the HIV-TB programme (Table 4). This suggests that the QI intervention was implemented in a context where the practice of using data for monitoring programme performance was already embedded and may have contributed to the success of the QI intervention in improving IPT initiation rates. ‘Monitoring services for action’ was a sub-scale of the COACH tool (Supplementary Table 1) and the high scores achieved by both study arms at baseline and even after month 12, supports this finding that the study setting had a culture of data use for improvement.

Although statistically significant, we acknowledge that the association between MDI and IPT initiation rates is very weak (low beta coefficient). IPT initiation rates improved by small increments every month. The small monthly differences in improvement and inclusion of several factors (study arm, time, interaction of study arm, and time) in the model produced low beta coefficients.

The importance of MDI is highlighted in other studies. Two systematic reviews that aimed to extract OCFs which predict outcomes in QI interventions, also identified the practice of MDI as key in influencing success of QI interventions [17, 31]. A South African-based study that adopted the Breakthrough Series Collaborative, reduced HIV transmission from mothers to infants from 7.6 to 5.0% in one sub-district [32]. The researchers partially attributed this success to an existing culture of using routine data to reflect on clinic performance which facilitated the adoption of QI and was familiar to front-line staff [32]. Access to good quality routine data that is relevant to front-line staff was a further driver of uptake of the intervention that led to a positive outcome [32].

The low IPT initiation rates at baseline suggest that this indicator was not being monitored or if it was, little was done to improve performance. The FGDs confirmed that the poor performance went undetected until the QI intervention began and IPT initiation rates were presented to clinics. QI interventions to improve IPT initiation rates have been successful in other countries. A national QI programme in Namibia improved IPT by 16 to 28% [33]. In comparison, a Nigerian study made a larger improvement in IPT (11% to 50%); however, their efforts were focused at one busy facility [34]. Both studies attribute this success to QI interventions building skills and confidence among clinic teams to make improvements.

In our study, the FGDs also confirm that QI clinics felt a positive shift in team motivation, in addition, there were other contextual factors that may have influenced the uptake of QI in the study. We observed that at baseline and at month 12, the QI and SOC arms achieved high scores on the degree of integrated HIV-TB services survey, which suggests that clinic teams are well coordinated and prepared to offer integrated services. The high ART initiation rates (> 90%) among co-infected patients (Supplementary Figure 1E) support this finding. The implementation of the integrated HIV and TB electronic data system is an indication of the commitment of the South African Department of Health to HIV-TB integration. Similarly, clinic teams in both study arms perceived high levels of supportiveness (high COACH scores) in their clinic to implement changes and this persisted at month 12 in the study. Given that QI clinics showed high levels of organization to offer integrated services and felt supported to make changes in their clinics, the QI intervention thrived in these clinics, particularly when poor performance was detected.

There were no significant differences in any OCF scores between QI and SOC arms. SOC arm clinics were similar to QI arm clinics for perceived organization to offer HIV-TB integrated services and supportiveness of contexts for change. The FGDs suggest that SOC clinics only lacked improvement “know how”. This is promising for any future scale-up of the QI intervention in this context which appears to have the correct conditions to embed a successful QI programme.

Recommendations

Little is known of how best to foster the practice of MDI among clinic teams. Very few systematic reviews and intervention studies have been conducted on this topic [30]. Based on our findings and a small number of studies and systematic reviews that have been conducted, we recommend promoting the practice of MDI through making routine data accessible to clinic staff, ensuring good quality data, and improving the technical skills of clinic staff to use and generate reports from electronic health information systems. A Nigerian study tested the QI collaborative approach in enhancing prevention of mother-to-child services and included data quality as a key indicator for improvement [35]. As data quality improved, clinic teams reported increased levels of confidence in their clinic data and the use of improvement cycles using routine data [35]. Two studies demonstrated that electronic health information management systems are effective in assisting clinic teams and managers in making decisions about health programmes [36, 37]. Effectiveness studies of electronic information systems show that clinic teams will use data from electronic systems provided that the quality of data is accurate and reliable, reports are easily generated, skills and capacity to use the system is present, and no major hardware and software malfunctions occur [36, 37].

In addition, we found rapid assessments of organizational context using structured surveys useful to understanding the setting in which our QI intervention was implemented, and future QI initiatives should consider this approach and add to the knowledge base of how OCFs influence the success of QI.

Limitations

The study has several limitations. The accuracy of data collected on surveys, such as the COACH survey, cannot be guaranteed. Social desirability bias may have influenced some responses particularly those of a sensitive nature, such as leadership and commitment to work. Two studies which tested the reliability of the COACH survey reported similar challenges of eliciting truthful responses and strongly recommend that confidentiality and privacy of data be emphasized to respondents [38, 39]. Despite assuring respondents’ confidentiality and anonymity, we received reports from study staff of hesitation among respondents to select answers that may reflect poorly on themselves, leaders, and the clinic team. Thus, COACH scores in this study may be inflated. In addition, we extended the use of the COACH survey to data capturers and lay counsellors, who may not have had some knowledge, such as clinic access to medication.

Using the validated measures repeatedly may not have been the ideal method to engage clinic staff. There were reports of “fatigue” among respondents regarding the time it takes to complete the surveys and being asked the same questions. The small sample size of 16 clusters restricted and affected the analyses. We were unable to perform regression models for each study arm. Secondly, the CPT was not a validated tool and the scoring system was developed by SG and CC. Future studies should consider development of a validated measure to assess aspects of physical infrastructure and resources in low- and middle-income countries. Thirdly, all OCF scores were at the cluster level and therefore highly summarized.

Conclusion

This study has shown that QI interventions are successful in contexts where clinic teams are encouraged and supported to use routine data for improvement. IPT is an important intervention in interrupting the transmission of TB and is seldom prioritized for improvement. Capacitating clinic teams with QI skills and tools, fostering the practice of using routine data to monitor improvement, and removing any threats to using routine data may be the key to improving IPT initiations and other poorly performing indicators.

Availability of data and materials

Individual participant data for completed studies is available on requests through the CAPRISA website; after approval of a proposal, data can be shared through a secure online platform.

Abbreviations

ART:

Antiretroviral Therapy

COACH:

Context Assessment for Community Health

FGD:

Focus Group Discussions

IHI:

Institute for Healthcare Improvement

IPT:

Isoniazid Preventive Therapy

MDI:

Monitoring data for improvement

MES:

Median Effect Size

OCF:

Organizational Contextual Factors

PARIHS:

Promoting Action on Research Implementation in Health Services

PHC:

Primary Healthcare

QI:

Quality Improvement

SD:

Standard Deviation

SOC:

Standard of Care

SUTHI:

Scaling up TB HIV Integration

TB:

Tuberculosis

TPT:

TB Preventive Therapy

References

  1. 1.

    World Health Organization. Global tuberculosis report 2020. Geneva: World Health Organization; 2020. CC BY-NC-SA 3.0 IGO; 2020

    Google Scholar 

  2. 2.

    World Health Organization. WHO policy on collaborative TB/HIV activities. Guidelines for national programmes and other stakeholders. Geneva: WHO; 2012.

    Google Scholar 

  3. 3.

    Sinai I, Cleghorn F, Kinkel HF. Improving management of tuberculosis in people living with HIV in South Africa through integration of HIV and tuberculosis services: a proof of concept study. BMC Health Serv Res. 2018;18(1):711. https://doi.org/10.1186/s12913-018-3524-9.

    Article  PubMed  PubMed Central  Google Scholar 

  4. 4.

    South African National Department of Health. National consolidated guidelines: for the prevention of mother-to-child transmission of HIV (PMTCT) and the management of HIV in children, adolescents and adults. 2020. https://sahivsoc.org/Files/National%20Consolidated%20Guidelines%2030062020%20signed%20PRINT%20v7.pdf.

  5. 5.

    Chihota VN, Ginindza S, McCarthy K, Grant AD, Churchyard G, Fielding K. Missed opportunities for TB investigation in primary care clinics in South Africa: experience from the XTEND trial. PloS One. 2015;10(9):e0138149. https://doi.org/10.1371/journal.pone.0138149.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Kweza PF, Van Schalkwyk C, Abraham N, Uys M, Claassens MM, Medina-Marino A. Estimating the magnitude of pulmonary tuberculosis patients missed by primary health care clinics in South Africa. Int J Tuberc Lung Dis. 2018;22(3):264–72. https://doi.org/10.5588/ijtld.17.0491.

    CAS  Article  PubMed  Google Scholar 

  7. 7.

    Massyn N, Peer N, English R, Padarath A, Barron P, Day C, editors. District health barometer 2015/2016. Durban Health Systems Trust; 2016.

    Google Scholar 

  8. 8.

    Sunpath H, Hatlen TJ, Naidu KK, Msimango P, Adams RN, Moosa MS, et al. Targeting the third ʻ90ʼ: introducing the viral load champion. Public Health Action. 2018;8(4):225–31. https://doi.org/10.5588/pha.18.0063.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  9. 9.

    Naidoo K, Gengiah S, Yende-Zuma N, Padayatchi N, Barker P, Nunn A, et al. Addressing challenges in scaling up TB and HIV treatment integration in rural primary healthcare clinics in South Africa (SUTHI): a cluster randomized controlled trial protocol. Implement Sci. 2017;12(1):129. https://doi.org/10.1186/s13012-017-0661-1.

    Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Wells S, Tamir O, Gray J, Naidoo D, Bekhit M, Goldmann D. Are quality improvement collaboratives effective? A systematic review. BMJ Qual Saf. 2018;27(3):226–40. https://doi.org/10.1136/bmjqs-2017-006926.

    Article  PubMed  Google Scholar 

  11. 11.

    Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol RP. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ. 2008;336(7659):1491–4. https://doi.org/10.1136/bmj.39570.749884.BE.

    Article  PubMed  PubMed Central  Google Scholar 

  12. 12.

    de la Perrelle L, Radisic G, Cations M, Kaambwa B, Barbery G, Laver K. Costs and economic evaluations of quality improvement collaboratives in healthcare: a systematic review. BMC Health Serv Res. 2020;20(1):155. https://doi.org/10.1186/s12913-020-4981-5.

    Article  PubMed  PubMed Central  Google Scholar 

  13. 13.

    Franco LM, Marquez L. Effectiveness of collaborative improvement: evidence from 27 applications in 12 less-developed and middle-income countries. BMJ Qual Saf. 2011;20(8):658–65. https://doi.org/10.1136/bmjqs.2010.044388.

    Article  PubMed  Google Scholar 

  14. 14.

    Garcia-Elorrio E, Rowe SY, Teijeiro ME, Ciapponi A, Rowe AK. The effectiveness of the quality improvement collaborative strategy in low- and middle-income countries: a systematic review and meta-analysis. PLoS One. 2019;14(10):e0221919. https://doi.org/10.1371/journal.pone.0221919.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  15. 15.

    Zamboni K, Baker U, Tyagi M, Schellenberg J, Hill Z, Hanson C. How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review. Implement Sci. 2020;15(1):27. https://doi.org/10.1186/s13012-020-0978-z.

    Article  PubMed  PubMed Central  Google Scholar 

  16. 16.

    Hargreaves S, Rustage K, Nellums LB, Bardfield JE, Agins B, Barker P, et al. Do quality improvement initiatives improve outcomes for patients in antiretroviral programs in low- and middle-income countries? A systematic review. J Acquir Immune Defic Syndr. 2019;81(5):487–96. https://doi.org/10.1097/QAI.0000000000002085.

    Article  PubMed  PubMed Central  Google Scholar 

  17. 17.

    Coles E, Anderson J, Maxwell M, Harris FM, Gray NM, Milner G, et al. The influence of contextual factors on healthcare quality improvement initiatives: a realist review. Syst Rev. 2020;9(1):94. https://doi.org/10.1186/s13643-020-01344-3.

    Article  PubMed  PubMed Central  Google Scholar 

  18. 18.

    Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998;7(3):149–58. https://doi.org/10.1136/qshc.7.3.149.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  19. 19.

    Rycroft-Malone J. The PARIHS framework--a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual. 2004;19(4):297–304. https://doi.org/10.1097/00001786-200410000-00002.

    Article  PubMed  Google Scholar 

  20. 20.

    Ovretveit JC, Shekelle PG, Dy SM, McDonald KM, Hempel S, Pronovost P, et al. How does context affect interventions to improve patient safety? An assessment of evidence from studies of five patient safety practices and proposals for research. BMJ Qual Saf. 2011;20(7):604–10. https://doi.org/10.1136/bmjqs.2010.047035.

    Article  PubMed  Google Scholar 

  21. 21.

    Brennan SE, Bosch M, Buchan H, Green SE. Measuring team factors thought to influence the success of quality improvement in primary care: a systematic review of instruments. Implement Sci. 2013;8(1):20. https://doi.org/10.1186/1748-5908-8-20.

    Article  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Brennan SE, Bosch M, Buchan H, Green SE. Measuring organizational and individual factors thought to influence the success of quality improvement in primary care: a systematic review of instruments. Implement Sci. 2012;7(1):121. https://doi.org/10.1186/1748-5908-7-121.

    Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Bergstrom A, Skeen S, Duc DM, Blandon EZ, Estabrooks C, Gustavsson P, et al. Health system context and implementation of evidence-based practices-development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings. Implement Sci. 2015;10(1):120. https://doi.org/10.1186/s13012-015-0305-2.

    Article  PubMed  PubMed Central  Google Scholar 

  24. 24.

    Li SA, Jeffs L, Barwick M, Stevens B. Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: a systematic integrative review. Syst Rev. 2018;7(1):72. https://doi.org/10.1186/s13643-018-0734-5.

    Article  PubMed  PubMed Central  Google Scholar 

  25. 25.

    Institute for Healthcare Improvement. The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement, vol. 2003. Cambridge: Institute for Healthcare Improvement; 2003.

    Google Scholar 

  26. 26.

    Langley GJ, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A practical approach to enhancing organizational performance. Second ed. United States of America; 2009.

    Google Scholar 

  27. 27.

    Charalambous S, Grant AD, Churchyard GJ, Mukora R, Schneider H, Fielding KL. Clinic-level factors influencing patient outcomes on antiretroviral therapy in primary health clinics in South Africa. AIDS. 2016;30(7):1099–109. https://doi.org/10.1097/QAD.0000000000001014.

    CAS  Article  PubMed  Google Scholar 

  28. 28.

    Uyei J, Coetzee D, Macinko J, Weinberg SL, Guttmacher S. Measuring the degree of integrated tuberculosis and HIV service delivery in Cape Town, South Africa. Health Policy Plan. 2014;29(1):42–55. https://doi.org/10.1093/heapol/czs131.

    Article  PubMed  Google Scholar 

  29. 29.

    Gengiah S, Barker PM, Yende-Zuma N, Mbatha M, Naidoo S, Taylor M, et al. A cluster-randomized controlled trial to improve the quality of integrated HIV-Tuberculosis services in primary health care clinics in South Africa. JIAS. 2021. (in press).

  30. 30.

    Rendell N, Lokuge K, Rosewell A, Field E. Factors that influence data use to improve health service delivery in low- and middle-income countries. Glob Health Sci Pract. 2020;8(3):566–81. https://doi.org/10.9745/GHSP-D-19-00388.

    Article  PubMed  PubMed Central  Google Scholar 

  31. 31.

    Hulscher ME, Schouten LM, Grol RP, Buchan H. Determinants of success of quality improvement collaboratives: what does the literature show? BMJ Qual Saf. 2013;22(1):19–31. https://doi.org/10.1136/bmjqs-2011-000651.

    Article  PubMed  Google Scholar 

  32. 32.

    Youngleson MS, Nkurunziza P, Jennings K, Arendse J, Mate KS, Barker P. Improving a mother to child HIV transmission programme through health system redesign: quality improvement, protocol adjustment and resource addition. PLoS One. 2010;5(11):e13891. https://doi.org/10.1371/journal.pone.0013891.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  33. 33.

    Bardfield J, Agins B, Akiyama M, Basenero A, Luphala P, Kaindjee-Tjituka F, et al. A quality improvement approach to capacity building in low- and middle-income countries. AIDS. 2015;29(Suppl 2):S179–86. https://doi.org/10.1097/QAD.0000000000000719.

    Article  PubMed  Google Scholar 

  34. 34.

    Ogunsola OO, Ajayi O, Ojo O, Adeyeye O, Akinro Y, Oke O, et al. Improving coverage and completion rate of isoniazid preventive therapy among eligible HIV patients using quality improvement approaches: a case study of State Hospital, Ijebu Ode, Ogun State, Nigeria. Pan Afr Med J. 2019;34:193. https://doi.org/10.11604/pamj.2019.34.193.19360.

    Article  PubMed  PubMed Central  Google Scholar 

  35. 35.

    Osibo B, Oronsaye F, Alo OD, Phillips A, Becquet R, Shaffer N, et al. Using small tests of change to improve PMTCT services in Northern Nigeria: experiences from implementation of a continuous quality improvement and breakthrough series program. J Acquir Immune Defic Syndr. 2017;75(Suppl 2):S165–S72. https://doi.org/10.1097/QAI.0000000000001369.

    Article  PubMed  Google Scholar 

  36. 36.

    Krishnan A, Nongkynrih B, Yadav K, Singh S, Gupta V. Evaluation of computerized health management information system for primary health care in rural India. BMC Health Serv Res. 2010;10(1):310. https://doi.org/10.1186/1472-6963-10-310.

    Article  PubMed  PubMed Central  Google Scholar 

  37. 37.

    Nutley T, Gnassou L, Traore M, Bosso AE, Mullen S. Moving data off the shelf and into action: an intervention to improve data-informed decision making in Cote d'Ivoire. Glob Health Action. 2014;7(1):25035. https://doi.org/10.3402/gha.v7.25035.

    Article  PubMed  Google Scholar 

  38. 38.

    Duc DM, Bergstrom A, Eriksson L, Selling K, Thi Thu Ha B, Wallin L. Response process and test-retest reliability of the Context Assessment for Community Health tool in Vietnam. Glob Health Action. 2016;9(1):31572. https://doi.org/10.3402/gha.v9.31572.

    Article  PubMed  Google Scholar 

  39. 39.

    Mocumbi S, McKee K, Munguambe K, Chiau R, Hogberg U, Hanson C, et al. Ready to deliver maternal and newborn care? Health providers' perceptions of their work context in rural Mozambique. Glob Health Action. 2018;11(1):1532631. https://doi.org/10.1080/16549716.2018.1532631.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

• SG is a University of KwaZulu-Natal (UKZN) Developing Research Innovation, Localisation and Leadership in South Africa (DRILL) fellow. DRILL is a NIH D43 grant (D43TW010131) awarded to UKZN in 2015 to support a research training and induction programme for early career academics. The content is solely the responsibility of the authors and does not necessarily represent the official views of DRILL and the National Institutes of Health.

• SG was supported by European and Developing Countries Clinical Trials Partnership (EDCTP) Grant TMA2018SF-2467

• We thank the South African Department of Health, District Management Teams of the Ugu and King Cetshwayo Districts for supporting this study; all the healthcare workers that participated in the trial, and all members of the SUTHI study field team.

Funding

The research reported in this paper was supported by the South African Medical Research Council with funds received from the South African National Department of Health, and the UK Medical Research Council, with funds received from the UK government’s Newton Fund. This UK funded award is part of the EDCTP2 programme supported by the European Union.

Author information

Affiliations

Authors

Contributions

SG was responsible for the study conduct, development of the first draft of the manuscript, intellectual input, analysis, and interpretation of the results. CC and NYZ provided input on the analysis and interpretation of results, PB provided input on the study design and manuscript. AJN, ML, and MT edited the manuscript and provided intellectual input. KN and NP provided intellectual input, provided oversight of the SUTHI trial, and contributed to the writing of the manuscript. KN is the grant holder. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Santhanalakshmi Gengiah.

Ethics declarations

Ethics approval and consent to participate

The SUTHI trial was approved by the Biomedical Research Ethics Committee of the University of KwaZulu-Natal (BF108/14). The KwaZulu-Natal (KZN) Health Research and Knowledge Management committee granted permission to access PHC clinics in the study districts of KZN (HRKM309/14). All healthcare workers who agreed to participate in the study signed an ethics-approved informed consent form which was available in English and the local language.

Consent for publication

Not applicable.

Competing interests

The authors declare they have no conflicts of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gengiah, S., Connolly, C., Yende-Zuma, N. et al. Organizational contextual factors that predict success of a quality improvement collaborative approach to enhance integrated HIV-tuberculosis services: a sub-study of the Scaling up TB/HIV Integration trial. Implementation Sci 16, 88 (2021). https://doi.org/10.1186/s13012-021-01155-7

Download citation

Keywords

  • Quality improvement collaboratives
  • HIV-TB integration
  • Cluster-randomized trial
  • Organizational contextual factors
  • South Africa