Skip to main content
Download PDF
Download ePub

Systems analysis and improvement approach to optimize outpatient mental health treatment cascades in Mozambique (SAIA-MH): study protocol for a cluster randomized trial

Implementation Science volume 17, Article number: 37 (2022) Cite this article

Abstract

Background

Significant investments are being made to close the mental health (MH) treatment gap, which often exceeds 90% in many low- and middle-income countries (LMICs). However, limited attention has been paid to patient quality of care in nascent and evolving LMIC MH systems. In system assessments across sub-Saharan Africa, MH loss-to-follow-up often exceeds 50% and sub-optimal medication adherence often exceeds 60%. This study aims to fill a gap of evidence-based implementation strategies targeting the optimization of MH treatment cascades in LMICs by testing a low-cost multicomponent implementation strategy integrated into routine government MH care in Mozambique.

Methods

Using a cluster-randomized trial design, 16 clinics (8 intervention and 8 control) providing primary MH care will be randomized to the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH) or an attentional placebo control. SAIA-MH is a multicomponent implementation strategy blending external facilitation, clinical consultation, and provider team meetings with system-engineering tools in an overall continuous quality improvement framework. Following a 6-month baseline period, intervention facilities will implement the SAIA-MH strategy for a 2-year intensive implementation period, followed by a 1-year sustainment phase. Primary outcomes will be the proportion of all patients diagnosed with a MH condition and receiving pharmaceutical-based treatment who achieve functional improvement, adherence to medication, and retention in MH care. The Consolidated Framework for Implementation Research (CFIR) will be used to assess determinants of implementation success. Specific Aim 1b will include the evaluation of mechanisms of the SAIA-MH strategy using longitudinal structural equation modeling as well as specific aim 2 estimating cost and cost-effectiveness of scaling-up SAIA-MH in Mozambique to provincial and national levels.

Discussion

This study is innovative in being the first, to our knowledge, to test a multicomponent implementation strategy for MH care cascade optimization in LMICs. By design, SAIA-MH is a low-cost strategy to generate contextually relevant solutions to barriers to effective primary MH care, and thus focuses on system improvements that can be sustained over the long term. Since SAIA-MH is integrated into routine government MH service delivery, this pragmatic trial has the potential to inform potential SAIA-MH scale-up in Mozambique and other similar LMICs.

Trial registration

ClinicalTrials.gov; NCT05103033; 11/2/2021.

Background

Mental disorders are the leading cause of disability worldwide. In Mozambique, mental disorders account for 16.5% of years lived with disability (YLD) [1]. The treatment gap for mental disorders exceeds 90% in many low- and middle-income countries (LMICs) [2,3,4], in part due to a shortage of 1.2 million mental health (MH) workers globally [5]. To address this gap, academic and policy leaders have advocated for investments in rapid training programs for lower-level MH providers [6,7,8,9,10,11,12]. These investments are reaping benefits: since 2011 the number of MH nurses globally has increased by 35% [13].

Task-shared providers can deliver effective, evidence-based treatments for mental disorders in LMICs, and Mozambique has been a regional leader in MH task-sharing since 1996. It has been > 15 years since the first trials showed that MH treatments led by task-shared providers in LMICs could effectively treat mental disorders [14,15,16]. The 2016 Disease Control Priorities declared that task-shared outpatient treatment of common mental disorders in LMICs is cost-effective [17]. Mozambique has successfully scaled-up task-shared MH care despite only 0.16% of the health budget allocated to MH. The majority of MH care is provided by psychiatric technicians who complete a 2-year training program after acquiring at least a 10th-grade education [18]. The number of technicians working nationally increased from 66 in 2010 to 305 in 2019, achieving 100% coverage of at least one technician per district in 2014. In the province of Sofala (Fig. 1), increased coverage of technicians resulted in an 100% increase in outpatient MH consultations from 2012 to 2014 [20].

Fig. 1
figure 1

Mozambique with focal provinces of Sofala and Manica outlined in red. Figure sourced from Fernandes QF et al. (2014) [19]

Full size image

With most global investment focused on scaling-up task-sharing to close the MH treatment gap, less attention has been paid to implementation strategies that can assess and improve the quality of care delivered by existing providers with limited training, resources, and supervision. For example, in system assessments across Southern Africa, diagnosis was not recorded in 44% of cases, there was low awareness of non-psychotic mental illness, psychiatric history was inadequate in 89% of cases, inappropriate polypharmacy was prescribed in 88% of cases, and loss-to-follow-up (LTFU) exceeded 40% while sub-optimal medication adherence exceeded 60% [21,22,23,24]. Task-shared outpatient management of common mental disorders in Mozambique has shown LTFU rates > 50%, sub-optimal medication adherence > 80%, and < 30% of patients achieving functional improvement [25]. Similar patterns of suboptimal adherence, LTFU, and improvement outcomes have been reported in Nepal, Guinea-Bissau, and Tanzania [24,25,26]. There is an urgent need for strategies to optimize the task-shared MH treatment cascade. Until this need is met, the health effects of scaling-up task-shared MH care will be, at best, limited, and at worst, harmful in some settings.

The MH treatment cascade is a model that outlines the sequential, linked treatment steps that people with mental illness must frequently navigate, from initial diagnosis to symptom/functional improvement. Quality problems in one step of a treatment cascade can have non-linear and compounding impacts across the larger complex care system. Implementation strategies focused on only one step in a cascade can potentially contribute to unintended system bottlenecks and quality of care issues. By contrast, the “Systems Analysis and Improvement Approach (SAIA)” is a multicomponent implementation strategy focused on optimizing treatment cascades in their entirety. SAIA blends facilitation, ongoing clinical consultation, and routine service provider team implementation meetings with systems-engineering tools in a 5-step approach specifically developed for task-shared providers. The five steps of SAIA include (1) cascade analysis to visualize treatment cascade drop-offs and prioritize areas for system improvements; (2) process mapping to identify modifiable facility-level bottlenecks; (3) identification and implementation of modifications to improve system performance; (4) assessment of modification effects on the cascade; and (5) repeated analysis and improvement cycles.

A previous 9-month cluster RCT established the effectiveness of SAIA for HIV treatment cascade improvement across Mozambique, Cote-D’Ivoire, and Kenya, showing a 3.3-fold greater improvement in ART uptake for HIV-infected pregnant women (13.3% vs. 4.1% increase), and a 17-fold greater improvement in early infant diagnosis for HIV-exposed infants (11.6% vs. 0.7% increase) [27]. Intervention facilities tested an average of 9 system modifications over the 9-month intervention, and over 80% of those “micro-changes” were considered successful by facility personnel. However, no effectiveness data exist on SAIA applied to non-HIV treatment cascades such as task-shared MH care. Preliminary data suggest that applying SAIA to MH treatment cascade optimization is feasible, acceptable, and can result in clinically significant treatment cascade improvements. Five months of pilot SAIA-MH implementation resulted in a 1.5-times higher odds of medication adherence (aOR 1.5; CI 1.2, 1.9) and a 3.7-times higher odds of functional improvement (aOR 3.7; CI 2.5, 5.4) [25]. Systems changes tested included service re-organization, new interventions, patient education, and improving data and its use. The SAIA-MH strategy was feasible: all teams adopted the strategy and conducted 5 optimization cycles during the 5-month period. These pilot data suggest that SAIA-MH is a promising strategy for task-shared MH systems improvement globally.

The overall goal of the present study is to evaluate the real-world effectiveness of the multi-component SAIA-MH implementation strategy to optimize outpatient mental health services in Mozambique. Specific aim 1a will assess effectiveness using a 3-year parallel cluster randomized trial across 8 intervention and 8 attentional control facilities. Specific aim 1b will test causal pathway models to analyze mechanisms of action for effects (or non-effects) of the SAIA-MH implementation strategy. Specific aim 2 will focus on estimating the cost and cost-effectiveness of scaling-up SAIA-MH in Mozambique. This study is innovative in extending an evidence-based implementation strategy from the field of HIV to MH treatment, while also being the first to test not only whether the SAIA implementation strategy works—but how it works—through testing causal pathway models. If effective, the SAIA-MH implementation strategy has large potential to be rapidly scaled-up to decrease gaps in task-shared MH systems globally.

Methods

Systems Analysis and Improvement Approach for Mental Health (SAIA-MH) Implementation Strategy Specification

The SAIA-MH implementation strategy builds off continuous quality improvement [28] (CQI) [29] influenced by the Donabedian model [30], a framework for assessing quality of care based on interconnected components of structure, process, and outcomes. Structure describes attributes of the setting in which a provider delivers care (organizational structure, along with material/human resources). Process describes what is done to the patient as that individual receives services. Both can be measured and manipulated. Health outcomes are a function of the structure and process and changes made to them. The SAIA strategy is a multicomponent package of distinct implementation strategies [31] targeting a variety of individual, organizational, and contextual factors that foster or impede implementation success [32,33,34] (Table 1). In addition, SAIA uses systems-engineering [35] principles to support task-shared workers to diagnose and prioritize problems; evaluate decision options using optimization; and recommend, implement, and evaluate actions considering the treatment cascade as a whole. By design, the SAIA model guides workers to generate ideas to address local problems in the treatment cascade, resulting in a high likelihood that proposed changes are appropriate for the local context and acceptable to the implementing team. Thus, the SAIA model focuses on increasing the chances of system changes being sustained over the long term.

Table 1 Distinct SAIA-MH implementation strategies coded to the Expert Recommendations for Implementing Change (ERIC) framework [31]
Full size table

The SAIA-MH implementation strategy blends external facilitation [36], ongoing clinical consultation, and service provider implementation team meetings with system-engineering tools in an overall CQI framework [28, 37] (Table 1; Fig. 2). External experts in the SAIA implementation strategy partner with respected, knowledgeable local district/provincial management in the health system. These local leaders work with the external SAIA team to contextualize and adapt the SAIA strategy to the specific context. Such adaptation has been performed as part of the original HIV-focused SAIA trial [29], as well as the SAIA-MH pilot study [25]. This included (1) specifying a MH “care cascade” to be optimized that was both feasible to populate with routine health systems data as well as relevant in the Mozambican public-sector context; (2) developing enhanced MH clinical registries and patient tracking tools to allow population of the MH care cascade; (3) finalizing the Mozambican mental health cascade analysis tool (MHCAT) used to provide a rapid systems-level view of drop-offs along the MH care cascade with an optimization function to help prioritize which step in the care cascade is weakest. Following contextualization, clinic directors, managers, and all staff directly involved in MH care delivery form a facility-level MH systems improvement team and attend a 1-week training co-led by the local district/provincial management and external SAIA-MH experts. Afterwards, the external SAIA-MH team and local management provide facilitation with facility-level system improvement teams, guiding them through the monthly iterative 5-step SAIA-MH implementation strategy process [29]. This process gives facility teams a systems-wide view of the treatment cascade performance within their health facility, guides them in collaboratively understanding their current system, helps them to diagnose problems, prioritize areas for improvement, and then recommend, implement, and evaluate improvements [35].

Fig. 2
figure 2

Systems Analysis and Improvement Approach for Mental Health (SAIA-MH) implementation strategy processes

Full size image

The 5 steps of SAIA-MH are the following:

  • Step 1: Cascade analysis—understand MH cascade performance and identify priority areas for systems improvement [38, 39]. The MHCAT (Fig. 3) uses routine facility-level data to provide a systems-level view of the interdependent components determining care delivery. As an analytic tool, the MHCAT provides MH staff with a view of the weakest areas of the current treatment cascade and aids teams in prioritizing targets for systems improvement. The MHCAT, along with run charts and process indicators, will also help facility teams evaluate the impact of small tests of systems change in later steps.

  • Step 2: Process mapping—diagnose facility-level modifiable bottlenecks and gain consensus on current system. Enabling facility-level staff to diagnose and gain consensus on bottlenecks to address in their MH systems is essential to defining system improvements to test. SAIA-MH applies sequential process mapping, coupled with workflow observation, to build teamwork and consensus on existing system structure and guide discussion on opportunities for improvements.

  • Step 3: Identify, formally plan, and implement system change. After diagnosing modifiable system barriers, facility teams identify a simple, specific change to improve performance within the prioritized cascade step. Selected system changes should be feasible to implement, be within the scope of influence of facility teams, and be expected to lead to rapid, substantial improvement. Facilitation is provided to help teams identify improvements. Facility teams make a formal implementation plan to ensure consensus and clarify operational design/roles. Steps 3–5 borrow heavily from CQI.

  • Step 4: Monitor changes in routine performance. Facility teams monitor changes in a selected process indicator, as well as track patients progressing through targeted steps using run charts. These indicators capture large, rapid improvements.

  • Step 5: Repeat analysis and improvement cycles. Systems-engineering [35] is by definition iterative given that system innovations must respond to evolving, contextually specific barriers. Facility teams repeat steps 1–5 during each monthly cycle to identify new approaches to modify identified barriers, or if the first cycle was successful, focus on identifying new priority barriers.

Fig. 3
figure 3

Mental health cascade analysis tool (MHCAT)

Full size image

Study aims

The present project proposes to conduct a 3-year parallel cluster RCT across 8 intervention and 8 attentional control facilities to evaluate the real-world effectiveness of the SAIA-MH implementation strategy on mental health functional improvement (primary) and patient retention in care/medication adherence (secondary). Specific aim 1b will include testing causal pathway models to analyze mechanisms of action of the SAIA-MH strategy using longitudinal structural equation modeling as well as specific aim 2 estimating the incremental cost and cost-effectiveness of scaling-up SAIA-MH in Mozambique using micro-costing, time-and-motion observation, and a Markov model parametrized with cost and outcome data from the SAIA-MH cluster RCT.

Overview of study design

We will implement a 3-year cluster RCT, randomizing 16 clinics currently providing outpatient mental health care to treatment (8 clinics) and attentional placebo control (8 clinics). We will include a 6-month baseline period prior to randomization, followed by a 2-year intervention or attentional placebo control period, and an additional 1-year sustainment phase (Table 2). During the 6-month baseline phase, all clinics will receive enhanced patient registries and patient tracking tools as used in the pilot study [25]. All facilities will receive monthly in-facility mentorship on the new data tools, allowing for examination of baseline cascade performance. During these monthly visits, data from the patient registries and tracking tools will be entered into the CommCare database by study staff. Following the baseline period, clinics will be randomized to the SAIA-MH implementation strategy or attentional placebo control.

Table 2 Research project activities and timeline
Full size table

Those receiving SAIA-MH will attend a 1-week in-person training for facility improvement teams. Facility-level teams will be determined by facility staff but will need to include a minimum of one psychiatric technician, one psychologist, and the clinic manager/director who oversees MH delivery. Following the 1-week in-person training, SAIA-MH standard operating procedures will be implemented, including: (1) structured external facilitation and clinical consultation following tablet-based guides used in pilot study (1× per week first month; 2× per month for next 2 months; 1× per month for remainder); (2) facilitation in the 5-step SAIA-MH improvement process, including: (3) cascade analysis and prioritization using the MHCAT; (4) process mapping for bottleneck identification; (5) defining system changes/developing implementation plans; and (6) evaluating and revising system changes. As in the original HIV-focused SAIA trial [29], facilities will be able to request reimbursement of up to $50/month for essential supplies needed to test systems changes. During these regular facilitation visits, clinic staff who are participating in the intervention will complete questionnaires designed to measure moderators, preconditions, and hypothesized mechanisms of SAIA-MH. In-facility facilitation and clinical consultation early in SAIA-MH implementation is intensive and has proven critical to helping teams begin to think from a “systems-engineering perspective” [35]. SAIA-MH protocols suggest cycles occur monthly, corresponding with monthly MHCAT systems analysis data, thus, we anticipate that facilities will complete 24 systems improvement cycles during the 2-year intensive implementation phase and 12 during the sustainment phase.

We will employ an attentional placebo control design whereby control facilities will mimic activities of the intervention group in time and contacts, but without the “active ingredient” of the SAIA-MH implementation strategy [40]. This design protects against key threats to internal validity, such as the Hawthorne effect [40], testing effects [41], and generally accounts for non-specific effects of SAIA-MH activities in a similar way that a placebo is used to account for expectancy effects in a drug trial [40]. Thus, facilities randomized to attentional placebo control will attend a 1-week in-person training for facility learning collaboratives determined by facility staff, including the same minimum staff above for SAIA-MH. This training will focus on a review of the enhanced patient registries and patient tracking tools, as well as discuss ethics, mental health stigma, and burnout for mental health professionals. Following the 1-week in person training, attentional placebo control facilities will receive monthly in-clinic facilitation following the same schedule as SAIA-MH paired with monthly data collection on moderator, precondition, and hypothesized mechanisms. External facilitation will focus on review of the use of the enhanced patient registries and patient tracking tools. As the SAIA-MH implementation strategy occurs at the systems level in public sector clinics in Mozambique, there will be no restrictions placed on concomitant care. Any observed external changes in the health system suspected of influencing study outcomes will be noted in final published materials. Important protocol modifications will be communicated to both ethics committees at the University of Washington and in Mozambique and a communication plan with stakeholders developed under their simultaneous guidance.

During the 1-year sustainment phase, management of data collection to produce the MHCAT will transfer fully to the provincial statistics office and SAIA-MH facilitation will be done only by local management to examine effectiveness and sustainability under fully routine conditions. Provincial authorities currently collect patient-level MH data monthly to report diagnostic and outcome data within the Ministry of Health system. Primary effectiveness analyses of the SAIA-MH implementation strategy will occur at the end of the 2-year intervention period. This project will partner with Ministry of Health staff and support the automation of existing monthly reports, with the simultaneous creation of the MHCAT cascade analysis for data feedback to the facility level.

Study setting

Manica and Sofala Provinces (population: approximately 4 million; Fig. 1) were selected because of the deep relationship between investigators and the local Ministry of Health as well as the absence of existing structural interventions for MH systems improvement. The Mozambican Ministry of Health has been a leader in sub-Saharan Africa in scaling-up task-shared outpatient mental healthcare led by mid-level specialist providers called psychiatric technicians. These providers can treat all categories of mental, neurological, and substance-use disorders using both pharmacological and psychosocial evidence-based interventions. In 2014, Mozambique achieved its goal of having at least one psychiatric technician providing outpatient mental healthcare in each of the 128 districts of Mozambique [18]. However, existing evidence suggests that the quality of mental healthcare in Mozambique remains low [42] due to limited financial and human resources, few opportunities for supportive supervision, training, and re-training [18], stock-outs of essential mental health medications [43], and mental health stigma [44] among other challenges [25]. In central Mozambique, over 98% of formal health services are offered through the public sector [45], improving the potential for population-level impacts for supply-side systems interventions.

Study outcomes

Primary SAIA-MH study outcomes will be individual-level patient functional improvement (primary), medication adherence (secondary), and retention in care (secondary) (Table 3). Patient functional improvement was selected as the primary trial outcome because it had the largest relative increase in the pilot study (aOR = 3.7, 8.9% increase, improvement from 4.2 to 13.1%) and it is the ultimate goal of MH treatment as the last step in the care cascade; therefore, improvements in upstream cascade steps and process/quality indicators should be expected to also improve functional improvement. The WHODAS 2.0 for primary outcome measurement was developed by the World Health Organization (WHO) specifically to allow cross-cultural disability measurement across settings and patient populations [46]. It has good reliability, item-response characteristics, and a robust factor structure that has been replicated across sub-Saharan Africa [46,47,48,49]. In the original WHO validation study across 19 countries (9 LMICs) the test-retest reliability had an intra-class coefficient of 0.98, the total Cronbach’s alpha (n = 1565 across 19 countries) was 0.98 and also 0.98 for MH patients [46, 49]. The WHODAS 2.0 has been used extensively in LMICs [46,47,48,49,50], shows good performance among diverse MH diagnoses [51,52,53], has been applied successfully in Mozambique in previous studies [50], including our pilot study, and has been extensively applied in LMICs (including Sub-Saharan Africa) to evaluate functional improvement in pragmatic MH trials [54,55,56,57,58,59]. Primary study outcomes will be sourced from a census of individual-level patient records from outpatient routine care, which will also be used to populate the MHCAT tool.

Table 3 Primary study outcome definitions. All patient and facility-level outcomes refer to all patients attended in outpatient mental health services in target health facilities
Full size table

Sampling and randomization procedures

Eligible facilities are public-sector Ministry of Health clinics naïve to SAIA-MH, with ≥ 100 average annual outpatient MH consultations from 2019 to 2020 and the presence of a minimum of one psychiatric technician and one psychologist. For feasibility regarding project implementation in an area with challenges during rainy season, inclusion criteria for facilities included being within a 3-h one-way drive from Chimoio City, Manica Province, or Beira City, Sofala Province. Last, quaternary or provincial hospitals were excluded due to the complexity of mental health workflows across specialty services. Given these criteria, there were 9 eligible facilities in Manica Province and 8 eligible facilities in Sofala Province (Table 5). Eligible facilities will be allocated 1:1 to intervention or control using constrained randomization to maximally balance province, level of health facility, MH human resources, number of annual outpatient MH consultations, and baseline cascade performance. Baseline cascade performance will be facility-level achievement of patient functional improvement during the 6-month baseline period (Table 3). There will be no blinding of facility assignment at the level of providers, managers, or investigators as it is not feasible given SAIA-MH implementation procedures. The only blinding will occur at the level of the outcome assessor (statistical analyst). Randomization will be conducted in Stata 16.0 using the ccrand command for covariate-constrained randomization in cluster-randomized trials [60]. The allocation sequence will be generated by the senior author (BHW) and enrollment—including informed consent—led by technical implementation leads in Mozambique (VFJC and AM).

Data collection and management

Patient- and facility-level process and quality information (Table 3) will be abstracted into a separate CommCare [61] database by study team staff. CommCare is an android-based platform to support mobile data collection specifically in low-resource settings globally. De-identified aggregate patient and visit-level data from CommCare will be directly linked to Tableau [62] data visualization software to visualize facility-level MHCAT results in real-time. MHCAT visualizations will be shared with each facility during the scheduled supervision and implementation team meetings. Separate CommCare forms will be used to track action planning, facilitation activities, mechanisms, and costs of SAIA-MH implementation. These supervision forms will be filled out by the study team during each supervision visit. Thus, primary study outcomes, implementation outcomes, and other process measures will be sourced from: (1) CommCare patient-level data collection and supervision forms/questionnaires; and (2) qualitative CFIR interviews. Monthly data will include longitudinal documentation of changes in mechanisms of effects for all 16 facilities (Fig. 4), fidelity to the SAIA-MH model (intervention only), and barriers/facilitators to implementation. During the sustainment phase, collection, and management of the CommCare and Tableau data will transition fully to the provincial statistics offices of Sofala and Manica provinces. Research and clinical staff will be trained to identify potential adverse events and instructed to report them immediately to the senior author (BHW), implementation leads (VFJC and AM) and the country director for implementation activities in Mozambique (IR). Adverse events will be reported under the schedule expected by the University of Washington ethics committee.

Fig. 4
figure 4

Causal pathway model of hypothesized mechanisms of SAIA-MH effects

Full size image

Primary statistical analyses

Individual-level primary, secondary, and process/quality outcome data will be analyzed using generalized linear mixed models using random effects at the facility and individual level. Family and link functions will be determined based on outcome distributions, although a priori specification will utilize binomial family and logit links to evaluate the odds of progressing through the MH care cascade. Absolute and percentage changes in WHODAS 2.0 scores between patients in intervention and control facilities will also be computed. Primary analyses will compare changes from the 6-month baseline to the 2-year intensive implementation period for intervention versus control facilities. Analyses will be repeated comparing effects over the sustainability phase to both baseline and implementation phases. Patient (age, sex, HIV/TB diagnosis, suicidal ideation) and health-facility adjustment factors (patient load, staff turnover or absence, distance from district/provincial headquarters) will be considered for inclusion. Sex as a biological variable will be assessed using pre-specified interaction sub-analyses to examine differential SAIA-MH effects for females vs. males. Dose-response analyses by levels of fidelity to protocol will also be examined. We will also conduct a controlled, segmented time-series analysis [63, 64] that incorporates monthly facility-level estimates from the entire 42-month study period (segmented into 6-month pre-intervention; 24-month intervention; and 12-month sustainment periods). This secondary analysis will allow assessment of SAIA-MH on number of new diagnoses, treatment initiation, total patient load, as well all other MH cascade outcomes, addressing both serial/intra-cluster correlation, and temporal patterns in SAIA-MH effectiveness. These facility-level time-series analyses will also potentially allow exploratory examination of the effects of specific systems modifications. Modifications found to have large effects will be documented as part of identifying best practices for MH care cascade improvement. All statistical analyses will be conducted blinded to study arm (intervention vs. control).

Primary study power assessment

In the pilot, SAIA-MH increased patient functional improvement with an adjusted odds ratio of 3.7 and an absolute increase of 8.9% (4.2% control; 13.1% intervention) [25]. The intra-cluster correlation coefficient using outpatient mental health data in the same setting is estimated at 0.26 [https://pophealthmetrics.biomedcentral.com/articles/10.1186/s12963-015-0043-3]. Selected clinics in Table 4 conducted an average of 774 outpatient MH visits per 6 months in 2019–2020. In the pilot, 40% of patients returned for follow-up within 60 days in control periods; thus, we estimate target clinics will see approximately 300 follow-up patients per 6 months. Using these assumptions, we will have ≥ 90% power to detect an increase as small as 2.2% in functional improvement (primary), 3.6% for adherence (secondary), and 4.9% for retention (secondary); (Table 5).

Table 4 Facility-level characteristics
Full size table
Table 5 Power under various conditions. Green indicates power greater than or equal to 90%
Full size table

Qualitative data collection and analyses

At the end of the 2-year intensive implementation phase, we will conduct focus group discussions (FGDs) among each facility-level improvement team (n = 16) to collect in-depth qualitative data on CFIR constructs of interest (Table 6). Questions will be sourced from CFIR interview guides (http://cfirguide.org/), supplemented by targeted elicitation of satisfaction with tools/protocols, intent-to-continue-use, and deviations from SAIA protocols. FGDs will occur after primary quantitative analyses, with facilities classified as high, medium, or low performing in terms of fidelity to SAIA-MH as well as cascade improvements. By classifying clinics, we intend to uncover salient features and determinants of successful implementation. Following analyses of mechanisms of effect, we will also classify a sub-set of individual staff (n = 10–20) and facilities (n = 4–6) into groups who showed large improvements in causal mechanistic pathways and those with little improvement across targeted mechanisms. These staff will be targeted for additional in-depth-interviews (IDIs, individuals) or FGDs (facilities) organized around the CFIR and paired with targeted elicitation of qualitative explanatory data on barriers/facilitators to specific mechanisms of action. These data will help: (1) inform the need for tailoring of SAIA-MH for specific facility and individual-level contexts that may hinder or enhance mechanism activation; (2) understand other potential moderators, mediators, preconditions, or mechanisms not captured in current causal pathway models; and (3) further unpack the mechanistic reasons SAIA-MH may show effects in some settings, and for some individuals, and not others. To refine models and understand implementation mechanisms, processes, and determinants under routine conditions, both rounds of FGDs and IDIs will be repeated at the end of the sustainment phase. IDIs and FGDs will be conducted in Portuguese by an experienced facilitator accompanied by a note-taker, audio-recorded, and transcribed verbatim into Portuguese by trained Mozambican staff. Using existing CFIR codebooks as a guide, two researchers will use a stepwise, iterative process to review transcripts and identify key deductive themes (using select CFIR constructs, implementation outcome, and targeted mechanism themes) while allowing for flexibility of other inductive themes to emerge. Following iterative coding of transcripts, the team will convene with the PI to identify coding discrepancies with the coding process repeated until consensus is achieved.

Table 6 Consolidated Framework for Implementation Research (CFIR) constructs of interest
Full size table

SAIA-MH mechanisms assessment approach

Using theoretical bases informing SAIA implementation strategy development, a review of literature for similar distinct implementation strategies (audit and feedback [65,66,67,68,69], quality improvement [70,71,72,73,74,75], facilitation [76, 77]), and mixed-methods quantitative [27] and CFIR qualitative data [27] from the original HIV-focused SAIA trial and the SAIA-MH pilot study, we have developed an initial causal pathway model to test in the present study (Fig. 4). Each moderator, precondition, and hypothesized mechanism will be measured each month during baseline, intensive implementation, and sustainment periods across all 8 intervention and 8 attentional placebo control facilities during facility-level facilitation/supervision visits. For clarity, we hypothesize that attentional placebo controls will show no significant activation of SAIA-MH mechanisms but will account for key threats to validity such as testing, maturation, external shocks, and Hawthorne effects. Mechanisms will be assessed monthly at the individual provider level and then aggregated within facilities across time, while MH cascade performance will be sourced from individual patient CommCare records and then aggregated within facilities across time. Primary analyses of mechanisms will occur after the intensive phase (Table 2). Secondary analyses will occur at the end of the sustainment phase.

SAIA-MH mechanisms assessment power

Overall, this study was powered to specific aim 1a examining mean treatment effects of SAIA-MH on MH treatment cascade performance. Given the paucity of literature on methods, theories, and frameworks underlying mechanisms of multicomponent implementation strategy effects, this aim will focus on an exhaustive examination in search of practically and clinically meaningful mechanism, moderator, and precondition effect estimates for further examination in subsequent research. The goal of this aim is to contribute to theory building to inform future studies specifically powered to test hypothesized causal pathway models. No confirmatory hypothesis testing driven by p value inference will be performed, as recommended when conducting exploratory studies by Leon (2011) [78]. That said, we will have a sufficient sample size to have stable effect estimates with reasonable precision based on our design including 6 baseline measurements and 24 implementation measurements among a minimum of 3 providers across all 16 facilities. Assuming an intra-cluster correlation coefficient of 0.25 [20], if a binary mechanism leads to a function improvement odds ratio of 1.5, we will have > 0.85 power to detect the indirect effect with variance of the mediator assumed to be 0.5. Continuous mechanisms will have appreciably higher power.

SAIA-MH mechanisms assessment analyses

Our analytical plan will progress in 3 stages and across 3 separate patient-level outcomes: functional improvement (primary), medication adherence (secondary), and retention (secondary). Repeating the analytic process across 3 dependent variables provides built-in replication, allowing the identification of consistent effects (direct, indirect, and moderated). First, we will conduct univariate examination of hypothesized mechanism effects using multi-level longitudinal SEM accounting for multiple providers nested within facilities across time. The purpose of this will be to examine single main effect estimates for hypothesized mechanisms. For example, we will estimate a univariate multi-level longitudinal SEM of SAIA-MH strategy effect predicting changes in teamwork, then predicting MH treatment cascade outcomes (Fig. 4). We will conduct diagnostics, including assessing linearity of mechanism effects, model residual distributions, and variance accounted for in mechanisms and outcomes. We will then test potential moderators in univariate models, in search of those that strongly affect strength or direction of univariate effects. Moderators will be examined for their effect on SAIA-MH strategy activation of mechanisms (left side of Fig. 4) as well as mechanism effects on outcomes (right side of Fig. 4); preconditions will be tested alongside moderators. Second, we will include all mechanisms simultaneously in a multivariable longitudinal multi-level SEM. Including all mechanisms, even those showing no effects in univariate models accounts for potential suppressor effects [79]. Moderators that showed meaningful effects in univariate models will then be included in a final multivariable model. Final model diagnostics will be verified, and variance accounted for in mechanisms, as well as outcomes, will be reported. This multivariable model will allow examination of (1) mechanisms most affected by SAIA-MH (left side Fig. 4); (2) mechanisms that had an effect on outcomes (right side Fig. 4); and (3) the product of #1 and #2 path coefficients indicating the indirect effect of SAIA-MH on treatment outcomes operating through a given mechanism. Separating out paths #1 and #2 can inform improvement of the SAIA-MH strategy; for example, if a mechanism had a large effect on outcomes (path #2), although was only weakly activated by SAIA-MH (path #1), the SAIA-MH strategy could be modified to enhance activation of this potentially high-leverage mechanism. Findings from path analyses will be incorporated into targeted elicitation of explanatory qualitative information during the second round of CFIR interviews. Individuals/facilities will be classified into those showing large mechanism changes compared to those with little improvement.

Costs

We will conduct activity-based costing and time and motion observation at both intervention and control clinics to estimate the incremental cost of implementing SAIA-MH. Throughout study implementation, we will estimate all costs that would be incurred by the Ministry of Health to administer the SAIA-MH intervention. These costs will be collected and tabulated using activity-based cost menus during start-up, intensive implementation, and sustainability periods. Costs include patient visits, medication, personnel, supervision, supplies, buildings and overhead, strategy delivery, facility support for systems changes, training, and equipment. Time-motion observations following protocols previously used by our group in Mozambique [80] will estimate personnel time needed for SAIA-MH tasks.

Economic evaluation

We will develop a Markov cohort model using R software reflecting the natural history of MH disease progression and disability, and treatment, which has been used for previous cost-effectiveness analyses of MH interventions in LMICs [81,82,83,84,85,86,87]. The model will simulate a cohort with different MH conditions stratified by retention in care, medication adherence, and functional improvement. We will parameterize the model with cost and outcome data from the SAIA-MH cluster RCT and facility-level contextual data (patient loads and trends; human resources; facility types) and demographic data (gender; age distributions). In line with previous studies and economic guidelines [81, 88], costs and effectiveness will be discounted at 3% annually (varied from 0-5% in sensitivity analyses). Disability weights for DALYs will be obtained from the Global Burden of Disease 2017 [89]. Scale-up scenarios will model changes in patient retention, adherence, and functional improvement based on study results and varied in sensitivity analyses. Prior to conducting cost-effectiveness analyses, we will assess model validity/reliability by evaluating our model’s ability to accurately project trends in care cascade outcomes (retention, medication adherence, functional improvement).

We will simulate the health and economic impacts of scaling-up the SAIA-MH implementation strategy in Mozambique at district, provincial, and national levels. Health outcomes include effects on the outpatient mental health cascade (proportion retained, medication adherent, and with functional improvement). The model will estimate the effects of scale-up scenarios on: (1) retention and adherence in MH care by diagnosis; (2) functional improvement by diagnosis; and (3) Disability Adjusted Life Years (DALYs) [88] averted, cost per DALY averted, and budgetary impact from the payer (Ministry of Health) perspective. Different scenarios will examine rate and breadth of expansion (number/characteristics of future facilities/districts/provinces). We will estimate incremental costs, as well as treatment costs incurred and averted due to the implementation strategy.

We will project SAIA-MH effects on functional improvement, adherence, and retention compared to standard of care. We will calculate the cost per mental health patient achieving functional improvement and the incremental cost-effectiveness ratio (ICER) per new mental health diagnosis and DALY averted for each scale-up scenario. Analyses will be from the payer perspective, using economic productivity data to estimate mental health disability averted and incorporating costs of healthcare to evaluate economic impact.

Discussion

This study is innovative in extending a breakthrough from the field of HIV to MH treatment, while also being the first to assess not only whether the SAIA implementation strategy works—but how it works—through testing causal pathway models. Systematic reviews of implementation strategies for mental health have found that nearly half produce no statistically significant effects on targeted implementation or clinical outcomes [90]. Leaders in the field have advocated that future studies testing implementation strategies specify strategy-mechanism-outcome linkages and test these mechanisms through causal pathway analyses [91]. Causal pathway analyses, and the specification of mechanisms of effect, are necessary to: (1) better understand why unsuccessful strategies fail; (2) inform the development of more effective and efficient implementation strategies; (3) identify mutable targets for future strategies; and (4) inform tailoring and matching strategies to barriers and contexts [91, 92]. Despite this urgent need in the field, a systematic review found that of 88 RCTs testing implementation strategies for mental health, zero met minimum criteria necessary for testing mediation hypotheses [92]. Mechanistic models of effect are needed to match/tailor strategies to barriers/contexts, prioritize strategy use, and expedite the development of more effective, efficient, and feasible strategies.

Applying the SAIA implementation strategy to task-shared MH systems optimization will distinguish between the “hard core” and “adaptable periphery” of the SAIA strategy, as well as its “transferability” to other disease areas and treatment cascades. Understanding which components of an implementation strategy are essential to effectiveness and which components may be adapted to suit context is critical to informing strategy scale-up. The present study provides a unique opportunity to distinguish between these components. By fixing the contextual determinants and modifying the application of SAIA, this study builds directly from prior studies and will directly inform which components are essential to effectiveness across disease areas [27, 29]. This project will also help generate an understanding of the overall “transferability” of the SAIA strategy to meet the needs of different treatment cascades or disease areas [93, 94]. Data on SAIA-MH effectiveness, determinants of implementation success, and mechanisms of effect (or non-effect) will be used to inform the use of SAIA across varied local contexts and complex treatment cascades. We believe that our focus on testing the multicomponent SAIA-MH strategy to optimize the linked MH cascade rather than a single intervention on a single siloed health indicator is innovative. Furthermore, the SAIA strategy has longevity because, rather than testing a single intervention that may become irrelevant after policy, system, or technology changes, SAIA-MH packages protocols and tools for data-driven systems optimization appropriate for the changing landscape of health delivery systems across LMICs.

Extending breakthroughs of implementation strategies for HIV treatment cascade optimization to MH care is an efficient and promising method for scientific advancement. The original “Systems Analysis and Improvement Approach” (SAIA) implementation strategy improved HIV treatment cascade performance and was accessible/user-friendly for frontline workers/managers [27, 29, 95]. In our preliminary developmental/exploratory project, SAIA-MH showed strong promise for mental health treatment cascade gains [25]. While pharmacological treatment for MH has system idiosyncrasies compared to HIV, both MH and HIV are chronic illnesses requiring screening/identification in multiple settings, referral to a centralized treatment center with information systems enabling longitudinal care, intra- and inter-facility referrals for pharmacy, laboratory, or other support services, and facility-community linkages to manage case-finding, medication management, and patient follow-up. For these reasons, proven strategies developed for the HIV cascade are ideal scientific building blocks for the development of strategies for MH treatment cascade optimization. If effective, the SAIA-MH implementation strategy has a large potential to be rapidly scaled-up to decrease gaps in task-shared MH treatments globally.

Availability of data and materials

Data emanating from this study will be submitted to the National Institute of Mental Health Data Archive (NDA) at https://nda.nih.gov/edit_collection.html?id=3898.

Abbreviations

aOR:

Adjusted odds ratio

CFIR:

Consolidated Framework for Implementation Research

CI:

Confidence interval

CQI:

Continuous Quality Improvement

ERIC:

Expert Recommendations for Implementation Change

FGDs:

Focus Group Discussions

HIV:

Human immunodeficiency virus

IDIs:

In-depth interviews

LMIC:

Low- and middle-income-countries

LTFU:

Loss-to-follow-up

MH:

Mental health

MHCAT:

Mental Health Cascade Analysis Tool

RCT:

Randomized controlled trial

SAIA-MH:

Systems Analysis and Improvement Approach for Mental Health

SEM:

Structural equation modeling

TB:

Tuberculosis

WHODAS:

World Health Organization Disability Assessment Scale

YLD:

Years lived with disability

References

  1. Institute for Health Metrics and Evaluation (IHME). GBD Compare Data Visualization: Mozambique. 2017.

    Google Scholar 

  2. Patel V, Marino M, Flisher A, De Silva M, Koschorke M. Reducing the treatment gap for mental disorders: a WPA survey. World Psychiatry. 2010;9(3):169–76 http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=emed9&NEWS=N&AN=2011533889.

    Article  Google Scholar 

  3. Saxena S, Thornicroft G, Knapp M, Whiteford H. Resources for mental health: scarcity, inequity, and inefficiency. Lancet. 2007;370(9590):878–89. https://doi.org/10.1016/S0140-6736(07)61239-2.

    Article  PubMed  Google Scholar 

  4. Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care health. Bull World Health Organ. 2004;82(11):858–66. https://doi.org/10.1590/S0042-96862004001100011.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Kakuma R, Minas H, van Ginneken N, et al. Human resources for mental health care: current situation and strategies for action. Lancet. 2011;378(9803):1654–63. https://doi.org/10.1016/S0140-6736(11)61093-3.

    Article  PubMed  Google Scholar 

  6. World Health Organization. Task Shifting: Rational Redistribution of Tasks among Health Workforce Teams: Global Recommendations and Guidelines. 2008.

  7. Petersen I, Lund C, Bhana A, Flisher AJ. A task shifting approach to primary mental health care for adults in South Africa: human resource requirements and costs for rural settings. Health Policy Plan. 2012;27(1):42–51. https://doi.org/10.1093/heapol/czr012.

    Article  PubMed  Google Scholar 

  8. van Ginneken N, Tharayan P, Lwein S, et al. Non-specialist health worker interventions for the care of mental , neurological and substance-abuse disorders in low- and middle-income countries. Cochrane Database Syst Rev. 2013;(11):CD009149. https://doi.org/10.1002/14651858.CD009149.

  9. Buttorff C, Hock RS, Weiss HA, et al. Economic evaluation of a task-shifting intervention for common mental disorders in India. Bull World Health Organ. 2012;90(11):813–21. https://doi.org/10.2471/BLT.12.104133.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Patel V. The future of psychiatry in low- and middle-income countries. Psychol Med. 2009;39(11):1759. https://doi.org/10.1017/S0033291709005224.

    Article  CAS  PubMed  Google Scholar 

  11. Eaton J, McCay L, Semrau M, et al. Scale up of services for mental health in low-income and middle-income countries. Lancet. 2011;378(9802):1592–603. https://doi.org/10.1016/S0140-6736(11)60891-X.

    Article  PubMed  Google Scholar 

  12. Padmanathan P, De Silva MJ. The acceptability and feasibility of task-sharing for mental healthcare in low and middle income countries: a systematic review. Soc Sci Med. 2013;97:82–6. https://doi.org/10.1016/j.socscimed.2013.08.004.

    Article  PubMed  Google Scholar 

  13. World Health Organization. Mental Health Atlas 2014. 2014.

    Google Scholar 

  14. Araya R, Rojas G, Fritsch R, et al. Treating depression in primary care in low-income women in Santiago, Chile: a randomised controlled trial. Lancet. 2003;361(9362):995–1000. https://doi.org/10.1016/S0140-6736(03)12825-5.

    Article  PubMed  Google Scholar 

  15. Bolton P, Bass J, Verdeli H, Clougherty KF, Ndogoni L. Group interpersonal psychotherapy for depression in rural Uganda. JAMA. 2003;289(23):3117–24.

    Article  Google Scholar 

  16. Patel V, Chisholm D, Rabe-Hesketh S, Dias-Saxena F, Andrew G, Mann A. Efficacy and cost-effectiveness of drug and psychological treatments for common mental disorders in general health care in Goa, India: a randomised, controlled trial. Lancet. 2003;361(9351):33–9. https://doi.org/10.1016/S0140-6736(03)12119-8.

    Article  PubMed  Google Scholar 

  17. Patel V, Chisholm D, Dua T, Laxminarayan R, Medina-Mora ME. Disease control priorities, Third Edition (Volume 4): Mental, Neurological, and Substance Use Disorders; 2015. https://doi.org/10.1596/978-1-4648-0426-7.

    Book  Google Scholar 

  18. Dos Santos PF, Wainberg ML, Caldas-de-Almeida JM, Saraceno B, Mari JDJ. Overview of the mental health system in Mozambique: addressing the treatment gap with a task-shifting strategy in primary care. Int J Ment Health Syst. 2016;10:1. https://doi.org/10.1186/s13033-015-0032-8.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Fernandes QF, Wagenaar BH, Anselmi L, Pfeiffer J, Gloyd S, Sherr K. Effects of health-system strengthening on under-5, infant, and neonatal mortality: 11-year provincial-level time-series analyses in Mozambique. Lancet Glob Heal. 2014;2(8):e468–77. https://doi.org/10.1016/S2214-109X(14)70276-1.

    Article  Google Scholar 

  20. Wagenaar BH, Cumbe V, Raunig-Berho M, et al. Health facility determinants and trends of ICD-10 outpatient psychiatric consultations across Sofala, Mozambique: time-series analyses from 2012 to 2014. BMC Psychiatry. 2015;15(1):227. https://doi.org/10.1186/s12888-015-0609-4.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Sokhela NE, Uys LR. The integration of comprehensive psychiatric/mental health care into the primary health system: diagnosis and treatment. J Adv Nurs. 1999;30(1):229–37.

    Article  CAS  Google Scholar 

  22. Petersen I, Bhana A, Campbell-Hall V, et al. Planning for district mental health services in South Africa: a situational analysis of a rural district site. Health Policy Plan. 2009;24(2):140–50. https://doi.org/10.1093/heapol/czn049.

    Article  PubMed  Google Scholar 

  23. Couper I, Wright A, van Deventer C. Evaluation of primary mental helath care in north west province. Final Report; 2006.

    Google Scholar 

  24. Hanlon C, Wondimagegn D, Alem A. Lessons learned in developing community mental health care in Africa. World Psychiatry. 2010;9:185–9.

    Article  Google Scholar 

  25. Fabian KE, Muanido A, Cumbe VFJ, et al. Optimizing treatment cascades for mental healthcare in Mozambique: preliminary effectiveness of the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH). Health Policy Plan. 2020. https://doi.org/10.1093/heapol/czaa114.

  26. Acharya B, Swar SB. The role of consultant psychiatrists in ensuring high quality mental health care provision from non-specialists. Psychiatr Serv. 2016. https://doi.org/10.1176/appi.ps.670701.

  27. Gimbel S, Rustagi AS, Robinson J, et al. Evaluation of a systems analysis and improvement approach to optimize prevention of mother-to-child transmission of HIV using the Consolidated Framework for Implementation Research. J Acquir Immune Defic Syndr. 2016;72(Suppl 2):S108–16. https://doi.org/10.1097/QAI.0000000000001055.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Boaden R, Harvey G, Moxham C, Proudlove N. Quality Improvement: Theory and Practice in Healthcare; 2008.

    Google Scholar 

  29. Sherr K, Gimbel S, Rustagi A, et al. Systems analysis and improvement to optimize pMTCT (SAIA): a cluster randomized trial. Implement Sci. 2014;9(1):55. https://doi.org/10.1186/1748-5908-9-55.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1965;44(3):166–206.

    Article  Google Scholar 

  31. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):1–14. https://doi.org/10.1186/s13012-015-0209-1.

    Article  Google Scholar 

  32. Lau R, Stevenson F, Ong BN, et al. Achieving change in primary care-causes of the evidence to practice gap: Systematic reviews of reviews. Implement Sci. 2016;11(1). https://doi.org/10.1186/s13012-016-0396-4.

  33. Krause J, Van Lieshout J, Klomp R, et al. Identifying determinants of care for tailoring implementation in chronic diseases: An evaluation of different methods. Implement Sci. 2014;9(1):1–12. https://doi.org/10.1186/s13012-014-0102-3.

    Article  Google Scholar 

  34. Flottorp SA, Oxman AD, Krause J, et al. A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8(1). https://doi.org/10.1186/1748-5908-8-35.

  35. Wagner AD, Crocker J, Liu S, et al. Making smarter decisions faster: systems engineering to improve the global public health response to HIV. Curr HIV/AIDS Rep. 2019;16(4):279–91. https://doi.org/10.1007/s11904-019-00449-2.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Ritchie M, Dollar J, Miller C, et al. Implementation faciliation training manual: using implementation facilitation to improve care in the Veterans Health Administration. 2017.

    Google Scholar 

  37. Wagenaar BH, Hirschhorn LR, Henley C, et al. Data-driven quality improvement in low-and middle-income country health systems: Lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia. BMC Health Serv Res. 2017;17. https://doi.org/10.1186/s12913-017-2661-x.

  38. Wagner AD, Gimbel S, Ásbjörnsdóttir KH, et al. Cascade analysis: an adaptable implementation strategy across HIV and non-HIV delivery platforms. J Acquir Immune Defic Syndr. 2019;82. https://doi.org/10.1097/QAI.0000000000002220.

  39. Gimbel S, Voss J, Mercer MA, et al. The prevention of mother-to-child transmission of HIV cascade analysis tool: supporting health managers to improve facility-level service delivery. BMC Res Notes. 2014;7:743. https://doi.org/10.1186/1756-0500-7-743.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Aycock DM, Hayat MJ, Helvig A, Dunbar SB, Clark PC. Essential considerations in developing attention control groups in behavioral research. Res Nurs Health. 2018;41(3):320–8. https://doi.org/10.1002/nur.21870.

    Article  PubMed  Google Scholar 

  41. Shadish WRS, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company; 2002.

  42. Wagenaar BH, Cumbe V, Raunig-Berhó M, et al. Outpatient mental health services in Mozambique: use and treatments. Psychiatr Serv. 2016. https://doi.org/10.1176/appi.ps.201500508.

  43. Wagenaar BH, Stergachis A, Rao D, et al. The availability of essential medicines for mental healthcare in Sofala, Mozambique. Glob Health Action. 2015;8(1). https://doi.org/10.3402/gha.v8.27942.

  44. Zhang Y, Augusto O, Ásbjörnsdóttir K, et al. Geographic distribution and determinants of mental health stigma in central Mozambique. Soc Psychiatry Psychiatr Epidemiol. 2019;(0123456789). https://doi.org/10.1007/s00127-019-01708-8.

  45. Sherr K, Cuembelo F, Michel C, et al. Strengthening integrated primary health care in Sofala, Mozambique. BMC Health Serv Res. 2013;13(Suppl 2):S4. https://doi.org/10.1186/1472-6963-13-S2-S4.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Ustun TB, Kostanjsek N, Chatterji S, Rehm JT. Measuring health and disability, manual for WHO disability assessment schedule (WHODAS 2.0); 2010.

    Google Scholar 

  47. Federici S, Bracalenti M, Meloni F, Luciano JV. World Health Organization disability assessment schedule 2.0: An international systematic review. Disabil Rehabil. 2017;39(23):2347–80. https://doi.org/10.1080/09638288.2016.1223177.

    Article  PubMed  Google Scholar 

  48. Habtamu K, Alem A, Medhin G, et al. Validation of the World Health Organization Disability Assessment Schedule in people with severe mental disorders in rural Ethiopia. Health Qual Life Outcomes. 2017;15(1):64. https://doi.org/10.1186/s12955-017-0647-3.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Ustün TB, Chatterji S, Kostanjsek N, et al. Developing the World Health Organization Disability Assessment Schedule 2.0. Bull World Health Organ. 2010;88(11):815–23. https://doi.org/10.2471/BLT.09.067231.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Walters CB, Schwalbach T, Sevene E, et al. Chronic pain and associated factors in Maputo, Mozambique: a pilot study. Br J Pain. 2019;13(1):43–53. https://doi.org/10.1177/2049463718777009.

    Article  PubMed  Google Scholar 

  51. McKibbin C, Patterson TL, Jeste DV. Assessing disability in older patients with schizophrenia: results from the WHODAS-II. J Nerv Ment Dis. 2004;192(6):405–13.

    Article  Google Scholar 

  52. Guilera G, Gomez-Benito J, Pino O, et al. Utility of the World Health Organization Disability Assessment Schedule II in schizophrenia. Schizophr Res. 2012;138(2-3):240–7. https://doi.org/10.1016/j.schres.2012.03.031.

    Article  PubMed  Google Scholar 

  53. Hernandez-Orduna O, Robles-Garcia R, Martinez-Lopez N, et al. WHODAS and the evaluation of disability among people with mental disorders with and without psychotic symptomsd. Salud Ment. 2017;40(5):209–17.

    Article  Google Scholar 

  54. Hanlon C, Medhin G, Selamu M, et al. Impact of integrated district level mental health care on clinical and social outcomes of people with severe mental illness in rural Ethiopia: an intervention cohort study. Epidemiol Psychiatr Sci. 2019;29:e45. https://doi.org/10.1017/S2045796019000398.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Gureje O, Oladeji BD, Montgomery AA, et al. Effect of a stepped-care intervention delivered by lay health workers on major depressive disorder among primary care patients in Nigeria (STEPCARE): a cluster-randomised controlled trial. Lancet Glob Heal. 2019;7(7):e951–60. https://doi.org/10.1016/S2214-109X(19)30148-2.

    Article  Google Scholar 

  56. Lund C, Schneider M, Garman EC, et al. Task-sharing of psychological treatment for antenatal depression in Khayelitsha, South Africa: effects on antenatal and postnatal outcomes in an individual randomised controlled trial. Behav Res Ther. 2019:103466. https://doi.org/10.1016/j.brat.2019.103466.

  57. Jordans MJD, Luitel NP, Garman E, et al. Effectiveness of psychological treatments for depression and alcohol use disorder delivered by community-based counsellors: two pragmatic randomised controlled trials within primary healthcare in Nepal. Br J Psychiatry. 2019;215(2):485–93. https://doi.org/10.1192/bjp.2018.300.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Weobong B, Weiss H, McDaid D, et al. Sustained effectiveness and cost-effectiveness of the Healthy Activity Programme, a brief psychological treatment for depression delivered by lay counsellors in primary care: 12-month follow-up of a randomised controlled trial. PLoS Med. 2017;14(9 PG-e1002385):e1002385. https://doi.org/10.1371/journal.pmed.1002385.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Petersen I, Bhana A, Folb N, et al. Collaborative care for the detection and management of depression among adults with hypertension in South Africa: study protocol for the PRIME-SA randomised controlled trial. Trials. 2018;19:192. https://doi.org/10.1186/s13063-018-2518-6.

  60. Lorenz E, Gabrysch S. Covariate-constrained randomization routine for achieving baseline balance in cluster-randomized trials. Stata J. 2017;17(2):503–10. https://doi.org/10.1177/1536867x1701700214.

    Article  Google Scholar 

  61. Dimagi. Dimagi: CommCare App. 2021. Accessed 15 Nov 2021. https://www.dimagi.com/commcare/

    Google Scholar 

  62. Tableau. Tableau Software. 2021. Accessed 15 Nov 2021. https://www.tableau.com/

    Google Scholar 

  63. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: A tutorial. Int J Epidemiol. 2017;46(1):348–55. https://doi.org/10.1093/ije/dyw098.

    Article  PubMed  Google Scholar 

  64. Xiao H, Augusto O, Wagenaar BH. Reflection on modern methods: a common error in the segmented regression parameterization of interrupted time-series analyses. Int J Epidemiol. 2021;50(3):1011–5. https://doi.org/10.1093/ije/dyaa148.

    Article  PubMed  Google Scholar 

  65. Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane database Syst Rev. 2003;(3):CD000259. https://doi.org/10.1002/14651858.CD000259.

  66. Hysong SJ. Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care. 2009;47(3):356–63. https://doi.org/10.1097/MLR.0b013e3181893f6b.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Tuti T, Nzinga J, Njoroge M, et al. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implement Sci. 2017;12(1):61. https://doi.org/10.1186/s13012-017-0590-z.

    Article  PubMed  PubMed Central  Google Scholar 

  68. Colquhoun HL, Brehaut JC, Sales A, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8:66. https://doi.org/10.1186/1748-5908-8-66.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Colquhoun HL, Carroll K, Eva KW, et al. Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses. Implement Sci. 2017;12(1):117. https://doi.org/10.1186/s13012-017-0646-0.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Wells S, Tamir O, Gray J, Naidoo D, Bekhit M, Goldmann D. Are quality improvement collaboratives effective? A systematic review. BMJ Qual Saf. 2018;27(3):226–40. https://doi.org/10.1136/bmjqs-2017-006926.

    Article  PubMed  Google Scholar 

  71. Hargreaves S, Rustage K, Nellums LB, et al. Do quality improvement initiatives improve outcomes for patients in antiretroviral programs in low- and middle-income countries? A Systematic Review. J Acquir Immune Defic Syndr. 2019;81(5):487–96. https://doi.org/10.1097/QAI.0000000000002085.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298(9):1023–37. https://doi.org/10.1001/jama.298.9.1023.

    Article  CAS  PubMed  Google Scholar 

  73. Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–94. https://doi.org/10.1111/milq.12016.

    Article  PubMed  PubMed Central  Google Scholar 

  74. Hempel S, O’Hanlon C, Lim YW, Danz M, Larkin J, Rubenstein L. Spread tools: a systematic review of components, uptake, and effectiveness of quality improvement toolkits. Implement Sci. 2019;14(1):83. https://doi.org/10.1186/s13012-019-0929-8.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff (Millwood). 2005;24(1):138–50. https://doi.org/10.1377/hlthaff.24.1.138.

    Article  PubMed  Google Scholar 

  76. Cranley LA, Cummings GG, Profetto-McGrath J, Toth F, Estabrooks CA. Facilitation roles and characteristics associated with research use by healthcare professionals: a scoping review. BMJ Open. 2017;7(8):e014384. https://doi.org/10.1136/bmjopen-2016-014384.

    Article  PubMed  PubMed Central  Google Scholar 

  77. Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012;10(1):63–74. https://doi.org/10.1370/afm.1312.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45(5):626–9. https://doi.org/10.1016/j.jpsychires.2010.10.008.The.

    Article  PubMed  Google Scholar 

  79. Lynn HS. Suppression and Confounding in Action. Am Stat. 2003;57(1):58–61. https://doi.org/10.1198/0003130031090.

    Article  Google Scholar 

  80. Wagenaar BH, Gimbel S, Hoek R, et al. Wait and consult times for primary healthcare services in central Mozambique: a time-motion study. Glob Health Action. 2016;9(1):31980. https://doi.org/10.3402/gha.v9.31980.

    Article  PubMed  Google Scholar 

  81. Chisholm D, Sanderson K, Ayuso-Mateos JL, Saxena S. Reducing the global burden of depression: population-level analysis of intervention cost-effectiveness in 14 world regions. Br J Psychiatry. 2004;184:393–403. https://doi.org/10.1192/bjp.184.5.393.

    Article  PubMed  Google Scholar 

  82. Jack H, Wagner RG, Petersen I, et al. Closing the mental health treatment gap in South Africa: a review of costs and cost-effectiveness. Glob Health Action. 2014;7:23431. https://doi.org/10.3402/gha.v7.23431.

    Article  PubMed  Google Scholar 

  83. Chisholm D. Cost-effectiveness of first-line antiepileptic drug treatments in the developing world: a population-level analysis. Epilepsia. 2005;46(5):751–9. https://doi.org/10.1111/j.1528-1167.2005.52704.x.

    Article  PubMed  Google Scholar 

  84. Chisholm D, van Ommeren M, Ayuso-Mateos J-L, Saxena S. Cost-effectiveness of clinical interventions for reducing the global burden of bipolar disorder. Br J Psychiatry. 2005;187:559–67. https://doi.org/10.1192/bjp.187.6.559.

    Article  PubMed  Google Scholar 

  85. Chisholm D, Gureje O, Saldivia S, et al. Schizophrenia treatment in the developing world: an interregional and multinational cost-effectiveness analysis. Bull World Health Organ. 2008;86(7):542–51. https://doi.org/10.2471/blt.07.045377.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Siskind D, Baingana F, Kim J. Cost-effectiveness of group psychotherapy for depression in Uganda. J Ment Health Policy Econ. 2008;11(3):127–33.

    PubMed  Google Scholar 

  87. McBain RK, Salhi C, Hann K, Salomon JA, Kim JJ, Betancourt TS. Costs and cost-effectiveness of a mental health intervention for war-affected young persons: decision analysis based on a randomized controlled trial. Health Policy Plan. 2016;31(4):415–24. https://doi.org/10.1093/heapol/czv078.

    Article  PubMed  Google Scholar 

  88. Murray CJ. Quantifying the burden of disease: the technical basis for disability-adjusted life years. Bull World Health Organ. 1994;72(3):429–45.

    CAS  PubMed  PubMed Central  Google Scholar 

  89. GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet (London, England). 2018;392(10159):1789–858. https://doi.org/10.1016/S0140-6736(18)32279-7.

    Article  Google Scholar 

  90. Powell BJ, Proctor EK, Glass JE. A systematic review of strategies for implementing empirically supported mental health interventions. Res Soc Work Pract. 2014;24(2):192–212. https://doi.org/10.1177/1049731513505778.

    Article  PubMed  Google Scholar 

  91. Lewis CC, Klasnja P, Powell BJ, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Heal. 2018;6(May). https://doi.org/10.3389/fpubh.2018.00136.

  92. Williams NJ. Multilevel mechanisms of implementation strategies in mental health: integrating theory, research, and practice. Adm Policy Ment Heal Ment Heal Serv Res. 2016;43(5):783–98. https://doi.org/10.1007/s10488-015-0693-2.

    Article  Google Scholar 

  93. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x.

    Article  PubMed  PubMed Central  Google Scholar 

  94. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. https://doi.org/10.1186/1748-5908-4-50.

    Article  PubMed  PubMed Central  Google Scholar 

  95. Rustagi AS, Gimbel S, Nduati R, et al. Implementation and operational research: impact of a systems engineering intervention on PMTCT service delivery in Côte d’Ivoire, Kenya, Mozambique: a cluster randomized trial. J Acquir Immune Defic Syndr. 2016;72(3). https://doi.org/10.1097/QAI.0000000000001023.

Download references

Acknowledgements

We thank Pamela Collins and Ruanne Barnabas for serving as senior advisors for this study. We thank the Mozambican National Ministry of Health and the Provincial Health Departments of Sofala and Manica Provinces for continued support and collaboration throughout this study.

Funding

This work was supported by grant number R01MH123682 from the National Institute of Mental Health (NIMH). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Author notes

  1. Vasco F.J. Cumbe and Alberto Gabriel Muanido are co-first authors.

Authors and Affiliations

  1. Provincial Health Directorate, Sofala Province, Ministry of Health, Beira, Mozambique

    Vasco F. J. Cumbe & Ernesto Rodrigo Xerinda

  2. Faculty of Medicine, Eduardo Mondlane University, Maputo, Mozambique

    Vasco F. J. Cumbe

  3. Department of Psychiatry, Beira Central Hospital, Beira, Mozambique

    Vasco F. J. Cumbe

  4. Mozambican Health Committee, Beira, Mozambique

    Alberto Gabriel Muanido & Isaias Ramiro

  5. Department of Global Health, University of Washington, Seattle, WA, USA

    Morgan Turner, Kenneth Sherr, Bryan J. Weiner, Monisha Sharma & Bradley H. Wagenaar

  6. Department of Epidemiology, University of Washington, Seattle, WA, USA

    Kenneth Sherr & Bradley H. Wagenaar

  7. Department of Industrial & Systems Engineering, University of Washington, Seattle, WA, USA

    Kenneth Sherr

  8. Department of Psychology, University of Washington, Seattle, WA, USA

    Brian P. Flaherty

  9. Provincial Health Directorate, Manica Province, Ministry of Health, Chimoio, Mozambique

    Flávia Faduque

Authors
  1. Vasco F. J. Cumbe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alberto Gabriel Muanido
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Morgan Turner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Isaias Ramiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kenneth Sherr
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bryan J. Weiner
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Brian P. Flaherty
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Monisha Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Flávia Faduque
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Ernesto Rodrigo Xerinda
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Bradley H. Wagenaar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

VFJC, AGM, and BHW conceived of the study. VFJC, AGM, IR, KS, BW, BF, MS, FF, ERX, and BHW contributed to the study design and protocol. IR, AGM, VFJC, ERX, IR, and FF advised on local conditions. VFJC, AGM, KS, BW, BF, MS, and BHW contributed to methodology. VFJC, FF, AGM, BHW, EX, and IR contributed to supervision. VFJC, AGM, BHW, and MT contributed to writing first draft. All authors reviewed the original draft and provided scientific input and editing for the final draft. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Vasco F. J. Cumbe.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the institutional review boards at the University of Washington and the Mozambican National Committee of Bioethics in Health, Ministry of Health (CNBS) with registration IRB00002657.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cumbe, V.F.J., Muanido, A.G., Turner, M. et al. Systems analysis and improvement approach to optimize outpatient mental health treatment cascades in Mozambique (SAIA-MH): study protocol for a cluster randomized trial. Implementation Sci 17, 37 (2022). https://doi.org/10.1186/s13012-022-01213-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-022-01213-8

Keywords

Implementation Science

ISSN: 1748-5908