This article has Open Peer Review reports available.
Integrating addiction treatment into primary care using mobile health technology: protocol for an implementation research study
© Quanbeck et al.; licensee BioMed Central Ltd. 2014
Received: 17 February 2014
Accepted: 17 May 2014
Published: 29 May 2014
Healthcare reform in the United States is encouraging Federally Qualified Health Centers and other primary-care practices to integrate treatment for addiction and other behavioral health conditions into their practices. The potential of mobile health technologies to manage addiction and comorbidities such as HIV in these settings is substantial but largely untested. This paper describes a protocol to evaluate the implementation of an E-Health integrated communication technology delivered via mobile phones, called Seva, into primary-care settings. Seva is an evidence-based system of addiction treatment and recovery support for patients and real-time caseload monitoring for clinicians.
Our implementation strategy uses three models of organizational change: the Program Planning Model to promote acceptance and sustainability, the NIATx quality improvement model to create a welcoming environment for change, and Rogers’s diffusion of innovations research, which facilitates adaptations of innovations to maximize their adoption potential. We will implement Seva and conduct an intensive, mixed-methods assessment at three diverse Federally Qualified Healthcare Centers in the United States. Our non-concurrent multiple-baseline design includes three periods — pretest (ending in four months of implementation preparation), active Seva implementation, and maintenance — with implementation staggered at six-month intervals across sites. The first site will serve as a pilot clinic. We will track the timing of intervention elements and assess study outcomes within each dimension of the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework, including effects on clinicians, patients, and practices. Our mixed-methods approach will include quantitative (e.g., interrupted time-series analysis of treatment attendance, with clinics as the unit of analysis) and qualitative (e.g., staff interviews regarding adaptations to implementation protocol) methods, and assessment of implementation costs.
If implementation is successful, the field will have a proven technology that helps Federally Qualified Health Centers and affiliated organizations provide addiction treatment and recovery support, as well as a proven strategy for implementing the technology. Seva also has the potential to improve core elements of addiction treatment, such as referral and treatment processes. A mobile technology for addiction treatment and accompanying implementation model could provide a cost-effective means to improve the lives of patients with drug and alcohol problems.
This paper describes the protocol for a study on implementing mobile-health technology in primary care. The study is funded by the United States’ National Institute on Drug Abuse as part of a request for proposals aimed at integrating addiction treatment into primary care. Although numerous smartphone applications exist that address many health problems, primary care clinicians do not commonly incorporate mobile technology into patient care. One reason is that very few smartphone applications have been proven in rigorous trials of effectiveness. Even if an application does have a reliable evidence base, it must still demonstrate its value to various stakeholders if it is to be widely used. Patients must see value in the application or they are likely to stop using it. Clinicians must find it useful to incorporate it in day-to-day clinical management. Finally, clinic administrators must find that the application works both clinically and financially to support its use by patients and staff.
Our study focuses on Federally Qualified Health Centers (FQHCs) in the United States, community-based organizations that provide primary care and preventive health services to patients regardless of their ability to pay. FQHCs (and U.S. primary-care practices in general) are increasingly being encouraged by healthcare reform to integrate treatment for addiction and behavioral health into their practices. HIV testing and prevention efforts are particularly important for the health of men and women who use illicit drugs because they are at higher risk than the general population for acquiring HIV[2, 3]. But transforming primary care to address addiction and HIV will likely add to the work of clinicians, many of whom already feel overburdened. Previous attempts to integrate addiction treatment into primary care have largely relied on labor-intensive solutions (e.g., adding more staff or having staff do tasks differently), with mixed results.
Our premise is that mobile health technology holds great promise for FQHCs to reach this complex patient population. More than two-thirds of individuals with substance use disorders (SUDs) have had contact with a primary-care provider in the previous six months, for reasons that may or may not relate to their SUD. But primary-care providers frequently do not screen for or treat SUDs, even though recovery support can be effectively administered in primary care[6, 7]. Administering recovery support requires that clinicians receive support and training in addiction, which are uncommon partly because of the traditional separation of addiction treatment from primary care[8–10]. Technology may help address these problems. People with addictions tend to view technology favorably. They acknowledge more drug use and psychiatric symptoms online than in face-to-face interviews. A recent review found positive outcomes in 29 of 32 randomized trials using personal computers and cell phones for managing chronic diseases. While some information technologies have proven efficacious, penetration into practice has been slow. The potential of mobile health technologies to manage chronic diseases in FQHCs is substantial, but remains largely untested.
The mobile-health technology
The integrated system we have developed is an E-Health integrated communication technology for patients with addiction called Seva (say’-va; a Sanskrit word for selfless caring). Seva consists of treatment and recovery support elements. The treatment component is based on the Therapeutic Education System (TES), a web- and mobile-based curriculum for addiction treatment with 65 interactive, multimedia, skill-building modules, including basic cognitive behavioral recovery support skills (e.g., refusing drugs, managing thoughts about drug use), life re-structuring skills (e.g., increasing recreational activities), and skills for preventing HIV, hepatitis, and sexually transmitted infections. Each module ends with a quiz structured to ensure mastery of key content and skills. TES uses interactive videos to create experiential learning environments that help patients learn modeled behaviors (e.g., progressive muscle relaxation). Randomized trials have found that TES is as efficacious as behavioral therapy delivered by therapists and is superior to standard substance abuse treatment; that it increases HIV prevention knowledge and self-reported risk for HIV; and when it partially replaces clinician-delivered care in methadone treatment, it also improves opioid abstinence rates.
Seva’s recovery support elements are based on A-CHESS, an evidence-based smartphone program designed to help people in recovery from drug and alcohol addiction prevent relapse. A-CHESS helps patients meet the challenges they often face, such as loneliness and isolation, transportation, managing the treatment regimen, and getting informal support. A-CHESS also addresses issues such as cravings and insufficient coping skills in high-risk situations. A-CHESS is based on research on the principles of effective continuing care for substance use disorders (long duration, assertive outreach, monitoring, prompts, action planning, peer and family support, and case management). The research team that developed A-CHESS has studied and designed similar systems that have been effective for patients suffering from lung cancer, breast cancer, asthma, and HIV[2, 3, 18–36]. In a randomized trial of patients discharged from residential addiction treatment, patients with A-CHESS had 57% fewer risky drinking days than patients in the control group.
Although Seva was built from two existing programs, TES and A-CHESS, the Seva user experiences one seamless system with many links between elements. For example, upon completing a skill-building module, the patient may be shown a link to a discussion group where peers are discussing the topic of the module, and provided links to further information on the topic. Similarly, patients whose responses to check-in items indicate a potential problem may see a link to skill-building related to that problem, or be shown personal stories from other patients who have dealt with the same problem. Patient-entered data are also uploaded and presented via a web-based dashboard that FQHC clinicians can use to monitor the status of their Seva users. The dashboard includes items that patients self-report using their mobile devices, such as sleep troubles and relationship problems that may signal an imminent relapse; whether the patient has attended recovery meetings or completed other elements of the treatment plan; and the patient’s relapse risk and protection factors. The dashboard can be customized for each clinic to display indicators that clinicians deem important for clinical decision-making.
Study focus: implementation
Because Seva has an established evidence base, this research focuses mainly on implementation and organizational impact; some patient-level outcomes are included, but they will be examined in the aggregate. Our goal is to implement a system that has already proven effective in specialty addiction treatment in primary-care settings. The implementation model we are testing (detailed below) is distinguished by several characteristics: an effort to actively prepare the environment for implementation; the use of an organizational coach; the use of rapid-cycle tests; and an early, specific effort to address sustainability. The implementation model is rooted in the field of systems engineering, a multi-disciplinary field that integrates concepts from psychology, economics, social science, and statistics. The implementation model reflects the multi-disciplinary ethic of systems engineering.
We will study organizational change resulting from implementing Seva in 3 FQHCs over a three-year period. We will study elements of our implementation model by using mixed-methods analysis. Our study seeks answers to three broad research questions: how can Seva be implemented in primary care settings efficiently and effectively; to what extent do patients and staff accept and use Seva; and how does Seva affect clinical care for patients and staff?
The model has 4 phases: i) ‘Initiate’ involves bringing together and training key people at each site; ii) ‘Prepare’ ensures that the FQHC environment is welcoming to Seva, the implementation plan adapts to the unique characteristics of the FQHC, and the technology is developed to maximize its chance of success; iii) ‘Improve’ involves conducting a series of rapid-cycle tests of ideas that emerge from earlier phases; and iv) ‘Implement’ involves using, monitoring and sustaining the technology. Organizational coaching will play a key role in helping clinics implement Seva successfully. We recently completed a randomized trial in 201 addiction treatment agencies in which coaching proved to be the most cost-effective approach to quality improvement among several strategies tested. An implementation coach will work with each FQHC to advise, encourage and monitor progress.
Settings for implementation
We worked with the National Association of Community Health Centers to identify a set of FQHCs with established electronic health records. We focused on FQHCs with electronic health records because healthcare reform has been pushing FQHCs in this direction, and we believe it will be important to understand how Seva relates to existing clinic technology. From the pool of FQHCs that met these criteria, we selected an FQHC affiliated with the University of Wisconsin as our first implementation site; as a second site, a relatively small, rural, freestanding FQHC with integrated behavioral health services (including addiction treatment); and as a third site, an urban FQHC that serves a largely minority population. In selecting sites, we strove to maximize diversity in both patient populations and organizational structures to better understand how environment affects implementation. We included only three FQHCs in order to conduct an intensive, mixed-methods assessment at each clinic, which we believe will be more informative for the ultimate purpose of dissemination than randomizing a larger sample of organizations and studying implementation at a surface level.
Coaching for organizational change
The organizational coach for the first (pilot) site will be a member of the research team with extensive experience in organizational coaching. The next two sites will be coached by a physician from the pilot site, who will share firsthand experience in implementing Seva with subsequent clinics. Each site will have a designated ‘change leader’, a clinical leader at the site who is the point of contact with the coach and will coordinate implementation activities at the clinic; a ‘site coordinator’, a clinic employee who will enroll patients in the study and manage the day-to-day operation of Seva; and a ‘change team’, a group of four to eight clinical and/or administrative staff members who will help identify and make the organizational changes necessary to implement Seva. The coach will make an initial site visit during the four-month period of active implementation preparation to create a welcoming environment for Seva. During this visit, the coach will conduct a walk-through with change team members (an exercise in which employees experience the clinic as patients do, to reveal issues that need to be addressed); a work flow assessment using flow charting; and a technical assessment of data to be gathered and procedures needed to conduct the study. During this preparation period, the coach will also work with the change leader to ensure that pre-test data are collected; reviews of Seva are completed by clinical teams and their concerns addressed; and barriers to implementation rectified. After the initial site visit, for the remainder of the implementation preparation period and for the 12-month intervention period, the coach will have monthly telephone conferences with clinic staff and email correspondence as necessary to monitor implementation progress and offer advice and feedback to the staff. The coach and change leader will also work together to reduce organizational barriers to implementation.
Our study uses a non-concurrent multiple-baseline design in which the start of the intervention period is staggered at the three sites by six months. The multiple-baseline design provides the intervention to all participants, but isolates the independent variable by introducing the intervention at specified points. Using the clinic as the unit of analysis, this single-subject approach will closely track the timing of intervention elements and repeatedly measure outcomes at the organizational level, including aggregated effects on clinicians, patients and practices. After pilot testing Seva at the first FQHC and improving Seva and the overall implementation strategy, we will introduce Seva at the second FQHC and then, six months later, at the third FQHC. Each FQHC acts as its own pretest control. During the 48-month data collection period, core organizational indicators will be measured via a staff survey (administered every 6 months) and the clinics’ electronic health records (aggregated across patients). While the non-concurrent multiple-baseline design may not provide as strong a basis for causal inference as a large RCT, we believe it is the appropriate design for evaluating the organizational impact of a system that has already been shown effective in RCTs of patients. We will also collect a set of standard CHESS/TES measures via a patient survey for comparison to previous studies that tested these technologies in RCTs, and to conduct exploratory analyses. The patient survey also provides supplemental clinical information not routinely collected in the clinics (e.g., addiction severity, quality of life, etc.).
Number of Seva patients (eligible, excluded, enrolled)
Characteristics of participating patients
Qualitative assessment- reach
Interview- clinic director
No. patients screened for HIV
Healthcare utilization (hospitalizations, ER visits, and residential addiction treatment)
0 m, 6 m, 12 m, 18 m
0 m, 6 m, 12 m, 18 m
HIV risk behaviors
0 m, 6 m, 12 m, 18 m
0 m, 6 m, 12 m, 18m
Quality of life
0 m, 6 m, 12 m, 18 m
Qualitative assessment - effectiveness
Interview - clinic director
Characteristics of participating clinics vs. general FQHC population
Publicly available uniform data system reports for FQHCs
Readiness for implementation
Pre, every 6 m
Use of Seva by staff (including characteristics)
Seva log files
Use of Seva by patients (including characteristics)
Seva log files
Adoption (staff and patient)
Qualitative assessment- adoption
Pre, every 6 m
Stages of Implementation Completion
Pre, every 6 m
Pre, every 6 m
Adaptations to protocol during intervention period
Pre, every 6 m
Cost of intervention
Pre, every 6 m
Qualitative assessment - implementation
Interview with clinic director, coach
Pre, every 6 m
Six-month follow-up on all effectiveness measures (see above) and use of Seva
Seva, Patient survey, EHR
1, 2, 3
Qualitative assessment - maintenance
Interview - clinic director
In assessing reach, we will determine how many patients are eligible to use Seva within each clinic, how many are excluded, how many are invited to participate, and how many enroll in the study. We will also study how quickly patients are enrolled after the beginning of the enrollment period. We will assess how representative Seva enrollees are of the target population (substance using patients within each clinic) by comparing users to the target population on key clinical characteristics (e.g., treatment attendance, healthcare utilization). Qualitative assessment will supplement our data collection on reach. Our goal is to understand how clinicians perceive or anticipate which patients may benefit most from the system, because — for budget reasons — not all patients can have Seva, either in this study or in a broader dissemination. (In this study, we can offer Seva to a maximum of 100 patients per clinic.) For example, clinicians may select patients to enroll who they think will be strong models for other patients, or patients who have had many relapses and may benefit from a different treatment approach. As an implementation-focused study, we strive not for random selection but instead encourage clinicians to enroll patients for whom they believe Seva will be most helpful.
Our primary effectiveness outcome is substance use. Other outcomes in the effectiveness domain will help us understand the impact of Seva on patients’ quality of life, HIV risk behaviors, healthcare utilization, and treatment attendance. We will also examine the extent to which the system promotes an overall increase in the proportion of substance-use patients screened for HIV. Healthcare utilization data (hospitalization, emergency room visits, and in-patient psychiatric treatment) will also be used in the cost analysis described below. Qualitative assessment of effectiveness will take place through an interview with the clinic director.
In assessing adoption at the clinic level, we will report how many FQHCs were approached to participate and how many signed on to implement Seva. We will compare organizational characteristics of participating FQHCs to the more than 1,000 FQHCs in the United States, based on public reports required of these organizations by the federal government. We will gather qualitative data to assess which aspects of organizational structure affect implementation. To assess adoption at the staff level, clinic site coordinators will make a list of all staff members who see potential substance-using patients in primary or behavioral care, and invite these staff to complete surveys six months prior to the implementation period and every six months thereafter. The surveys include questions on job function (i.e., physician, behavioral health staff, or RN), which will allow us to assess how representative the respondents are of the clinic staff as a whole, and the Readiness for Implementation Survey, an instrument used to predict the adoption success of information technologies in healthcare. Staff members and patients will take part in qualitative interviews to supplement data provided in the surveys to assess organizational barriers and facilitators related to implementation. We will analyze Seva log files to produce metrics about patterns of system adoption and use by patients and staff and assess the characteristics of individuals who use Seva most and least actively.
To measure implementation progress, we will adapt our implementation model into a specific implementation plan (including timetables) for each clinic. We will employ the Stages of Implementation Completion model as a guide to assess the degree of implementation. Each phase of the implementation completion model is broken down into discrete tasks and events and presented as a checklist. We will assess whether each task/event was implemented, and the length of time each one took to implement. Starting at the beginning of the implementation-planning period (four months prior to implementation), the checklist will be reviewed and updated every six months by the site coordinator in conjunction with the implementation coach. Adaptations to the protocol will be noted. We will also assess implementation costs (see below) and administer the Technology Acceptance Model, which helps explain factors that influence the use of a new technology. Qualitative assessment of the consistency of the intervention will occur through interviews with the clinic director and the implementation coach.
To assess maintenance, we will administer the British National Health Service’s Sustainability Index. The Sustainability Index assesses factors associated with sustainability potential, such as fit with organizational infrastructure, clinical leadership engagement, and progress monitoring. The Index also offers guidance, developed by an international panel of experts, on overcoming weaknesses. Factors measured by the Sustainability Index, along with those included in the Readiness for Implementation Survey, will guide us in building a conceptual model of barriers and facilitators to implementation. Assessing maintenance at the individual level includes monitoring of effectiveness measures (including substance use and quality of life) at the 18-month mark, 6 months after grant funding for phones and internet access ends. We will examine data on Seva use at the staff and patient levels to assess long-term maintenance of Seva. Qualitative interviews with clinic directors will assess the degree of institutionalization of Seva within organizations, how Seva aligns with clinical business models, and whether and how Seva was adapted after the active implementation period to promote maintenance.
We will use a mixed-method approach, using both quantitative and qualitative methods, to assess our implementation model. Our analysis follows guidelines for mixed-methods research promulgated by the NIH Office of Behavioral and Social Sciences Research. Because this is a study of the organizational impact of Seva, there will be no single-patient analyses; all analyses will be aggregated across patients.
Descriptive statistics will be used to assess relevant Reach outcomes.
Interrupted time series analysis will compare selected Effectiveness and Maintenance outcomes within a clinic before and after implementation. We will assess whether our implementation model promotes the successful installation and use of Seva using linear regression models, with each analysis having two time-series segments representing means and slopes before and after implementation. A test of the regression coefficient for implementation (a dichotomous variable indicating whether the time point occurred before or after implementation) will allow us to examine differences in the mean change at implementation. Differences in slope from pre- to post-implementation will be tested using the coefficient for the implementation x month interaction, where ‘month’ is a continuous variable indicating the month of measurement. Weighted least squares analysis will be used to account for potential differences in the number of patients or staff reporting outcomes within each clinic across time. Given the limited degrees of freedom, no covariates will be included in the models.
Drawing from conventional approaches to single-subject analysis, we will also compare selected RE-AIM outcomes within and across the clinics using data visualization  to determine whether the shifts between intervention phases of the study generated shifts in the level, trend, and variability of performance . This will allow us to evaluate the immediacy, consistency and magnitude of the observed effects .
Understand the environments in which Seva is implemented, how Seva was tailored to fit each environment, and the processes used to tailor Seva. Focus groups and observational and interview data will help us understand the environment and the processes used to tailor Seva.
Provide insights into the quantitative results. We will interview key stakeholders following each wave of surveys. As within- and across-site findings emerge, we will interview people who can help inform the findings.
Understand the barriers to and facilitators of implementing and sustaining Seva and how they relate to outcomes. Key stakeholders will be asked to reflect on reach, outcomes, adoption at the setting level, staff participation, implementation, and long-term effects.
Prompt informants to reflect upon how they might advise a similar FQHC to integrate Seva.
We will collect, transcribe and analyze field notes and interviews collected during each implementation phase using AtlasTi. The qualitative analysis will follow grounded theory to build a conceptual model of Seva’s implementation. Two researchers will independently analyze interviews that represent different organization-level stakeholders, and then meet to discuss and reconcile differences in interpretation. Because Readiness for Implementation Survey and Sustainability Index factors are associated with adoption and sustainability, we will include these factors as a priori themes, while seeking new insights generated from the interviews. Researchers will then code the remaining interviews, comparing themes to distinguish different types of organizations and conditions. To assure coding consistency, every fourth interview will be double-coded. Significant inconsistencies will be discussed and recoded. Finally, we will categorize the themes into a working model.
The sustainability of Seva will depend on both its effects and its cost. Our cost analysis is conducted from the adopting FQHC’s perspective and is intended to provide estimates of the costs to implement and operate Seva within an FQHC. The cost data collection instrument follows guidelines for economic evaluation of implementation strategies. It has been adapted from a cost collection instrument we used in a previous quality improvement study. The cost collection instrument will track fixed costs of implementation, including system development costs and research costs related to development of the implementation strategy. The analysis separates these one-time, fixed costs from variable costs related to executing the implementation strategy, including staff salaries (with fringe benefits) for training patients and attending project meetings. We will also track operating costs, such as smartphones, monthly data plans, and IT and clinician time for maintaining and improving Seva. The cost data collection instrument will be administered in semi-annual interviews with staff members from the participating clinics and research team during the intervention period. The coach will also keep a detailed log of contacts with clinic staff to assess staff participation during the intervention. We will estimate the cost of implementing Seva by totaling the time spent by coaches and clinicians during the implementation phase (using coach logs) and then multiplying the hours by relevant wage rates by job classification. We will add any non-personnel costs, such as travel to site visits, the cost of teleconferencing services for follow-up calls, etc. Any free services will be documented. Costs incurred by patients are outside the scope of the evaluation.
Evidence suggests that using Seva in a manner that promotes integrative care will benefit FQHCs financially. Research has shown that drug users who receive regular drug abuse care with regular medical care are less likely to be hospitalized than those who receive only regular medical care alone, regular drug abuse care alone, or neither. For patients with substance-abuse-related medical conditions, integrated care can lower costs through reduced hospitalization rates, inpatient days, and emergency-room use. We will assess changes in the healthcare services provided to patients by collecting healthcare utilization data using a combination of electronic health records and self-report. We will analyze these outcomes using our multiple baseline design to determine whether the costs of implementation are offset by benefits in terms of reduced healthcare utilization.
The study received approval from the Medical Sciences Institutional Review Board at the University of Wisconsin–Madison and is registered at ClinicalTrials.gov (NCT01963234).
At time of submission, the research team is entering the pre-implementation period at the pilot clinic. Implementation of Seva with patients and staff is slated to begin during March 2014.
This research adds to the literature on strategies for integrating addiction treatment into primary care. Our study represents an advanced use of mobile health technology in primary care. Seva may function as a clinician-extender because it can provide much of the education and ongoing therapeutic support that patients with substance use disorders need, allowing clinicians to focus on providing the primary care they are trained to deliver. Seva is unique for giving clinicians a way to regularly monitor their patients’ progress outside the clinic through self-reported substance use, appointment attendance, and risk/protection factors. If implementation proves successful, the field will have a proven strategy for implementing and sustaining an effective technology to help address addiction treatment and recovery support in FQHCs.
Early experience has uncovered a host of unanswered questions related to data sharing, security and interoperability that have been difficult to resolve. For instance, receiving IRB approval for the project was a complex undertaking. Issues related to data sharing are proving to be difficult to standardize from one clinic (and one electronic health record system) to the next. Risk aversion to data sharing may be a formidable challenge to full-scale integration of mobile health technology in primary care. We are currently developing a framework that describes different arrangements for integrating mobile health data into electronic health records, personal health records, mobile devices, and clinical data warehouses.
Our early encounters with FQHCs have revealed organizational differences that will be difficult to account for statistically. For instance, one clinic routinely screens for addiction using standardized screening instruments, while another clinic identifies addiction through conversations with patients. Such pre-implementation differences must be accounted for qualitatively to accurately assess changes in the process of care. If implementation is successful, Seva may help reduce variation in clinics’ approaches to core elements of addiction treatment (including referral and treatment processes), although it is evident that different clinics start from very different places in important respects.
Finally, the patient-level RCTs that form the evidence base for Seva did not include measures of healthcare utilization (e.g., ER visits). As we shift towards an organization-level focus in this study of implementation, we seek to understand whether Seva might help patients access health resources more predictably and cost-efficiently. We do not, however, have direct evidence from previous RCTs to support the hypothesis that Seva will reduce healthcare utilization, and analyses related to healthcare utilization should be considered exploratory. Likewise, other outcomes included in our evaluation (for example, quality of life) were unmeasured in the previous RCTs that evaluated TES and A-CHESS. We added these measures after consulting the RE-AIM model dimension checklist (available at http://www.re-aim.hnfe.vt.edu).
Developing an effective technology for the treatment and continuing care of substance use disorders, and disseminating the technology widely and efficiently, is a matter of urgency given the profound costs of SUDs. Although evidence shows that treatment for substance use disorders has wide-ranging benefits, such care — routine for many other chronic illnesses — is rare in primary care. A model for implementing a mobile technology to support the treatment of substance use disorders could provide a cost-effective means to improve the lives of patients with drug and alcohol problems. Results of the research will be published as they become available.
The National Institute on Drug Abuse (NIDA) is funding the study (R01 DA034279-01). A second NIDA grant (R01DAO30431-01) provides other research support. The authors wish to thank the dedicated clinicians and staff members at the three clinics where the study is being conducted. We also thank Lori Ducharme, Haile Behre, Dhavan Shah, Andrew Isham, Michael Grabinski, Aman Tandias, Wen-Jan Tuan, Chantelle Thomas, Mary-Jane Nealon, Paula Goode, Skyler Neylon, Ming-Yuan Chih, and Michael R. Lardieri for the work they have done in support of the project.
- Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003, 362: 1225-1230.View ArticlePubMedGoogle Scholar
- Seale JP, Shellenberger S, Velasquez MM, Boltri JM, Okosun I, Guyinn M, Vinson D, Cornelius M, Johnson JA: Impact of vital signs screening & clinician prompting on alcohol and tobacco screening and intervention rates: a pre-post intervention comparison. BMC Fam Pract. 2010, 11 (1): 18-PMCID: PMC2844356View ArticlePubMedPubMed CentralGoogle Scholar
- Witkiewitz K, Marlatt GA: High-risk situations: relapse as a dynamic process. Therapist's Guide to Evidence-Based Relapse Prevention. Edited by: Witkiewitz K, Marlatt GA. 2007, London: Academic Press, 19-33.View ArticleGoogle Scholar
- Weisner C, Mertens J, Parthasarathy S, Moore C, Lu Y: Integrating primary medical care with addiction treatment: a randomized controlled trial. JAMA. 2001, 286 (14): 1715-1723. PMCID: PMC3056510View ArticlePubMedPubMed CentralGoogle Scholar
- Weisner C, Schmidt LA: Expanding the frame of health services research in the drug abuse field. Health Serv Res. 1995, 30 (5): 707-726. PMCID: PMC1070086PubMedPubMed CentralGoogle Scholar
- Fiellin DA, O'Connor PG, Chawarski M, Pakes JP, Pantalon MV, Schottenfeld RS: Methadone maintenance in primary care: a randomized controlled trial. JAMA. 2001, 286 (14): 1724-1731.View ArticlePubMedGoogle Scholar
- Gibson AE, Doran CM, Bell JR, Ryan A, Lintzeris N: A comparison of buprenorphine treatment in clinic and primary care settings: a randomised trial. Med J Aust. 2003, 179 (1): 38-42.PubMedGoogle Scholar
- Fucito L, Gomes B, Murnion B, Haber P: General practitioners’ diagnostic skills and referral practices in managing patients with drug and alcohol-related health problems: implications for medical training and education programmes. Drug Alcohol Rev. 2003, 22 (4): 417-424.View ArticlePubMedGoogle Scholar
- Olfson M, Tobin JN, Cassells A, Weissman M: Improving the detection of drug abuse, alcohol abuse, and depression in community health centers. J Health Care Poor Underserved. 2003, 14 (3): 386-402.View ArticlePubMedGoogle Scholar
- Spandorfer JM, Israel Y, Turner BJ: Primary care physicians’ views on screening and management of alcohol abuse inconsistencies with national guidelines. J Fam Pract. 1999, 48 (11): 899-902.PubMedGoogle Scholar
- Cunningham J, Wild TC, Walsh GW: Interest in self-help materials in a general population sample of drinkers. Drugs: Educ Prev Polic. 1999, 6 (2): 209-213.Google Scholar
- Rosen CS, Henson BR, Finney JW, Moos RH: Consistency of self-administered and interview-based: addiction Severity Index composite scores. Addiction. 2000, 95 (3): 419-425.View ArticlePubMedGoogle Scholar
- Gustafson DH, Boyle MG, Shaw BR, Isham A, McTavish F, Richards S, Schubert C, Levy M, Johnson K: An e-Health solution for people with alcohol problems. Alcohol Res Health. 2011, 33 (4): 327-337. PMCID: PMC3536059PubMedPubMed CentralGoogle Scholar
- Kotter JP: Leading change: why transformation efforts fail. Harv Bus Rev. 1995, 73 (2): 59-67.Google Scholar
- Bickel WK, Marsch LA, Buchhalter AR, Badger GJ: Computerized behavior therapy for opioid-dependent outpatients: a randomized controlled trial. Exp Clin Psychopharmacol. 2008, 16 (2): 132-143. PMCID: PMC2746734View ArticlePubMedPubMed CentralGoogle Scholar
- Marsch LA, Grabinski MJ, Bickel WK, Desrosiers A, Guarino H, Muehlbach B, Solhkhah R, Talfique S, Acosta M: Computer-assisted HIV prevention for youth with substance use disorders. Subst Use Misuse. 2011, 46 (1): 46-56. PMCID: PMC3091163View ArticlePubMedPubMed CentralGoogle Scholar
- Marsch LA, Guarino H, Acosta M, Aponte-Melendez Y, Cleland C, Grabinski M, Brady R, Edwards J: Web-based behavioral treatment for substance use disorders as a partial replacement of standard methadone maintenance treatment. J Subst Abuse Treat. 2014, 46 (1): 43-51. PMCID: PMC3839618View ArticlePubMedGoogle Scholar
- McKay J, Van Horn DHA, Oslin D, Lynch KG, Ivey M, Ward K, Drapkin ML, Becher JR, Coviello DM: A randomized trial of extended telephone-based continuing care for alcohol dependence: within treatment substance use outcomes. J Consult Clin Psychol. 2010, 78 (6): 912-923. PMCID: PMC3082847View ArticlePubMedPubMed CentralGoogle Scholar
- Maisto SA, Zywiak WH, Connors GJ: Course of functioning 1 year following admission for treatment of alcohol use disorders. Addict Behav. 2006, 31 (1): 69-79.View ArticlePubMedGoogle Scholar
- Godley MD, Godley SH, Dennis MI, Funk R, Passetti LL: Preliminary outcomes from the assertive continuing care experiment for adolescents discharged from residential treatment. J Subst Abuse Treat. 2002, 23: 21-32.View ArticlePubMedGoogle Scholar
- Scherr D, Kastner P, Kollmann A, Hallas A, Auer J, Krappinger H, Schuchlenz H, Stark G, Grander W, Jakl G, Schreier G, Fruhwald FM, the MOBITEL investigators: Effect of home-based telemonitoring using mobile phone technology on the outcome of heart failure patients after an episode of acute decompensation: randomized controlled trial. J Med Internet Res. 2009, 11 (3): e34-PMCID: PMC2762855View ArticlePubMedPubMed CentralGoogle Scholar
- Noh JH, Cho YJ, Nam HW, Kim JH, Kim DJ, Yoo HS, Kwon YW, Woo MH, Cho JW, Hong MH, Yoo JH, Gu MJ, Kim SA, An KE, Jang SM, Kim EK, Yoo HJ: Web-based comprehensive information system for self-management of diabetes mellitus. Diabetes Technol Ther. 2010, 12 (5): 333-337.View ArticlePubMedGoogle Scholar
- Helzer JE, Badger GJ, Rose GL, Mongeon JA, Searles JS: Decline in alcohol consumption during two years of daily reporting. J Stud Alcohol. 2002, 63: 551-558.View ArticlePubMedGoogle Scholar
- Davis JR, Glaros AG: Relapse prevention and smoking cessation. Addict Behav. 1986, 11 (2): 105-114.View ArticlePubMedGoogle Scholar
- Larimer ME, Palmer RS, Marlatt GA: Relapse prevention: an overview of Marlatt’s cognitive-behavioral model. Alcohol Res Health. 1999, 23 (2): 151-160.PubMedGoogle Scholar
- van Osch L, Lechner L, Reubsaet A, Wigger S, de Vries H: Relapse prevention in a national smoking cessation contest: effects of coping planning. Br J Health Psychol. 2008, 13 (3): 525-535.View ArticlePubMedGoogle Scholar
- Washton AM: Cocaine Addiction: Treatment, Recovery, and Relapse Prevention. 1989, New York: W. W. Norton & CompanyGoogle Scholar
- Lorig KR, Ritter PL, Laurent DD, Fries JF: Long-term randomized controlled trials of tailored-print and small-group arthritis self-management interventions. Med Care. 2004, 42 (4): 346-354.View ArticlePubMedGoogle Scholar
- Rapp RC, Siegal HA, Fisher JH: A strengths-based model of case management/advocacy: adapting a mental health model to practice work with persons who have substance abuse problems. NIDA Res Monogr. 1992, 127: 79-91.PubMedGoogle Scholar
- Shaw BR, Han JY, Hawkins RP, McTavish FM, Gustafson DH: Communicating about self and others within an online support group for women with breast cancer and subsequent outcomes. J Health Psychol. 2008, 13 (7): 930-939. PMCID: PMC3632281View ArticlePubMedPubMed CentralGoogle Scholar
- Palermo TM, Wilson AC, Peters M, Lewandowski A, Somhegyi H: Randomized controlled trial of an Internet-delivered family cognitive-behavioral therapy intervention for children and adolescents with chronic pain. Pain. 2009, 146 (1–2): 205-213. PMCID: PMC2760656View ArticlePubMedPubMed CentralGoogle Scholar
- DuBenske LL, Gustafson DH, Shaw BR, Cleary JF: Web-based cancer communication and decision making systems: connecting patients, caregivers, and clinicians for improved health outcomes. Med Decis Making. 2010, 30 (6): 732-744. PMCID: PMC3085247View ArticlePubMedPubMed CentralGoogle Scholar
- Gustafson D, Wise M, Bhattacharya A, Pulvermacher A, Shanovich K, Phillips B, Lehman E, Chinchilli V, Hawkins R, Kim JS: The effects of combining web-based eHealth with telephone nurse case management for pediatric asthma control: a randomized control trial. J Med Internet Res. 2012, 14 (4): e101-PMCID: PMC3409549View ArticlePubMedPubMed CentralGoogle Scholar
- Sullivan WP, Wolk JL, Hartmann DJ: Case management in alcohol and drug treatment: improving client outcomes. Fam Soc. 1992, 73 (4): 195-204.Google Scholar
- Dennis M, Scott CK, Funk R: An experimental evaluation of recovery management check-ups (RMC) for people with chronic substance use disorders. Eval Program Plann. 2003, 26: 339-352.View ArticleGoogle Scholar
- McLellan AT, Hagan TA, Levine M, Meyers K, Gould F, Bencivengo M, Durell J, Jaffe J: Does clinical case management improve outpatient addiction treatment. Drug Alcohol Depend. 1999, 55 (1–2): 91-103.View ArticlePubMedGoogle Scholar
- Gustafson DH, McTavish FM, Chih MY, Atwood AK, Johnson RA, Boyle MG, Levy MS, Driscoll H, Chisholm SM, Dillenburg L, Isham A, Shah D: A smartphone application to support recovery from alcoholism: a randomized controlled trial. JAMA Psychiatry. 2014, 71 (5): 566-572.View ArticlePubMedPubMed CentralGoogle Scholar
- Van de Ven AH: Problem solving, planning, and innovation: part I. test of the program planning model. Hum Relat. 1980, 33 (10): 711-740.View ArticleGoogle Scholar
- Van de Ven AH: Problem solving, planning and innovation: part II. speculations for theory and practice. Hum Relat. 1980, 33 (11): 757-779.View ArticleGoogle Scholar
- McCarty D, Gustafson DH, Wisdom JP, Ford J, Choi D, Molfenter T, Capoccia V, Cotter F: The Network for the Improvement of Addiction Treatment (NIATx): enhancing access and retention. Drug Alcohol Depend. 2007, 88 (2–3): 138-145. PMCID: PMC1896099View ArticlePubMedGoogle Scholar
- Hoffman KA, Ford JH, Choi D, Gustafson DH, McCarty D: Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction Treatment. Drug Alcohol Depend. 2008, 98 (1–2): 63-69. PMCID: PMC2607248View ArticlePubMedPubMed CentralGoogle Scholar
- Rogers EM: Diffusion of Innovations. 1983, New York: The Free Press, 2Google Scholar
- Gustafson DH, Quanbeck AR, Robinson JM, Ford JH, Pulvermacher A, French MT, McConnell KJ, Batalden PB, Hoffman KA, McCarty D: Which elements of improvement collaboratives are most effective? a cluster-randomized trial. Addiction. 2013, 108 (6): 1145-1157. PMCID: PMC3651751View ArticlePubMedPubMed CentralGoogle Scholar
- Gustafson DH, Johnson KA, Capoccia V, Cotter F, Ford JH, Holloway D, Lea D, McCarty D, Molfenter T, Owens B: The NIATx Model: Process Improvement in Behavioral Health. 2011, Madison, WI: University of Wisconsin-MadisonGoogle Scholar
- Hussey MA, Hughes JP: Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007, 28 (2): 182-191.View ArticlePubMedGoogle Scholar
- Glasgow RE, Vogt TM, Boles SM: Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999, 89 (9): 1322-1327. PMCID: PMC1508772View ArticlePubMedPubMed CentralGoogle Scholar
- Wen KY, Gustafson DH, Hawkins RP, Brennan PF, Dinauer S, Johnson PR, Siegler T: Developing and validating a model to predict the success of an IHCS implementation: the Readiness for Implementation Model. J Am Med Inform Assoc. 2010, 17 (6): 707-713. PMCID: PMC3000759View ArticlePubMedPubMed CentralGoogle Scholar
- Chamberlain P, Brown CH, Saldana L: Observational measure of implementation progress in community based settings: the Stages of implementation completion (SIC). Implementation Sci. 2011, 6 (1): 116-PMCID: PMC3197550View ArticleGoogle Scholar
- Legris P, Ingham J, Collerette P: Why do people use information technology? A critical review of the technology acceptance model. Inf Manage. 2003, 40 (3): 191-204.View ArticleGoogle Scholar
- Maher L, Gustafson D, Evans A: Sustainability, Model and Guide. 2004, Liecester, England: NHS Institute for Innovation and Improvement, British National Health Service (Modernisation Agency), [http://www.qihub.scot.nhs.uk]Google Scholar
- Creswell JW, Klassen AC, Plano Clark VL, Smith KC: Best Practices for Mixed Methods Research in the Health Sciences. 2011, National Institutes of Health: Bethesda, MDView ArticleGoogle Scholar
- Bobrovitz CD, Ottenbacher KJ: Comparison of visual inspection and statistical analysis of single-subject data in rehabilitation research. Am J Phys Med Rehabil. 1998, 77 (2): 94-102.View ArticlePubMedGoogle Scholar
- Horner RH, Carr EG, Halle J, McGee G, Odom S, Wolery M: The use of single-subject research to identify evidence-based practice in special education. Except Child. 2005, 71 (2): 165-179.View ArticleGoogle Scholar
- Severens JL, Hoomans T, Adang E, Wensing M: Economic evaluation of implementation strategies. Improving Patient Care: The Implementation of Change in Health Care. Edited by: Grol R, Wensing M, Eccles M, Davis D. 2013, Chichester West Sussex, UK: Wiley-Blackwell, 350-2View ArticleGoogle Scholar
- Laine C, Hauck WW, Gourevitch MN, Rothman J, Cohen A, Turner BJ: Regular outpatient medical and drug abuse care and subsequent hospitalization of persons who use illicit drugs. JAMA. 2001, 285 (18): 2355-2362.View ArticlePubMedGoogle Scholar
- Parthasarathy S, Mertens J, Moore C, Weisner C: Utilization and cost impact of integrating substance abuse treatment and primary care. Med Care. 2003, 41 (3): 357-367.PubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.