Skip to main content

Factors influencing implementation of a care coordination intervention for cancer survivors with multiple comorbidities in a safety-net system: an application of the Implementation Research Logic Model



Under- and uninsured cancer survivors have significant medical, social, and economic complexity. For these survivors, effective care coordination between oncology and primary care teams is critical for high-quality, comprehensive care. While evidence-based interventions exist to improve coordination between healthcare teams, testing implementation of these interventions for cancer survivors seen in real-world safety-net settings has been limited. This study aimed to (1) identify factors influencing implementation of a multicomponent care coordination intervention (nurse coordinator plus patient registry) focused on cancer survivors with multiple comorbidities in an integrated safety-net system and (2) identify mechanisms through which the factors impacted implementation outcomes.


We conducted semi-structured interviews (patients, providers, and system leaders), structured observations of primary care and oncology operations, and document analysis during intervention implementation between 2016 and 2020. The practice change model (PCM) guided data collection to identify barriers and facilitators of implementation; the PCM, Consolidated Framework for Implementation Research, and Implementation Research Logic Model guided four immersion/crystallization data analysis and synthesis cycles to identify mechanisms and assess outcomes. Implementation outcomes included appropriateness, acceptability, adoption, and penetration.


The intervention was appropriate and acceptable to primary care and oncology teams based on reported patient needs and resources and the strength of the evidence supporting intervention components. Active and sustained partnership with system leaders facilitated these outcomes. There was limited adoption and penetration early in implementation because the study was narrowly focused on just breast and colorectal cancer patients. This created barriers to real-world practice where patients with all cancer types receive care. Over time, flexibility intentionally designed into intervention implementation facilitated adoption and penetration. Regular feedback from system partners and rapid cycles of implementation and evaluation led to real-time adaptations increasing adoption and penetration.


Evidence-based interventions to coordinate care for underserved cancer survivors across oncology and primary care teams can be implemented successfully when system leaders are actively engaged and with flexibility in implementation embedded intentionally to continuously facilitate adoption and penetration across the health system.

Peer Review reports


Early detection and treatment advances are driving steady increases in the number of cancer survivors. In many cases, living with cancer has become similar to living with common chronic conditions such as diabetes and heart disease. Most patients with cancer also have three or more chronic conditions [1, 2] requiring coordinated care between oncology, primary care, and other specialties. Primary care can play an important role in providing comprehensive, coordinated care for all conditions, including cancer. In fact, national cancer survivorship guidelines recommend that patients with cancer, known as cancer survivors, need primary care clinicians (PCC) as part of their care team because PCCs play an important role in providing comprehensive, whole person care [3,4,5]. To achieve these goals, communication and coordination between primary care and oncology are paramount [6,7,8,9]. Although central to the Institute of Medicine’s recommendations made nearly 20 years ago [10], the field is unclear how to ensure cancer survivors stay connected with primary care from start of cancer treatment and throughout their cancer survivorship journey.

Well-established evidence from primary care settings demonstrates the effectiveness of using patient registries and designated care coordinators for improving patient outcomes for many chronic conditions such as diabetes and hypertension [2, 11,12,13]. Patient registries enable identification of all patients with specific conditions (e.g., breast cancer, diabetes) to proactively plan care delivery for timely provision of preventive and chronic disease care. Care coordinators play a critical role in managing referrals and connections between specialties and in ensuring that relevant clinical information to manage patients’ care is available and accessible to clinicians at point of care. However, few trials have studied implementation of these evidence-based interventions among patients living with cancer and chronic conditions, particularly in safety-net healthcare settings.

Project CONNECT implemented these effective care coordination interventions (care coordinator plus patient registry) among cancer survivors with chronic conditions in an urban, integrated safety-net health system that serves disproportionately under- and uninsured ethnic/racial minority [14]. The aims of this study are to (1) identify factors influencing implementation of Project CONNECT and (2) identify mechanisms through which the factors influenced implementation outcomes.


This multi-method qualitative study was embedded within a pragmatic trial that tested the implementation of a multicomponent evidence-based intervention aimed at enhancing care coordination for breast and colorectal cancer survivors with chronic conditions [15]. Study procedures were approved by the University of Texas Southwestern Institutional Review Board (STU 102015–090), the University of Texas Health Science Center at Houston, and by Parkland Health Office of Research Administration, and reporting follows the Standards for Reporting Qualitative Research guidelines [16].


This study was conducted at Parkland Health (Parkland), the safety-net health system serving Dallas County, TX, USA [17, 18]. “Safety-net” healthcare systems are those that deliver healthcare primarily to uninsured, Medicaid, and other low-income and vulnerable patient populations [19]. Parkland includes a network of 13 primary care clinics located in predominantly under resourced, ethnic/racial minority communities across Dallas County, and a centrally located main campus. The main campus consists of an inpatient hospital, outpatient surgery center, and specialty care clinics, which include multidisciplinary cancer clinics (i.e., medical, surgical, and radiation oncology clinics).

Breast and colorectal cancer are the top two types of cancers treated at Parkland. Twenty-four percent of patients with breast cancer present with stages 3 or 4 breast cancer compared to 10% nationally; 61% of patients with colorectal cancer present with stages 3 and 4 cancer compared to 45% nationally [20].

Evidence-based intervention components and implementation strategies

Project CONNECT was a multicomponent evidence-based intervention and included (1) an electronic medical record (EMR)-based patient registry and (2) a care coordinator [15]. The registry identified patients diagnosed with stages I–III breast or colorectal cancers plus one or more of the following chronic conditions: diabetes, hypertension, heart disease, chronic kidney disease, and/or chronic lung disease. The care coordinator was a registered nurse employed by Parkland who helped connect study eligible cancer survivors to primary care by facilitating appointments with primary care and coordinated care for patients between oncology and primary care. Strategies identified a priori to implement the intervention components into clinical practice including the following: identifying champions, changing records systems, creating new clinical workflows, and flexibility in implementation (Table 1) [15].

Table 1 Intervention components, functions, and implementations strategies

Guiding theoretical and conceptual frameworks

The practice change model (PCM) and the Consolidated Framework for Implementation Research (CFIR) are determinant frameworks that guided data collection to identify barriers and facilitators of implementation [22, 23]. The PCM includes four elements (e.g., internal motivators, external motivators, resources, and opportunities for change) and depicts how these multi-level elements can impact intervention implementation in healthcare settings over time [22]. The CFIR is a menu of individual-, program-, and organizational-level constructs consolidated from 19 theories and models related to intervention implementation [24]. The constructs are organized into five overarching domains: intervention characteristics, outer setting, inner setting, characteristics of individuals, and process. These frameworks are highly complementary. PCM grounded our focus on drivers of practice operations and potential interactions, while CFIR helped us attend to relationships [25] between our intervention, actors in the practice, and the structure and sequence of care delivery to maximize learning from our real-world setting [26].

Proctor and colleagues’ taxonomy of implementation outcomes [27] and the Implementation Research Logic Model (IRLM) [28] informed our data analysis and synthesis [27].This study used qualitative data to assess two implementation outcomes at the patient and provider levels (i.e., intervention acceptability and appropriateness) and two outcomes at the organizational level (e.g., intervention adoption and penetration). Acceptability is defined as patient and/or provider satisfaction with the intervention, and appropriateness is defined as the perceived fit of the intervention in the setting [27]. Adoption is defined as the initial uptake of the intervention; penetration is defined as the integration of an intervention within a clinical team, which is similar to the concept of “reach” in Glasgow’s RE-AIM framework [29]. Adoption and penetration were assessed longitudinally during intervention implementation (Phase 2, see below) allowing assessment of continued adoption or utilization of the intervention beyond initial uptake. Finally, the IRLM is a visualization tool to depict causal pathways between intervention components, determinants (i.e., barriers and facilitators) of implementation, implementation strategies, mechanisms of action, and implementation [28]. Mechanisms of action define how implementation strategies operate to influence outcomes. We used the IRLM to elucidate the relationships between determinants, mechanisms, and implementation outcomes.

Data collection

Trained investigators (R. T. H., P. M. C., S. C. L.) collected qualitative data throughout intervention implementation from September 2016 through June 2020. This included two phases of the study. Phase 1 of data collection occurred from September 2016 to September 2018, pre-intervention implementation. Phase 2 occurred from September 2018 to June 2020 during intervention implementation.

We used purposive sampling to select clinical team members who varied by their roles and specialty to identify barriers and facilitators to delivering coordinated care for patients with cancer and chronic conditions. Study participants included clinicians (e.g., physicians, nurse practitioners), clinic staff (e.g., nurses, care coordinators, social workers, financial services coordinators), and health system leaders (e.g., unit managers, clinic managers, and medical service chiefs) in oncology, primary care, and specialty care. We recruited multiple participants for each role and unit to solicit diverse perspectives.

Data sources included the following: (1) documents, (2) structured observations and field notes, and (3) semi-structured interviews with patients, providers, staff, and leaders from multiple departments across the integrated safety-net system.


Documents included meeting notes, policies and procedures, correspondence among stakeholders, EMR screenshots, patient-facing materials, tools and checklists, and other resources. Documents were requested from providers, staff, and leaders, and they were also offered unsolicited by interviewees and observed stakeholders to clarify processes, provide supplementary information, or serve as historical records.

Structured observations and field notes

We used structured observation guides to facilitate consistent data capture of care coordination and practice change processes [30]. Exemplar domains and questions included the following: evidence of team-based care (e.g., do oncology providers discuss other conditions or comorbidities?), documentation of practice (e.g., where do oncology providers document information related to the survivorship care plan or referrals for follow-up care after discharge?), patient access to information (e.g., do providers tell patients what to do in the event of acute needs?), continuity of care (e.g., how are subsequent appointments scheduled?), and team-based care (e.g., to what extent do providers engage patients in taking an active role in their care?). We also selected sites for observation to capture the patient pathway and provider/staff movement through the care coordination process (e.g., registration and intake areas, patient-provider interactions, provider-staff interactions and work areas, and nurse navigation, referral, and case management processes) [31].

Semi-structured interviews

Interviews were semi-structured to guide the interviewer through pre-planned topics while allowing for follow-up questions tailored to participant feedback and for additional unplanned questions to be incorporated as appropriate. Interview guides were iteratively developed by investigators and adapted to role and clinical unit. We anticipated barriers and facilitators to implementation based on the CFIR [23] and PCM [22] and, accordingly, focused the interviews on domains including the following: care coordination processes between oncology and primary care, perceptions of the role of the nurse coordinator and registry (interventions), challenges or gaps in care for cancer survivors with chronic conditions, communication about policies and procedures within clinics, EMR documentation and challenges, delineation of roles and expectations between oncology and primary care providers and staff, and patient feedback about areas of confusion or concern. Prior to participating in an interview, informed verbal consent was obtained from all study participants. Patients received a US $25 gift card in appreciation for their time. In accordance with Parkland policy, employees were not provided with an incentive to participate in research.

Data analysis

Immersion-crystallization processes

Data analysis proceeded in four immersion-crystallization cycles, or repeated exposure to and synthesizing of data, to identify themes and categories (Supplementary Materials) [32].

In cycle 1, the team developed two deductively driven thematic codebooks based on interview guide topics, pre- and post-intervention phases, and a preliminary review of documents (n = 259 unique documents), field observations (n = 11), and interview transcripts (n = 140). Additional emergent themes were incorporated into the initial codebook drafts for the first 10% of transcripts, and the finalized codebook was used for remaining transcripts. Codebooks for the two phases included many of the same codes; however, each also included additional unique codes given differences in thematic foci and emergent findings during each phase. For example, Phase 1 codes included existing barriers to care coordination, organizational structure, and processes; Phase 2 codes included patient experiences, acuity of care, and transitions in care. All coding was completed in NVivo 12.0 (QSR, Australia). After coding all data, the team created node reports, summaries of data collected, and exemplar quotes for each code and identified codes tying together steps in the cancer care continuum to the intervention components: care coordination, survivorship planning, intervention, and intervention impact.

In analysis cycle 2, the team applied codes from PCM and CFIR to the selected node reports focusing on identifying organizational inner setting characteristics, system resources, stakeholder motivations, and opportunities for change. The purpose of this cycle was to understand how and why care coordination processes occurred pre- and post-intervention. In cycle 3, the team returned to the findings from cycles 1 and 2, coding for in order to describe how the intervention components and care coordination processes mapped to implementation outcomes. Implementation outcomes were assessed from qualitative data; most validated quantitative implementation outcome measures were not available at the start of this study. In cycle 4, the team met weekly to interpret findings and synthesize data linking implementation strategies, determinants, mechanisms, and outcomes using the IRLM.


Figure 1 depicts the determinants, implementation strategies addressing the determinants, and observed mechanisms influencing each implementation outcome, and Table 2 provides illustrative quotes for determinants.

Fig. 1
figure 1

Determinants, implementation strategies, and mechanisms linked with implementation outcomes

Table 2 Determinants and exemplar quotes

Appropriateness and acceptability

The Project CONNECT intervention was appropriate for the safety-net healthcare setting and acceptable to system leaders. Three determinants (patient and clinician barriers, scientific evidence supporting the interventions, and engaged stakeholders) influenced appropriateness and acceptability primarily through three implementation strategies (creating new clinical workflows, changing records systems, identifying champions) during early implementation.

Patient and clinician barriers

The intervention addressed patient and clinician barriers to coordinating care between primary care and oncology. Patients with cancer and chronic conditions described confusion about which clinician to contact (i.e., primary care, oncology, or other specialists) for appointments, medication refills, questions, or concerns. Patients also assumed their providers were actively communicating through the shared EMR system about, for example, changes to medications or their treatment progress. Although clinicians documented these issues in the EMR, they were not proactively communicating across teams unless specific actions were warranted.

Clinicians also expressed challenges caring for patients with cancer and chronic conditions. Primary care clinicians noted patients often did not continue seeing their primary care providers after cancer diagnoses. They then assumed oncologists or other specialists were addressing patients’ chronic conditions. Oncology clinicians, on the other hand, felt patients could not access primary care appointments in a timely manner during active treatment, and they would therefore fill prescriptions for chronic conditions, sometimes for up to 1 year. Changing the EMR system and creating new clinical workflows as implementation strategies facilitated development of the patient registry, which enabled the nurse coordinator to identify patients with cancer and chronic conditions, to track them through their course of oncology treatment, and to proactively schedule appointments, thus addressing both patient and provider needs.

Strength and quality of intervention evidence

Oncology and primary care system leaders agreed the selected interventions were appropriate for their setting to address these patient needs, and leaders supported implementation based on the strength of the scientific evidence behind registry and nurse coordinator interventions in enhancing care coordination. These early champions recognized the interventions were familiar to clinical team members, had been deployed in other contexts across Parkland Health, and were reasonable to budget with the assistance of research funds. Identifying and engaging the champions further increased intervention acceptability.

Stakeholder engagement

The CONNECT intervention was designed with input from Parkland primary care and oncology leaders who were aware of patient and clinician barriers caring for patients with cancer and chronic conditions. Primary care and oncology system leaders informed the decision to have the nurse coordinator located in the oncology clinic rather than in primary care practices, arguing that patients with chronic conditions in “active” cancer treatment needed continuity with primary care for their chronic disease management needs. System leaders emphasized the need for care coordination from the time of cancer diagnosis, not just after end of active treatment. Over the course of the intervention, leaders proactively identified two nurse coordinators to fill the role. Thus, engaging system leaders in designing and implementing key intervention components facilitated a sense of ownership and influenced intervention appropriateness and acceptability.

Adoption and penetration

All physicians and advanced practice providers of the Parkland oncology clinic adopted the intervention initially. Initial adoption was facilitated by active stakeholder engagement and leadership support. System leaders facilitated warm handoffs (i.e., provider-to-provider exchange) [33] between primary care and oncology clinicians who served as local site champions to facilitate adoption (aka Parkland implementers). Primary care implementers identified gaps in chronic disease monitoring and management for patients in active cancer treatment. They shared clinical expertise to change the EMR system integrating the patient registry functionality into the EPIC Reporting Workbench. Primary care and oncology care teams both actively engaged with the research team in developing new clinical workflows establishing the nurse coordinator’s role in coordinating care. In turn, the research team was able to provide technical assistance related to the evidence base to inform implementer efforts.

As the intervention progressed over time, there were barriers to continued adoption which then also limited penetration. Two determinants (organizational communication networks and characteristics, intervention characteristics) limited continued adoption and penetration of intervention components during the mid-implementation phase of the study. Reflecting on and evaluating the implementation process and the flexibility embedded in implementation allowed the research team to be responsive to stakeholders, to identify new implementation strategies (e.g., remind providers and staff), and to ultimately facilitate continued use of the intervention and penetration over the course of implementation.

Organizational communication networks and characteristics

Organizational communication networks and characteristics impacted continued adoption. Communication about day-to-day activities related to the intervention occurred informally during staff huddles and often via word of mouth limiting dissemination of the intervention across the oncology clinic where the nurse coordinator was embedded. For example, the informal nature of team communication about the intervention limited the nurse coordinator’s ability to share information about the different ways she could support care coordination for cancer patients with chronic conditions. In addition, the safety-net system, a teaching hospital, also experienced frequent turnover in oncology fellows and staff. Without consistent communication and interaction with the nurse coordinator, many did not know the intervention was available to their patients. Compounding one another, these determinants led to limited staff and provider knowledge about the interventions and limited continued utilization of the intervention by oncology providers.

Intervention characteristics

Characteristics of the nurse coordinator also influenced continued adoption. At study rollout, a primary care registered nurse filled the role. Because the coordinator intervention was located in oncology, it took time for the primary care nurse to learn the communication networks and to integrate within the oncology team. One year into the study, the nurse left Parkland, and system leaders identified a seasoned oncology nurse to assume the nurse coordinator role. Her familiarity with oncology team members and clinic processes increased others’ adoption of the nurse coordinator intervention.

Barriers to continued adoption coupled with intervention scope further limited intervention penetration. Penetration, in this context, refers to the extent of the intervention’s spread or reach across all oncology clinicians. Advanced practice providers (APPs) expressed difficulty in changing their workflow to engage the nurse coordinator only for breast and colorectal cancer patients (as defined by the research study) when they also experienced challenges connecting patients with other cancer types with their primary care doctors. APPs include licensed nonphysician providers such as nurse practitioners, physician’s assistants, and medical assistants. Although the nurse coordinator was employed by Parkland, the APPs viewed her as only available for the Project CONNECT “research” study, thus focusing only on patients with breast and colorectal cancer. This narrowly defined scope limited the integration of nurse coordinator services into usual clinic workflows and therefore limited intervention penetration. In addition, the nurse coordinator found that not all patients with cancer presented through the medical oncology clinic. In particular, some patients with stage 1 colorectal cancer who started in surgical oncology did not receive follow-up through the medical oncology clinic after surgery, so limiting intervention scope to the medical oncology clinic limited penetration.

Reflecting and evaluating overtime

Iterative data collection throughout implementation allowed for reflection and evaluation of the implementation process and allowed the research team and Parkland implementers to respond to stakeholder feedback and implementation barriers. Along with flexibility in implementation, this feedback loop enabled the team to address changes in determinants and adapt or identify new implementation strategies needed to increase adoption and penetration. Based on feedback, the nurse coordinator began attending meetings and huddles to inform new colleagues about her role and to continually remind existing staff and providers about her role. In addition, the research team expanded the intervention’s scope to include patients presenting at multiple sites (e.g., surgical oncology, emergency department) to increase penetration. Finally, changes to the workflow to include all patients with cancer and chronic conditions enabled the nurse coordinator to connect any patient needing chronic disease management to a primary care clinician.

Flexibility in implementation

The implementation strategy “flexibility in implementation” influenced all four implementation outcomes. Specifically, changes in intervention scope to include patients with any type of cancer increased APP acceptance of the nurse coordinator and therefore their adoption of the intervention. The change in scope allowed the nurse coordinator to further integrate with—or penetrate—the oncology team and expand intervention reach. In addition, continual stakeholder engagement and flexibility ensured intervention components remained appropriate and acceptable throughout implementation.


There is significant interest among researchers, clinicians, health system leaders, and policy makers in identifying optimal ways to coordinate care for cancer survivors, especially those who are under- and uninsured and most likely to have poor health outcomes. This study demonstrated that implementing a system-level evidence-based intervention to coordinate care for cancer survivors with chronic conditions between oncology and primary care in a safety-net health system was appropriate and acceptable to patients and health system stakeholders. While clinicians and clinical staff initially adopted the intervention, continued adoption and penetration of the intervention throughout the clinic were challenging even with support from motivated and engaged health system leaders. This is because the intervention, as designed, experienced challenges integrating into real-world practice. Continual evaluation and reflection allowed the research team to be responsive to stakeholder feedback in real time, to identify emerging determinants, and to develop new implementation strategies to increase acceptability, continued adoption, and penetration. Importantly, flexibility in implementation became a key implementation strategy to address barriers to adoption and penetration over time. While our intervention took place in a safety-net healthcare system in the USA, our findings about how to bridge cancer survivors’ care between PCCs and oncology clinicians are applicable more broadly as comprehensive cancer survivorship care approaches are needed globally for different health care system models [34]. Determinants—such as patient and clinician barriers, lack of stakeholder engagement, the strength and quality of scientific evidence supporting care coordination interventions, and intervention characteristics—have been explored in the context of other interventions and other disease/patient targets [35,36,37], such as for diabetes and hypertension management. However, our study is the first to examine these determinants in the context of implementing an intervention for patients with cancer receiving care in a safety-net setting. This is significant because an increasing number of patients with cancers such as early-stage breast, colon, and rectal cancer are living decades after their initial diagnosis, thanks to significant advances in early detection and treatments. In such cases, cancer becomes another chronic condition that patients and their clinicians must manage including timely surveillance for recurrence and managing risks associated with cancer treatments and its sequelae. Thus, delineating determinants of adoption and implementation of evidence-based care coordination interventions shown to be effective for routine chronic conditions such as those used in Project CONNECT can aid health systems in coordinating care for patients with cancer and chronic conditions. Importantly, our focus on safety-net health systems has the potential to increase health equity by identifying determinants relevant for patients with signficant social and economic challenges and for under-resourced systems. More cancer survivorship care delivery research embedded in safety-net systems and community health centers is needed to improve care delivery outcomes.

This study shows that determinants are dynamic rather than static constructs and change over time to influence multiple implementation outcomes. While researchers have theorized that determinants may change over time, few studies have embedded longitudinal evaluations such as ours that demonstrate how they change and the ways in which they influence implementation outcomes over time. For example, system leaders affirmed intervention appropriateness and contributed to initial acceptability at multiple levels. As implementation proceeded, it became clear that consistent, iterative engagement with site champions in primary and oncology care was necessary also to ensure continued adoption. Our study design enabled us to observe these changes and influences over time, and our iterative immersion-crystalization data analysis strategy enabled us to identify similar recursive relations between determinants, strategies, mechanisms, and outcomes.

Similarly, this study shows how implementation outcomes are interrelated, influenced by determinants and other outcomes, and how it may be unclear at what point one outcome ends and another begins. Although adoption has often been viewed as the intention or initial decision to use an innovation, we consider adoption as both the initial uptake and continued use of the interventions. For example, while stakeholders initially adopted the nurse coordinator intervention, continued adoption and interaction with the nurse coordinator later waned among APPs, who felt the intervention was only relevant for some of their patients. This limited overall penetration into the system. Flexibility in implementation meant that we could rapidly evaluate and adapt in real time to facilitate continued use of the intervention. Recognizing the need for adaptations and responding to dynamic contexts are recommended strategies when designing for dissemination and sustainability [38]. We hypothesize that the phenomena we captured in variable adoption and penetration may be early determinants of maintenance or institutionalization of an intervention into practice. In fact, in our recent meeting with director of Parkland Global Oncology, we learned that Project CONNECT interventions are still being used at Parkland in a modified form.

Describing these interdependent relations is critically important to keep the field moving forward and for research among healthcare systems as they are complex systems adaptive to internal and external factors. Thus, mechanisms of how strategies address determinants to improve implementation and service outcomes are more likely to be inter-dependent rather than linear. Our study’s design and analytic methods helped bring this reality to light.

Our data analysis strategy and use of the IRLM were fundamental to identifying, defining, often disentangling determinants and strategies, understanding their mechanisms, and linking them to implementation outcomes. Smith et al.’s IRLM has been used to guide design and evaluation of implementation studies, describe implementation barriers and facilitators, list hypothesized mechanisms, and engage stakeholders. [28, 39,40,41,42,43,44,45,46,47,48,49,50,51] Our analysis advanced the authors’ recommendation to use the tool to elucidate the evolving relations between determinants, implementation strategies, mechanisms, and implementation outcomes [28]. Few studies collect the data needed to describe these changes over time. Our analysis exemplifies why continuous process evaluation data are needed longitudinally and why investing in mixed-method, comprehensive, and longitudinal evaluation data is crucial for rigorous implementation research.

Our study also sheds light on the balance between degree to which an intervention is delivered as intended, i.e., implementation fidelity [52] and flexibility needed for integrating the intervention in real-world settings [53]. We argue that flexibility of implementation is necessary to accelerate translation of evidence-based interventions in real-world settings, and it does not necessarily constitute a decrease in fidelity to the evidence-based intervention. Both fidelity and flexibility are needed and can co-occur in equilibrium such that key functions of evidence-based interventions are implemented with fidelity, but the forms of the interventions themselves may differ across settings, or changes may be made to intervention forms in response to contextual barriers [54]. As shown in our study, implementation flexibility enhanced adoption and penetration of the intervention. This may be a critical nuance that bears further scrutiny and may be a key ingredient in increasing uptake of evidence-based interventions into real-world settings.

Study limitations

The onset of COVID in North Texas disrupted elective care across the Parkland system, oncology clinic teams pivoted to telephone appointments, and the nurse coordinator was able to continue work remotely. Although direct research observation was temporarily interrupted, our relationships with key stakeholders enabled the research team to continue to collect data through email and telephone exchanges. In addition, a challenge we did not anticipate, but did document, was that annual updates to the Epic EMR could also “break” links even to existing Epic functionality, such as the Reporting Workbench, and needed to be monitored to ensure registry tools remained active.

Future directions

While our analysis here reports key determinants affecting implementation outcomes, it is yet unclear how increased adoption and penetration may influence team processes supporting care coordination that we did not observe. Although the evidence base for care coordination interventions is strong, the field’s understanding of how factors relevant to local settings shape implementation is still emerging [13]. Forthcoming analyses of Project CONNECT intervention outcomes at the patient and system level could facilitate examination of maintenance and generate key questions to explore about earlier indicators around post-study intervention sustainability once the trial ended.

Having described implementation outcomes here, subsequent analyses of clinical and patient-reported outcomes will help advance our understanding of how these EBIs may help optimize care for these vulnerable patient populations. Similarly, we did not explicitly set out to assess the effectiveness of a bundled implementation strategy, nor to test the separability of our multicomponent intervention. While future work could mount studies to examine these issues, from the perspective of addressing disparities in survivorship care delivery, implementation research should focus on better characterizing the interface between primary care and oncology and identify strategies to better integrate care delivery for cancer survivors such that the care they receive is seamless and addresses survivorship care guidelines holistically [55].


Effective and accepted interventions such as using population-based registry to track patients with cancer and chronic conditions and assigning a care coordinator to enhance primary care access can be implemented effectively in safety-net health systems. Adoption and penetration across the system can be further enhanced by allowing flexibility in how health systems choose to implement these interventions. Doing so with active and continual engagement of patient and health system partners presents the most promising approach to quickly translate effective interventions into real-world practice to improve care delivery and health outcomes for cancer survivors.

Availability of data and materials

The data used and/or analyzed during the current study are available from the corresponding author on reasonable request.


  1. Bayliss EA, Balasubramianian BA, Gill JM, Stange KC. Perspectives in primary care: implementing patient-centered care coordination for individuals with multiple chronic medical conditions. Ann Fam Med. 2014;12(6):500–3.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Weaver SJ, Jacobsen PB. Cancer care coordination: opportunities for healthcare delivery research. Transl Behav Med. 2018;8(3):503–8.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Adler NE, Page AEK. Institute of Medicine (US) Committee on Psychosocial Services to Cancer Patients/Families in a Community Setting, eds. Cancer Care for the Whole Patient: Meeting Psychosocial Health Needs. Washington (DC): National Academies Press (US); 2008.

  4. McCabe MS, Bhatia S, Oeffinger KC, et al. American society of clinical oncology statement: achieving high-quality cancer survivorship care. J Clin Oncol. 2013;31(5):631–40.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Nekhlyudov L, O’Malley DM, Hudson SV. Integrating primary care providers in the care of cancer survivors: gaps in evidence and future opportunities. Lancet Oncol. 2017;18(1):e30–8.

    Article  PubMed  PubMed Central  Google Scholar 

  6. McCabe MS, Bhatia S, Oeffinger KC, et al. American Society of Clinical Oncology statement: achieving high-quality cancer survivorship care. J Clin Oncol. 2013;31(5):631–40.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Hudson SV, Miller SM, Hemler J, et al. Adult cancer survivors discuss follow-up in primary care: ‘not what i want, but maybe what i need.’ Ann Fam Med. 2012;10(5):418–27.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Dossett LA, Hudson JN, Morris AM, et al. The primary care provider (PCP)-cancer specialist relationship: a systematic review and mixed-methods meta-synthesis. CA A Cancer J Clin. 2017;67(2):156–69.

    Article  Google Scholar 

  9. Lee SJC, Clark MA, Cox JV, Needles BM, Seigel C, Balasubramanian BA. Achieving coordinated care for patients with complex cases of cancer: a multiteam system approach. J Oncol Pract. 2016;12:1029.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Hewitt M, Greenfield S, Stovall E. From Cancer Patient to Cancer Survivor: Lost in Translation. IOM Committee on Cancer Survivorship: improving care and quality of life. National Academies Press; 2005.

  11. McDonald K, Sundaram V, Bravata D, et al. Care coordination. In: KG S, McDonald K, Wachter R, Owens D, eds. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies. Agency for Healthcare Research & Quality; 2007:chap 7.

  12. Gorin SS, Haggstrom D, Han PKJ, Fairfield KM, Krebs P, Clauser SB. Cancer care coordination: a systematic review and meta-analysis of over 30 years of empirical studies. Ann Behav Med. 2016;51:532.

    Article  Google Scholar 

  13. Bodenheimer T. Coordinating care — a perilous journey through the health care system. N Engl J Med. 2008;358(10):1064–71.

    Article  PubMed  CAS  Google Scholar 

  14. Lee SJC, Jetelina KK, Marks E, et al. Care coordination for complex cancer survivors in an integrated safety-net system: a study protocol. BMC Cancer. 2018;18:1471–2407.

    Article  Google Scholar 

  15. Lee SJC, Jetelina KK, Marks E, et al. Care coordination for complex cancer survivors in an integrated safety-net system: a study protocol. BMC Cancer. 2018;18(1):1204.

    Article  PubMed  PubMed Central  Google Scholar 

  16. O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51.

    Article  PubMed  Google Scholar 

  17. Medicaid and CHIP Payment and Access Commission (MACPAC). Fact Sheet: Parkland Hospital. Accessed February 10, 2022.

  18. Clark D, Roberson B, Ramiah K. Essential data, our hospitals, our patients: results of America’s Essential Hospitals 2018 Annual Member Characteristics Survey. America’s Essential Hospitals. Accessed March 2023. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/

  19. Lewin ME, Altman SH, Institute of Medicine (U.S.). Committee on the Changing Market Managed Care and the Future Viability of Safety Net Providers. America's health care safety net : intact but endangered. Institute of Medicine : National Academy Press; 2000:xviii, 281 p.

  20. Parkland Cancer Program. Cancer Program Annual Report. Accessed February 15, 2023.

  21. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Cohen D, McDaniel RR, Crabtree BF, et al. A practice change model for quality improvement in primary care practice. J Healthc Manag. 2004;49(3):155–68.

    PubMed  Google Scholar 

  23. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Bunce AE, Gruss I, Davis JV, et al. Lessons learned about the effective operationalization of champions as an implementation strategy: results from a qualitative process evaluation of a pragmatic trial. Implement Sci. 2020;15(1):87.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Stroebel CK, McDaniel RR Jr, Crabtree BF, Miller WL, Nutting PA, Stange KC. How complexity science can inform a reflective process for improvement in primary care practices. Jt Comm J Qual Patient Saf. 2005;31(8):438–46.

    Article  PubMed  Google Scholar 

  26. Balasubramanian BA, Cohen DJ, Davis MM, et al. Learning evaluation: blending quality improvement and implementation research methods to study healthcare innovations. J Article Implement Sci. 2015;10(1):31.

    Article  Google Scholar 

  27. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

    Article  PubMed  Google Scholar 

  28. Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15(1):84.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res. 2006;21(5):688–94.

    Article  PubMed  Google Scholar 

  30. Jaen CR, Crabtree BF, Palmer RF, et al. Methods for evaluating practice change toward a patient-centered medical home. Ann Fam Med. 2010;8(Suppl 1):S9-20;S92.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Goldman RE, Parker DR, Brown J, Walker J, Eaton CB, Borkan JM. Recommendations for a mixed methods approach to evaluating the patient-centered medical home. Ann Fam Med. 2015;13(2):168–75.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Borkan JM. Immersion-crystallization: a valuable analytic tool for healthcare research. Fam Pract. 2022;39:785.

    Article  PubMed  Google Scholar 

  33. Agency for Healthcare Research and Quality. Warm handoff: intervention. Agency for Healthcare Research and Quality. Updated June 2023. Accessed October 1, 2023.

  34. Chan RJ, Crawford-Williams F, Crichton M, et al. Effectiveness and implementation of models of cancer survivorship care: an overview of systematic reviews. J Cancer Surviv. 2023;17:197.

    Article  PubMed  Google Scholar 

  35. Albers B, Auer R, Caci L, et al. Implementing organized colorectal cancer screening programs in Europe—protocol for a systematic review of determinants and strategies. Syst Rev. 2023;12(1):26.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Merle JL, Li D, Keiser B, et al. Categorising implementation determinants and strategies within the US HIV implementation literature: a systematic review protocol. BMJ Open. 2023;13(3): e070216.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Nelson LA, Roddy MK, Bergner EM, et al. Exploring determinants and strategies for implementing self-management support text messaging interventions in safety net clinics. J Clin Transl Sci. 2022;6(1): e126.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Kwan BM, Brownson RC, Glasgow RE, Morrato EH, Luke DA. Designing for dissemination and sustainability to promote equitable impacts on health. Annu Rev Public Health. 2022;43:331–53.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Ouyang M, Anderson CS, Song L, et al. Process evaluation of an implementation trial: design, rationale, and early lessons learnt from an international cluster clinical trial in intracerebral hemorrhage. Front Med (Lausanne). 2022;9: 813749.

    Article  PubMed  Google Scholar 

  40. Garner B, Knudsen H, Zulkiewicz B, et al. The setting-intervention fit of nine evidence-based interventions for substance use disorders within HIV service organizations across the United States: results of a national stakeholder-engaged real-time Delphi study. J Acquir Immune Defic Syndr. 2022;90(1):S206–14.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Fiechtner L, Castro I, Ayala SG, et al. Design and approach of the healthy weight clinic pediatric weight management intervention package and implementation: Massachusetts-CORD 3.0. Child Obes. 2021;17(S1):S48–54.

    Article  PubMed  Google Scholar 

  42. Sultanov M, Zeeuw J, Koot J, et al. Investigating feasibility of 2021 WHO protocol for cervical cancer screening in underscreened populations: PREvention and SCReening Innovation Project Toward Elimination of Cervical Cancer (PRESCRIP-TEC). BMC Public Health. 2022;22(1):1356.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Stump T, Ethier K, Hirschhorn L, et al. Development of an implementation facilitation strategy to link mental health screening and eHealth intervention for clients in Ryan Wite-funded clinics in Chicago. J Acquir Immune Defic Syndr. 2022;90(1):S197–205.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Lovero KL, Dos Santos PF, Adam S, et al. Leveraging stakeholder engagement and virtual environments to develop a strategy for implementation of adolescent depression services integrated within primary care clinics of Mozambique. Front Public Health. 2022;10: 876062.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Wood S, Meanley S, Bonett S, et al. Strengthening HIV prevention services through an implementation science approach: perspectives from the HIV testers in Philadelphia. J Acq Immune Defic Syndr. 2022;90(1):S90–7.

    Article  Google Scholar 

  46. Czosnek L, Zopf EM, Cormie P, Rosenbaum S, Richards J, Rankin NM. Developing an Implementation Research Logic Model: using a multiple case study design to establish a worked exemplar. Implement Sci Commun. 2022;3(1):90.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Knapp AA, Carroll AJ, Mohanty N, et al. A stakeholder-driven method for selecting implementation strategies: a case example of pediatric hypertension clinical practice guideline implementation. Implement Sci Commun. 2022;3(1):25.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Harkness A, Wawrzyniak A, Kolber M, et al. Multilevel determinants of rapid antiretroviral treatment implementation and demand in Miami-Dade County. J Acquir Immune Defic Syndr. 2022;90(1):S177–89.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Creger T, Burgan K, Turner W, et al. Using implementation mapping to ensure the success of PrEP optimization through enhanced contiuum tracking (PrOTECT) AL-a structural intervention to track statewide PrEP care continuum in Alabama. J Acquir Immune Defic Syndr. 2022;90(1):S161–6.

    Article  PubMed  PubMed Central  Google Scholar 

  50. King KM, Balan S, Kanamori M, et al. Feasibility and acceptability of HIV self-test kit distribution through PrEP clients’ social and sexual networks to increase HIV testing and PrEP information. J Acquir Immune Defic Syndr. 2022;90(1):S105–13.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Corneli A, Perry B, Wilson J, et al. Identification of determinants and implementation strategies to increase PrEP uptake among Black same gender-loving men in Mecklenburg County, North Carolina: the PrEP-Mech study. J Acquir Immune Defic Syndr. 2022;90(1):S149–60.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity Implementation Science. 2007;7(40)

  53. Cohen DJ, Crabtree BF, Etz RS, et al. Fidelity versus flexibility. Am J Prev Med. 2008;35(5):S381–9.

    Article  PubMed  Google Scholar 

  54. Perez Jolles M, Lengnick-Hall R, Mittman BS. Core functions and forms of complex health interventions: a patient-centered medical home illustration. J Gen Intern Med. 2019;34(6):1032–8.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Sanft T, Day A, Peterson L, et al. NCCN Guidelines(R) insights: survivorship Version 1.2022. J Natl Compr Canc Netw. 2022;20(10):1080–90.

    Article  PubMed  Google Scholar 

Download references


We thank Umber Dickerson for her ongoing support throughout Project CONNECT, the Project CONNECT nurse coordinators for their diligent work, and Parkland system and clinic leaders for their support and flexibility through the study. We thank our anonymous reviewers for deep engagement with our work and assistance in assuring clarity.


This study is funded by a research grant from the National Cancer Institute (R01CA203856).

Author information

Authors and Affiliations



SCL, BAB, and SAR conceptualized the study and provided feedback and edits throughout. SAR outlined and developed the first draft of the manuscript. RTH and PC collected data, and RTH, PC, and SAR conducted data analysis. SAR led data synthesis with SCL, BAB, RTH, and PC. RE, NS, and NS provided edits throughout. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Serena A. Rodriguez.

Ethics declarations

Ethics approval and consent to participate

Study procedures were approved by the University of Texas Southwestern (UTSW) IRB (STU 102015–090) and the Office of Research Administration, Parkland Health and Hospital System. UTSW is the IRB of record for Parkland Health and Hospital System.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table 1.

Immersion/crystallization cycles of data analysis. Table 2. Codes and themes for immersion/crystallization cycles of data analysis.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rodriguez, S.A., Lee, S.C., Higashi, R.T. et al. Factors influencing implementation of a care coordination intervention for cancer survivors with multiple comorbidities in a safety-net system: an application of the Implementation Research Logic Model. Implementation Sci 18, 68 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: