Skip to main content

Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR)

Abstract

Background

Qualitative approaches, alone or in mixed methods, are prominent within implementation science. However, traditional qualitative approaches are resource intensive, which has led to the development of rapid qualitative approaches. Published rapid approaches are often inductive in nature and rely on transcripts of interviews. We describe a deductive rapid analysis approach using the Consolidated Framework for Implementation Research (CFIR) that uses notes and audio recordings. This paper compares our rapid versus traditional deductive CFIR approach.

Methods

Semi-structured interviews were conducted for two cohorts of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE). The CFIR guided data collection and analysis. In cohort A, we used our traditional CFIR-based deductive analysis approach (directed content analysis), where two analysts completed independent in-depth manual coding of interview transcripts using qualitative software. In cohort B, we used our new rapid CFIR-based deductive analysis approach (directed content analysis), where the primary analyst wrote detailed notes during interviews and immediately “coded” notes into a MS Excel CFIR construct by facility matrix; a secondary analyst then listened to audio recordings and edited the matrix. We tracked time for our traditional and rapid deductive CFIR approaches using a spreadsheet and captured transcription costs from invoices. We retrospectively compared our approaches in terms of effectiveness and rigor.

Results

Cohorts A and B were similar in terms of the amount of data collected. However, our rapid deductive CFIR approach required 409.5 analyst hours compared to 683 h during the traditional deductive CFIR approach. The rapid deductive approach eliminated $7250 in transcription costs. The facility-level analysis phase provided the greatest savings: 14 h/facility for the traditional analysis versus 3.92 h/facility for the rapid analysis. Data interpretation required the same number of hours for both approaches.

Conclusion

Our rapid deductive CFIR approach was less time intensive and eliminated transcription costs, yet effective in meeting evaluation objectives and establishing rigor. Researchers should consider the following when employing our approach: (1) team expertise in the CFIR and qualitative methods, (2) level of detail needed to meet project aims, (3) mode of data to analyze, and (4) advantages and disadvantages of using the CFIR.

Peer Review reports

Background

Qualitative methods are invaluable for gathering in-depth information about “how and why efforts” to implement Evidence-Based Innovations (EBIs) succeed or fail [1]. As a result, qualitative approaches (alone or within mixed methods) are foundational for implementation scientists seeking to identify and understand factors that help or hinder the implementation and use of EBIs in real-world settings [2, 3]. Traditional qualitative approaches, however, are resource intensive, which challenges constrained study timelines and budgets. This is especially problematic in studies where scientists need real-time data to inform the process of implementation [4].

Consequently, qualitative researchers are working to develop methods that balance rigor and efficiency. The need for this balance is particularly salient in healthcare, where treatments and interventions are rapidly evolving, and evaluations of such interventions are constrained by limited timelines, funding, and staffing [5]. As a result, rapid assessment, which often involves streamlined processes for qualitative data collection and analysis, is gaining increased attention as a way to support quicker implementation and dissemination of EBIs to reduce delays in translating clinical research into practice [6,7,8,9,10].

An important element of rapid assessment is rapid qualitative analysis, which is the focus of this paper. Traditionally, qualitative analysis approaches have been resource intensive and occur over a longer timeframe; they include, but are not limited to, constant comparison, content, discourse, or thematic analysis [11,12,13,14,15,16]. Many traditional qualitative analysis approaches include in-depth manual coding of transcripts using software programs. In contrast, rapid qualitative analysis is deliberately streamlined and designed to be less resource intensive in order to meet a shorter timeframe [17,18,19]. Rapid qualitative analysis may involve eliminating transcription altogether or speeding up transcription processes [19] and then summarizing data into post-interview notes, templates based on the interview guide, and/or matrix summaries rather than in-depth manual coding of transcripts [9, 17,18,19,20,21]. Though rapid analysis is not wedded to a particular approach (e.g., content or thematic), some traditional qualitative analysis approaches may be more difficult to streamline. Rapid qualitative analysis is crucial when results are needed to quickly develop or modify implementation strategies and/or inform stakeholders or operational partners [5, 7, 9, 10, 19]. Rapid qualitative analysis is also useful during longitudinal implementation research since data points can become unwieldy, and results may be needed to inform future waves of data collection [22].

Hamilton developed a rapid qualitative analysis approach that summarizes transcript data into templates using domains aligned with interview questions; summary points are then distilled into a matrix organized by domain and participant for analysis and interpretation [18]. Gale et al. adapted this rapid approach in a process evaluation of academic detailing and compared it with a traditional analysis approach [23]. Their rapid approach involved summarizing transcripts into a template and then mapping themes onto the Consolidated Framework for Implementation Research (CFIR), a determinant framework that defines constructs across five domains of potential influences on implementation [23,24,25,26]. Gale et al. demonstrated consistency between results from rapid qualitative analysis versus traditional qualitative analysis. The traditional approach, however, “took considerably (69 days) longer than the rapid analysis to complete” [23]. Similarly, Holdsworth et al. noted that their modified version of rapid qualitative analysis “produced contextually-rich information” and can be used to save “days and weeks of costly transcription and analysis time” [27]. Except for Taylor et al.’s [20] comparison of rapid and thematic analysis, most rapid analysis literature focuses on daily duration and does not quantify reductions in analyst hours and costs at the activity level versus the project overall.

The rapid approaches described by Hamilton and Gale et al. rely on verbatim transcripts, which means teams must wait for transcription to be completed to proceed with rapid or traditional analyses. In contrast, Neal et al. [28] developed an approach to rapidly identify themes directly from audio recordings. However, Gale et al. [23] noted that because this approach relies on general domains, rather than framework informed codes, it “limits one’s ability to compare findings across projects unless findings are [subsequently] mapped to a framework.” As implementation scientists using the CFIR to guide our evaluations, we sought to build on prior rapid analysis approaches by developing a CFIR informed deductive rapid analysis process using notes and audio recordings. The objective of this article is to compare two different qualitative analysis processes using the CFIR: a traditional deductive approach using transcripts and a rapid deductive approach using notes and audio recordings.

Methods

Evaluation background

We conducted a mixed-methods evaluation of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE), which seeks to identify and diffuse EBIs. These EBIs include innovations supported by evidence from research studies and administrative or clinical experience [29, 30] and strive to address patient, staff, and/or facility needs. The DoE hosts an annual “Shark Tank” competition, in which VHA leaders compete to implement an EBI with 6 months of external implementation support; for additional detail, see previous publications [31,32,33,34]. As part of a national evaluation of the DoE, we identified barriers and facilitators to the implementation of these EBIs in VHA facilities using semi-structured interviews [31]. The qualitative interview and analysis team included CR (MPH, a senior qualitative analyst and CFIR expert user) and AN (PhD, a senior qualitative methodologist and CFIR intermediate user). Per regulations outlined in VHA Program Guide 1200.21, this evaluation has been designated a non-research quality improvement activity.

Methods for the traditional and rapid approaches

Data collection: semi-structured interviews

Data collection methods were the same across both approaches; in effect, they will not be discussed in detail in this paper. In brief, we conducted semi-structured telephone interviews with DoE participants involved with implementing an EBI; for additional detail, see previous publications [31, 35]. Interview guides were informed by the CFIR (see Additional File 1). Cohort A included 57 interviews across 17 facilities (1–4 interviews/facility) from June 2017 to September 2017; because one facility only had one interview, the need to aggregate data for that facility was eliminated. Cohort B included 72 interviews across 16 facilities (3–6 interviews/facility) from May 2019 to September 2019. Although cohort B included more interviews, the interviews were on average shorter (approximately 30 min), so both cohorts had approximately 50 audio hours total.

Data analysis: traditional and rapid approaches

The steps in our CFIR-based deductive traditional and deductive rapid qualitative analysis approaches are described in Table 1. The traditional CFIR approach is described in detail on www.cfirguide.org and in several publications [31, 36,37,38]. Our traditional CFIR approach is a form of directed content analysis [11] using transcripts and consisted of the following steps:

  1. 1.

    The analysts independently coded verbatim transcripts using Dedoose [39], a collaborative qualitative software program. The codebook included deductive CFIR constructs as well as inductive codes not captured in the CFIR that were relevant to the evaluation. Analysts used comments within coding software to flag sections of text for discussion or add additional notes.

  2. 2.

    The analysts met weekly to adjudicate differences in coding.

  3. 3.

    The primary analyst exported and aggregated coded data in MS Word CFIR facility memos (one for each facility). See Table 2 and Additional File 2.

  4. 4.

    The primary analyst summarized and rated coded data and wrote high-level facility summaries in each facility memo. The secondary analyst reviewed the primary analyst’s drafts of the facility memos and edited the summaries, ratings, and high-level facility summaries. Ratings were based on two factors: (1) valence (positive or negative influence on implementation) and (2) strength (weak or strong influence on implementation). Analysts used comments and highlighting in the facility memo to flag sections of text for discussion. Completed facility memos ranged from 68 to 148 pages with an average of 108 pages.

  5. 5.

    The analysts met weekly to adjudicate differences and refine the codebook.

  6. 6.

    The primary analyst copied the summaries, ratings, and high-level facility summaries from each facility memo into the MS Excel CFIR construct by facility matrix for interpretation; the matrix included all codes from the codebook (both deductive and inductive codes) as well as a row for high-level facility summaries. See Table 3 and Additional File 3.

Table 1 Traditional versus rapid approach using the CFIR
Table 2 Abridged CFIR facility memo template
Table 3 Snippet of CFIR construct by facility matrix

In contrast, our rapid CFIR approach is a form of directed content analysis [11] using interview notes and verification with audio recordings, which consisted of the following steps:

  1. 1.

    The primary analyst took notes and captured quotations during interviews. Immediately after the interviews, the primary analyst “coded” the notes into the MS Excel CFIR construct by facility matrix and noted when additional detail or a timestamp was needed. The secondary analyst then reviewed the matrix, listened to the audio recordings, and edited and built upon the primary analyst’s notes. Analysts coded based on a codebook with deductive CFIR constructs as well as inductive codes not captured in the CFIR that were relevant to the evaluation. Analysts used comments and highlighting in the matrix to flag sections of text for discussion.

  2. 2.

    Analysts met weekly to adjudicate differences and refine the codebook.

  3. 3.

    The primary analyst reviewed notes, rated CFIR constructs, and wrote a high-level facility summary for each facility in the matrix; the secondary analyst reviewed the matrix and edited ratings and high-level facility summaries. Ratings were determined based on two factors: (1) valence (positive or negative influence on implementation) and (2) strength (weak or strong influence on implementation). See Table 3 and Additional File 3.

  4. 4.

    Analysts met weekly to adjudicate differences.

Data interpretation: facility and construct analyses

Data interpretation methods were the same across both approaches and are discussed in detail on www.cfirguide.org. In brief, the analysts completed the following steps: (1) facility (case) analyses, to identify constructs that influenced implementation outcomes in each facility, and (2) construct analyses, to identify CFIR constructs that manifested positively or negatively across facilities or distinguished between facilities with high and low implementation success.

Methods for comparing traditional and rapid approaches

Comparing time and transcription costs

The team tracked time for data management, data collection, data analysis, and data interpretation for both approaches using MS Excel spreadsheets. Staff time for these tasks is based on hours. We also combined both analyst’s funded effort to determine the total available analyst hours for our evaluation. Transcription costs were obtained from invoices from a centralized VHA qualitative interview transcription service.

Comparing effectiveness and rigor

The team did not plan to compare the effectiveness or rigor of our traditional versus rapid approach (see the “Limitations” section). As a result, we defined and assessed these aspects retrospectively. Effectiveness was measured by whether we met our evaluation objective in each approach. Rigor was measured primarily by assessing the credibility of each approach, i.e., if evaluation processes established confidence that the results were accurate [40, 41].

Results

Comparing traditional and rapid approaches

Time and transcription costs

The traditional approach required more time than the rapid approach and included transcription costs. Cohort A, using the traditional deductive CFIR approach, required 683 total hours and $7250 in transcription costs. Cohort B, using the rapid deductive CFIR approach, required 409.5 total hours with no transcription costs. In effect, the rapid approach required 273.5 fewer total hours and saved $7250 in transcription costs. The evaluation funded two analysts with a combined total of 1305 h available for each year. Cohort A required 52.3% (683/1305 h) of the available hours while cohort B required 31.4% (409.5/1305 h) of the available hours, representing a significant reduction in time within the broader context of the evaluation. However, time savings during rapid analysis varied by phase, with the largest savings during the facility-level analysis. The following sections provide a summary of analyst hours and transcription costs for both approaches. See Table 1, Table 4, and Fig. 1 for additional description.

Table 4 Traditional versus rapid approach: differences in analyst hours and transcription costs
Fig. 1
figure 1

Comparison of analysis hours for the Traditional CFIR Approach (Cohort A) versus the Rapid CFIR Approach (Cohort B). This graph does not include data collection or data interpretation because both were equal across Cohort A and B

Data management

Data management in the traditional approach required 1 h to set-up the project and .5 h/interview plus .5 h/facility. In contrast, data management in the rapid approach required only .5 h to set-up the project with no other time needed. As shown in Table 1, the rapid approach eliminated data management steps except for creating the MS Excel CFIR construct by facility template. As a result, the rapid approach reduced analys time by 33.5 h. Though not directly impacting analyst hours, transcripts were not received for 2–6 weeks following interviews, significantly delaying analysis for the traditional approach. See Table 1, Table 4, and Fig. 1.

Data collection: semi-structured interviews

Data collection methods were the same across both approaches and the total number of audio hours was roughly equivalent between cohorts A and B; in effect, there were no significant differences in analyst hours between approaches. However, the rapid approach required blocking approximately 3 h for each interview: approximately 1 h for the interview plus 1–2 h to process the notes and “code” them into the CFIR construct by facility matrix immediately following the interview. The analyst’s immediate recall of the interview helped bolster the accuracy of the notes but intensified effort and cognitive load on interview days.

Data analysis

Data analysis in the traditional approach required 5.5 h/interview plus 14 h/facility versus 3.92 h/interview plus 3.92 h/facility in the rapid approach. In effect, the rapid approach reduced analys time by 79 h (275 versus 196 for traditional and rapid, respectively). The largest contributor to this reduction in analyst hours was in the facility-level analysis phase; where the rapid approach required 63 h, the traditional approach required 224 h. This difference was a result of how and when data were condensed and aggregated. In the traditional deductive CFIR approach, all coded data were aggregated in facility memos that were approximately 108 pages long; due to the relationships that often exist between constructs, the memos often included the same segments of text under multiple constructs. As a result, the same pieces of data were reviewed multiple times in full by each analyst independently before the data were condensed in the matrix. In contrast, the rapid deductive CFIR approach condensed data prior to aggregating by facility and was completed first by the primary analyst. Relationships between constructs were described once in the matrix, and notes in other cells referred back to this description, thus eliminating multiple references to the same data. The secondary analyst then built upon and confirmed the data in the matrix by listening to the audio recording. See Table 1, Table 4, and Fig. 1.

Data interpretation

Data interpretation methods were the same across both approaches, which consisted of reviewing the CFIR construct by facility matrix. Both approaches took approximately 100 h for data interpretation. See Table 1, Table 4, and Fig. 1.

Effectiveness and rigor

There were substantial differences in the number of hours and transcription costs between the traditional and rapid approaches; however, both approaches were systematic and there was concordance among many of the evaluation phases. Even when the analysis steps were different, both approaches followed the same general approach from data collection through data interpretation (see Table 1). Although data werecondensed earlier in the rapid approach than the traditional approach, i.e., following the interview versus following the facility memo, the depth of the data in the final matrices was similar for both approaches. For example, both matrices included brief direct quotes from participants. As a result, both approaches were effective in meeting our overall goal for the evaluation; we were able to identify and describe the factors influencing implementation in a high level of detail. However, the rapid approach also allowed us to share formal results more quickly with our operational partners (see Table 5).

Table 5 Traditional deductive CFIR approach versus rapid deductive CFIR approach: effectiveness and rigor

In addition, both approaches included processes to enhance methodological rigor [40, 41]. Credibility of results, a form of rigor, was most relevant when assessing tradeoffs between our rapid and traditional approaches [41]. We enhanced the credibility of results by having analysts with expertise in qualitative methods and the CFIR. To ensure participant responses were accurately captured in our summaries, we used two analysts per interview as a quality check and verified summaries with raw data (transcripts or audio recordings). Overall, the final summaries from both approaches were quite similar. See Table 5 for an additional description of the effectiveness and concordance of rigor between both approaches.

Discussion

Our rapid deductive CFIR approach has much potential value, given the urgent need for nearly real-time results, to guide the implementation and dissemination of EBIs. The goal of this paper was to compare two qualitative approaches using deductively derived codes based on the CFIR: a traditional deductive CFIR approach using verbatim transcripts versus a rapid deductive CFIR approach using notes and audio recordings. Although we used the CFIR, this approach can be used with other frameworks. Our paper enhances the literature by describing exactly how rapid deductive CFIR analysis versus traditional deductive CFIR analysis leads to less resource use without compromising rigor.

Although our rapid deductive CFIR approach was beneficial for our evaluation team, researchers should review four considerations before using this method: (1) team expertise in CFIR and qualitative methods, (2) level of detail needed to meet project aims, (3) mode of data to analyze, and (4) advantages and disadvantages of using the CFIR.

First, the team’s expertise in the CFIR and qualitative methods should be considered before deciding to employ a rapid approach. Prior literature suggests that traditional qualitative analysis requires more intense training than rapid analysis [23, 28]. In-depth qualitative methods should indeed be conducted by a skilled research team. However, we argue that our rapid deductive CFIR approach may be more suited to researchers who already have a strong foundation in qualitative methods and the CFIR. Qualitative researchers familiar with the CFIR are more equipped to rapidly “code” qualitative data into CFIR constructs in real time than a novice. However, even for skilled researchers, we found that rapid analysis intensified effort and cognitive load during the initial coding phase, e.g., requiring a 3-h calendar block. Although a more experienced team may cost more in terms of salaries, the experienced team works more efficiently and likely saves money overall by reducing time spent training and overseeing project staff. For less experienced teams, we suggest linking CFIR constructs and brief definitions directly to interview questions within a notes template; this will help guide the researcher when summarizing the interview and/or listening to the audio recording. However, it is important to note that participant responses to questions will not always address the intended construct. Furthermore, while we identified a high level of fidelity between the primary analyst’s notes and the audio recordings, the secondary analyst may serve as an essential quality check for less experienced teams.

Second, researchers should consider what level of detail is needed for data analysis and the presentation of results in order to meet the project’s aims [28]. As articulated in prior research, rapid approaches using notes and audio recordings may provide a “big picture” view, yielding a lower level of detail than transcript-based approaches [28]. A project that requires a high level of detail and/or long quotations may therefore not be appropriate for our rapid approach. Our rapid CFIR approach provided less detail, but in so doing, may have allowed us to see both the overall patterns and the important details in our data more efficiently, i.e., seeing both the forest and the trees.

Third, the mode of data (transcripts or audio recordings) should be considered since it is not necessarily associated with a traditional or rapid approach. For example, audio recordings can be used for traditional analysis, i.e., many types of qualitative software allow minute-by-minute coding of audio recordings, and transcripts can be used for rapid analysis, i.e., summaries can be developed based on transcripts instead of audio recordings. For our rapid approach, we chose to use post-interview notes and audio recordings instead of transcripts to help streamline our deductive CFIR analysis process, i.e., it eliminated transcription costs and delays, and provided a point of comparison with other existing rapid approaches that use transcripts. However, if a team desires a more rapid approach while also maintaining access to the data in written form, including transcripts may be an option.

Fourth, there are advantages and disadvantages to consider when opting to use the CFIR (or another framework) regardless of the rapid or traditional qualitative analysis approach. Using the CFIR is helpful because it is a comprehensive determinant framework that includes constructs from 19 other models, including work by Greenhalgh et al. [42] that reviewed 500 published sources across 13 scientific disciplines. In effect, the CFIR helps researchers identify determinants that may be overlooked in a purely inductive approach. In addition, the use of the CFIR assists researchers with sharing and comparing results across studies, which advances implementation science. However, if researchers overly rely on the CFIR (or another framework), they may overlook constructs or miss important insights not included in the framework. To address this concern, we included questions in our interview guide beyond the scope of the CFIR, e.g., anticipated sustainment, and added codes, as needed, to capture inductively derived determinants and outcomes. Overall, even when using a more deductive approach, it is important for researchers to be open to inductive topics or domains that may arise in the data. Ultimately, researchers should consider their goals when deciding whether to adopt a deductive rapid approach (i.e., more confirmatory to compare with existing constructs or knowledge) versus an inductive approach (i.e., more exploratory to generate new constructs or knowledge).

It is important to note that our rapid deductive CFIR approach was still time intensive; it took 409.5 h to complete the analysis, including the rating process, for cohort B. However, because the analysts completed interview notes and coding in the matrix immediately after each interview, we were able to share preliminary results during regularly scheduled meetings with our operational partners on an ongoing basis. Regardless, some researchers may need additional ways to streamline our rapid CFIR analysis process. As long as a team considers both strengths and limitations, the following strategies may provide ways to streamline our rapid CFIR approach:

  • The team could eliminate the second analyst entirely or only use a second analyst on a subset of interviews, e.g., on the first 10 interviews or a random sample.

  • The team could include only the CFIR constructs expected to be most relevant to the research question in the matrix.

  • The team could seek to obtain project artifacts, e.g., meeting minutes, to analyze in the place of interviews.

  • The team could omit the rating process following coding.

Although rapid approaches are becoming more alluring to many implementation science researchers, they should not be considered a quick and easy replacement for traditional approaches or a substitute for having a skilled research team. Teams must carefully consider the best approach for their project while also exploring how to maintain scientific rigor. Qualitative expert oversight and/or training, analyst familiarity with the framework, review by a secondary analyst, and interview data quality are some important aspects of methodological rigor.

Limitations

Several limitations should be noted. First, both analysts on this project were intermediate to expert CFIR users. Our approach may be more difficult for new CFIR users, i.e., it may be difficult to translate interview notes into “coded” data in the matrix or to “code” while listening to an audio recording, unless the researchers are very familiar with the constructs. Second, the same analysts were involved in analyzing both cohorts. It is possible the analysts were more familiar with the broader findings from the study based on the traditional analysis of cohort A, which may have allowed them to progress more quickly in the rapid analysis of cohort B. However, using the same analysts improves comparability of coding between the two different cohorts of data and streamlined the process because additional analysts did not need to be trained in using the CFIR. Future research is needed to assess the extent and the conditions under which our approach works for other CFIR users. Third, we focused on differences in time and transcription costs rather than specifically testing the effectiveness or rigor of our rapid versus traditional approach, which has been discussed in prior literature [23, 28]. While the rigor of the results was the same with both approaches, future researchers should likewise assess the rigor of this deductive rapid approach within their circumstances.

Conclusions

Our deductive rapid approach using the CFIR, involving notes and audio recordings, is an effective and rigorous approach for analyzing qualitative data that resulted in substantial reductions in time and transcription costs. We intend to use this approach for similar studies in the future. Overall, a deductive rapid approach using the CFIR (or another framework) is especially beneficial when (1) the research team has strong qualitative methods and skills using the framework, (2) the research timeline is relatively short or real-time feedback is needed, (3) the budget  does not support transcription, and (4) the research team wants to compare results across studies.

Availability of data and materials

The datasets generated and/or analyzed during the current evaluation are not available due to participant privacy but may be available from the corresponding author on reasonable request.

Abbreviations

CFIR:

Consolidated Framework for Implementation Research

DoE:

Diffusion of Excellence

EBI:

Evidence-Based Innovation

VHA:

Veterans Health Administration

References

  1. Hamilton AB, Finley EP. Qualitative methods in implementation research: an introduction. Psychiatry Res. 2019;280:112516. https://doi.org/10.1016/j.psychres.2019.112516.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Cohen D, Crabtree BF, Damschroder L, et al. Qualitative methods in implementation science. National cancer iInstitute. Division of cancer control and population sciences. https://cancercontrol.cancer.gov/sites/default/files/2020-09/nci-dccps-implementationsciencewhite-paper.pdf. Accessed 14 Apr 2021.

  3. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health Ment Health Serv Res. 2011;38(1):44–53. https://doi.org/10.1007/s10488-010-0314-z.

    Article  Google Scholar 

  4. Glasgow RE, Chambers D. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clin Transl Sci. 2012;5(1):48–55. https://doi.org/10.1111/j.1752-8062.2011.00383.x.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Ash J, Sittig D, McMullen C, Guappone K, Dykstra R, Carpenter J. A rapid assessment process for clinical informatics interventions. AMIA Ann Symp Proc. 2008;2008:26–30.

    Google Scholar 

  6. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20. https://doi.org/10.1258/jrsm.2011.110180.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise. Clin Transl Med. 2013;2(1). https://doi.org/10.1186/2001-1326-2-10.

  8. Vindrola-Padros C, Vindrola-Padros B. Quick and dirty? A systematic review of the use of rapid ethnographies in healthcare organisation and delivery. BMJ Qual Saf. 2018;27(4):321–30. https://doi.org/10.1136/bmjqs-2017-007226.

    Article  PubMed  Google Scholar 

  9. Palinkas LA, Zatzick D. Rapid assessment procedure informed clinical ethnography (RAPICE) in pragmatic clinical trials of mental health services implementation: methods and applied case study. Adm Policy Ment Health Ment Health Serv Res. 2019;46(2):255–70. https://doi.org/10.1007/s10488-018-0909-3.

    Article  Google Scholar 

  10. Palinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health. 2019;40(1):423–42. https://doi.org/10.1146/annurev-publhealth-040218-044215.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88. https://doi.org/10.1177/1049732305276687.

    Article  PubMed  Google Scholar 

  12. Saldana J. The coding manual for qualitative researchers. 2nd ed: Sage Publications; 2015.

  13. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa.

    Article  Google Scholar 

  14. Kenny M, Fourie R. Contrasting classic, Straussian, and constructivist grounded theory: methodological and philosophical conflicts. Qual Rep. Published online August 17, 2015. https://doi.org/10.46743/2160-3715/2015.2251.

  15. Bernard HR, Wutich A, Ryan GW. Analyzing qualitative data: systematic approaches: SAGE; 2017.

  16. Hodges BD, Kuper A, Reeves S. Discourse analysis. BMJ. 2008;337(aug07 3):a879. https://doi.org/10.1136/bmj.a879.

    Article  PubMed  Google Scholar 

  17. Sobo EJ, Simmes DR, Landsverk JA, Kurtin PS. Rapid assessment with qualitative telephone interviews: lessons from an evaluation of California’s healthy families program & Medi-Cal for children. Am J Eval. 2003;24(3):399–408. https://doi.org/10.1177/109821400302400308.

    Article  Google Scholar 

  18. Hamilton AB. Qualitative methods in rapid turn-around health services research. PowerPoint Present 2013 VA HSRD Cyberseminar Spotlight Womens Health. Published online 2013. Accessed November 16, 2020. www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780.

  19. Vindrola-Padros C, Johnson GA. Rapid techniques in qualitative research: a critical review of the literature. Qual Health Res. 2020;30(10):1596–604. https://doi.org/10.1177/1049732320921835.

    Article  PubMed  Google Scholar 

  20. Taylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S. Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open. 2018;8(10):e019993. https://doi.org/10.1136/bmjopen-2017-019993.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Koenig CJ, Abraham T, Zamora KA, Hill C, Kelly PA, Uddo M, et al. Pre-implementation strategies to adapt and implement a Veteran Peer Coaching intervention to improve mental health treatment engagement among rural veterans: pre-implementation of a mental health intervention. J Rural Health. 2016;32(4):418–28. https://doi.org/10.1111/jrh.12201.

    Article  PubMed  Google Scholar 

  22. Nevedal AL, Ayalon L, Briller SH. A qualitative evidence synthesis review of longitudinal qualitative research in gerontology. Heyn PC, ed. Gerontologist. 2019;59(6):e791–801. https://doi.org/10.1093/geront/gny134.

    Article  PubMed  Google Scholar 

  23. Gale RC, Wu J, Erhardt T, Bounthavong M, Reardon CM, Damschroder LJ, et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implement Sci. 2019;14(1):11. https://doi.org/10.1186/s13012-019-0853-y.

    Article  PubMed  PubMed Central  Google Scholar 

  24. The Consolidated Framework for Implementation Research (CFIR) technical assistance website. Accessed May 1, 2020. https://cfirguide.org/

  25. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. https://doi.org/10.1186/1748-5908-4-50.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013;8(1):51. https://doi.org/10.1186/1748-5908-8-51.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Holdsworth LM, Safaeinili N, Winget M, Lorenz KA, Lough M, Asch S, et al. Adapting rapid assessment procedures for implementation research using a team-based approach to analysis: a case example of patient quality and safety interventions in the ICU. Implement Sci. 2020;15(1):12. https://doi.org/10.1186/s13012-020-0972-5.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Neal JW, Neal ZP, VanDyke E, Kornbluh M. Expediting the analysis of qualitative data in evaluation: a procedure for the rapid identification of themes from audio recordings (RITA). Am J Eval. 2015;36(1):118–32. https://doi.org/10.1177/1098214014536601.

    Article  Google Scholar 

  29. Kilbourne AM, Goodrich DE, Miake-Lye I, Braganza M, Bowersox NW. Quality enhancement research initiative implementation (QUERI) roadmap: towards sustainability of evidence-based practices in a learning health system. Med Care. 2019;57(Suppl 3):S286–93. https://doi.org/10.1097/MLR.0000000000001144.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Rycroft-Malone J, Harvey G, Kitson A, McCormack B, Seers K, Titchen A. Getting evidence into practice: ingredients for change. Nurs Stand. 2002;16(37):38–43. https://doi.org/10.7748/ns2002.05.16.37.38.c3201.

    Article  PubMed  Google Scholar 

  31. Nevedal AL, Reardon CM, Jackson GL, Cutrona SL, White B, Gifford AL, et al. Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study. Implement Sci Commun. 2020;1(1):61. https://doi.org/10.1186/s43058-020-00053-1.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Clancy CM. Creating world-class care and service for our nation’s finest: how Veterans Health Administration diffusion of excellence initiative is innovating and transforming Veterans Affairs health care. Perm J. 2019;23:18.309. https://doi.org/10.7812/TPP/18.301.

  33. Vega RJ, Jackson GL, Henderson B, et al. Diffusion of excellence: accelerating the spread of clinical innovation and best practices across the nation’s largest health system. Perm J. 2019;23:18.309.

  34. Jackson GL, Cutrona SL, White B, et al. Implementation practice and science to scale-up innovative practice: VHA diffusion of excellence. Jt Comm J Qual Patient Saf. 2021;47(4):217–27.

  35. Jackson G, Cutrona S, White B, Reardon C, Orvek E, Nevedal A, et al. Identifying, replicating, and spreading health care innovations across a nation-wide health care system: VHA diffusion of excellence. Health Serv Res. 2020;55(S1):60. https://doi.org/10.1111/1475-6773.13411.

    Article  Google Scholar 

  36. Damschroder LJ, Reardon CM, Sperber N, Robinson CH, Fickel JJ, Oddone EZ. Implementation evaluation of the Telephone Lifestyle Coaching (TLC) program: organizational factors associated with successful implementation. Transl Behav Med. 2017;7(2):233–41. https://doi.org/10.1007/s13142-016-0424-6.

    Article  PubMed  Google Scholar 

  37. Damschroder LJ, Goodrich DE, Robinson CH, Fletcher CE, Lowery JC. A systematic exploration of differences in contextual factors related to implementing the MOVE! weight management program in VA: a mixed methods study. BMC Health Serv Res. 2011;11(1):248. https://doi.org/10.1186/1472-6963-11-248.

    Article  PubMed  Google Scholar 

  38. Damschroder LJ, Reardon CM, AuYoung M, Moin T, Datta SK, Sparks JB, et al. Implementation findings from a hybrid III implementation-effectiveness trial of the Diabetes Prevention Program (DPP) in the Veterans Health Administration (VHA). Implement Sci. 2017;12(1):94. https://doi.org/10.1186/s13012-017-0619-3.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Dedoose Version 8.0.35, Web application for managing, analyzing, and presenting qualitative and mixed method research data. SocioCultural Research Consultants, LLC; 2018. www.dedoose.com. Accessed 13 Apr 2021.

  40. Maher C, Hadfield M, Hutchings M, de Eyto A. Ensuring rigor in qualitative data analysis: a design research approach to coding combining NVivo with traditional material methods. Int J Qual Methods. 2018;17(1):160940691878636. https://doi.org/10.1177/1609406918786362.

    Article  Google Scholar 

  41. Forero R, Nahidi S, De Costa J, et al. Application of four-dimension criteria to assess rigour of qualitative research in emergency medicine. BMC Health Serv Res. 2018;18(1):120. https://doi.org/10.1186/s12913-018-2915-2.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The opinions expressed in this article are those of the authors and do not represent the views of VHA or the US Government. We are grateful to the VHA employees who participated in this evaluation and shared their experiences with us.

Funding

This evaluation was funded by the Veterans Health Administration (VHA) Quality Enhancement Research Initiative (QUERI) [PEC-17-002] with additional funding subsequently provided by the VHA Office of Rural Health through the Diffusion of Excellence.

Author information

Authors and Affiliations

Authors

Contributions

LD, GJ, SC, CR, BW, AN, and MOW were engaged in the national evaluation of the Diffusion of Excellence. GJ, LD, and SC designed and supervised the overall evaluation. AN, CR, LD, and MOW led the data collection, analysis, and manuscript writing for this aspect of the evaluation. AN, CR, LD, MOW, GJ, SC, and BW were involved in the critical revision of the manuscript for intellectual content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Andrea L. Nevedal.

Ethics declarations

Ethics approval and consent to participate

Per regulations outlined in VHA Program Guide 1200.21, this evaluation has been designated a non-research quality improvement activity.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Interview Guide.

Additional file 2.

Unabridged MS Word CFIR Facility Memo Template.

Additional file 3.

Unabridged MS Excel CFIR Construct by Facility Matrix Template.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nevedal, A.L., Reardon, C.M., Opra Widerquist, M.A. et al. Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR). Implementation Sci 16, 67 (2021). https://doi.org/10.1186/s13012-021-01111-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-021-01111-5

Keywords