Skip to main content

Table 1 Statements by cluster by bridging

From: Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support

Cluster 1: RQ: organizational issues
83 Paying special attention to issues related to the organizational learning curve and other issues related to evolution or drift of the practices .41
77 Understanding when innovation or the natural life cycle of a program takes precedence over sustainability .43
34 Identifying organizational adopter/sustainer archetypes .45
30 Considering the organization as a learning organization rather than sustainability as an end-point .52
76 Developing guidance for how organizations should decide to end a program and instead adopt a newer/better/more effective one .69
5 Understanding the “rapid learning” or problem solving skills needed by key individuals/leaders and organizations in order to respond to changing environmental challenges .78
  Count: 6 Std. Dev.: 0.14 Minimum .041 Average: .55
Variance: 0.02 Maximum .078 Median: 0.49
Cluster 2: research and program funding
70 Funding .09
67 Finding funding sources for conducting sustainability research .22
86 Understanding and commitment on the part of funders for such work .46
82 The availability of RFAs that explicitly call for sustainability research .48
6 Determining why implementing agencies do/do not seek continued funding .51
19 Fully exploring the role of adequate funding in sustainability .62
41 Funds available to sustain a program .70
36 Making sure that the research addresses required infrastructure and a viable business model to provide longstanding revenue support for the program .73
  Count: 8 Std. Dev.: 0.21 Minimum 0.09 Average: .47
Variance: 0.04 Maximum 0.73 Median: 0.49
Cluster 3: research practice and training
22 Establishing training in sustainability research .20
15 Developing and implementing graduate curriculum relevant for sustainability researchers .29
56 Training and capacity building in a public health agency .42
91 Training and technical support to providers/deliverers of the program/intervention .49
45 Identifying a broad set of journals and professional conferences that are good places for dissemination of sustainability research .68
  Count: 5 Std. Dev.: 0.16 Minimum 0.20 Average: .42
Variance: 0.03 Maximum 0.68 Median: 0.42
Cluster 4: research question: ROI
78 Determining the return-on-investment of sustainability given a variety of likely parameters (e.g., number of individuals reached; number of infections averted) .43
50 Conducting ROI (return on investment) studies to make it clear to stakeholders and funders how much is actually gained when effective programs are sustained .43
33 Determining the return-on-investment of sustainability during different time periods (e.g., 6-months, 12-months, 24-months) .46
9 Investigating to what extent benefits (e.g. cost-savings, improved clinical outcomes) are sustained along with sustained use/behavior .47
84 Conducting longitudinal cost-benefit analysis comparing implementation vs. sustainability .50
46 Considering cost and economic issues from multiple perspectives .66
23 Taking advantage of research already conducted in other fields and industries regarding sustained implementation of technologies and practices .75
74 Creating survey of disciplines most involved in sustainability research 1.00
  Count: 8 Std. Dev.: 0.19 Minimum 0.43 Average: .59
Variance: 0.04 Maximum 1.00 Median: 0.48
Cluster 5: research question: factors affecting sustainability
13 Figuring out how to predict sustainability based on the experience or knowledge of context gained through implementation .23
39 Identifying key features of Evidenced Based Practice associated with variations in sustainability (e.g., strength of evidence; consistency with established practice; size of difference between prior and the new/evidence-based practices; staffing/other) .25
53 Understanding which variables and factors are more important for sustainability than others .25
7 Identifying key or core program sustainability components .25
75 Investigating the relationship between sustained use, routinization and resistance to change .29
12 Exploring whether the factors influencing sustainability differ from those influencing implementation .30
60 Understanding if certain types of programs are less likely to be sustained .30
69 Identifying common and independent factors that drive adoption vs. initial implementation vs. long-term use .35
42 Exploring the supporting interventions (e.g., feedback) that are needed to sustain behaviors/use, for how long, and what intensity .35
88 Understanding the reasons why strategies are/are not sustained .39
  Count: 10 Std. Dev.: 0.05 Minimum 0.23 Average: .30
Variance: 0.00 Maximum 0.39 Median: 0.30
Cluster 6: research question: adaptation
64 Understanding the tension between fidelity and adaptation as it pertains to sustained use or feasibility of continued use .13
21 Documenting adaptations and their impact on the effectiveness of evidence-based practices .14
47 Determining the core vs. peripheral or adaptable components of interventions .14
55 Figuring out how to characterize adaptations .16
54 Determining the point at which the intervention or program can no longer be considered sustained because of extensive adaptations .18
66 Considering how the evidence-based program may change over time .19
38 Understanding the implications of “partial” sustainability and adaptations .24
28 Identifying interventions that are effective and cost-effective in fostering sustainability .33
4 Framing sustainability as a partnership in which participants continue to adapt an intervention in response to changing conditions, while trying to remain true to its core principles .34
  Count: 9 Std. Dev.: 0.08 Minimum 0.13 Average: .21
Variance: 0.01 Maximum 0.34 Median: 0.18
Cluster 7: research question: environment
17 Defining the key attributes of organizations and systems that successfully sustain effective practice (e.g., ongoing leadership attention, ongoing measurement, systematic hardwiring of effective innovation, etc.) .20
49 Identifying the key contextual factors (e.g., organizational characteristics) associated with variations in sustainability .21
58 Defining and assessing the multiplicity of environmental variables that are likely to affect sustainability .26
51 Discerning situations in which sustained use may be at odds with adopters’ (e.g., organizations) best interests .31
8 Characterizing the context or environment of the intervention to be sustained .33
65 Understanding cultural barriers to adoption .33
62 Identifying which factors that advance or inhibit sustainability are amenable to management intervention .53
63 Understanding how to sustain programs/policies in low resource settings .71
  Count: 8 Std. Dev.: 0.16 Minimum 0.20 Average: .36
Variance: 0.03 Maximum 0.71 Median: 0.32
Cluster 8: research stage: measurement
2 Developing methods for studying sustainability across the complexity dimension (e.g., sustainability of a specific clinical treatment vs. sustainability of a complex state-level chronic disease program) .00
3 Using multilevel measurement .00
37 Developing measures of sustainability (overall and sub-dimensions) .01
18 Determining which analytic methods are most appropriate for sustainability research .02
48 Identifying appropriate study designs for measuring sustainability .04
85 Considering the role of self-reported data in assessing sustainability outcomes .09
68 Identifying valid data sources for assessing sustainability .10
81 Deciding how long to follow-up on a newly implemented program to determine whether it has been sustained .10
27 Developing and validating fidelity measures for assessing adherence of a program to an evidence-based model .11
89 Constructing reliable and validated tools to measure core sustainability constructs .13
61 Having an operational definition of sustainability, with measurable criteria .14
44 Determining “what” should measured as an outcome (patient level outcomes? fidelity? program activities? capacity?) .26
14 The ability to move rapidly and use innovative research methods to learn from emerging opportunities and examples .29
  Count: 13 Std. Dev.: 0.09 Minimum 0.00 Average: .10
Variance: 0.01 Maximum 0.29 Median: 0.10
Cluster 9: research stage: design and analysis
72 Discussing how systems science methods such as modeling and network analysis can be used to study important sustainability questions .03
59 Increasing use of non-experimental study designs in sustainability research .08
43 Using a participatory approach to research that values the perspective of developers of original idea as well as the target group .14
35 Clearly defining research hypotheses and goals linked to real-world application outcomes .19
87 Identifying the indicators of sustained use so that we will know it when we see it .25
29 Developing ways researchers can better integrate their documentation needs into the agency, so that it creates a smaller burden on those who deliver care (e.g., integrated data collection with electronic records) .39
90 Conducting observational research of implemented programs to identify barriers/facilitators to sustainability .42
  Count: 7 Std. Dev.: 0.14 Minimum 0.03 Average: .21
Variance: 0.02 Maximum 0.42 Median: 0.19
Cluster 10: research stage: frameworks
11 Discussing whether to study sustainability separately from implementation (e.g., How are the two related? not related?) .28
10 Testing of theories/frameworks for sustainability .40
31 Creating greater distinction between predictors of sustainability (e.g., organizational capacity) and sustainability outcomes (sustained programming) .42
26 Studying a greater variety of sustained activities, including interventions, programs, and policies .43
16 Assessing the dynamic processes underlying sustained use .50
57 Developing case studies to identify key characteristics of those that do sustain vs. do not .63
  Count: 6 Std. Dev.: 0.11 Minimum 0.28 Average: .44
Variance: 0.01 Maximum 0.63 Median: 0.42
Cluster 11: research stage: definitions
24 Conceptualizing and defining “sustainability”, its sub-dimensions, and related concepts (e.g., fidelity, routinization, institutionalization) .08
79 Defining sustainability outcomes .10
73 Creating a visual depiction of a sustainability model .16
32 Identifying dimensions and degrees of sustained use .16
80 Clarifying terminology (e.g., assimilation, institutionalization, continued use) .17
71 Developing a formal conceptual model that links dissemination, implementation, and sustainability .17
40 Balancing the use of pertinent theory, evidence, and experience .29
20 Determining whether there is a standard level of initial “success” necessary before a project or organization or community is “eligible” to be considered for a “sustainability” evaluation .32
  Count: 8 Std. Dev.: 0.08 Minimum 0.08 Average: .18
Variance: 0.01 Maximum 0.32 Median: 0.17