Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):2059–67.
Article
Google Scholar
Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39:55–76.
Article
Google Scholar
Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7(1):1–19.
Article
Google Scholar
Herlitz L, MacIntyre H, Osborn T, Bonell C. The sustainability of public health interventions in schools: a systematic review. Implement Sci. 2020;15(4). https://doi.org/10.1186/s13012-019-0961-8.
Ament SMC, de Groot JJA, Maessen JMC, Dirksen CD, van der Weijden T, Kleinjnen J. Sustainability of professionals' adherence to clinical practice guidelines in medical care: a systematic review. BMJ Open. 2015;5:e008073.
Article
Google Scholar
Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10:88.
Article
Google Scholar
Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017;12(1):110.
Article
Google Scholar
Moullin JC, Sklar M, Green A, Dickson KS, Stadnick NA, Reeder K, et al. Advancing the pragmatic measurement of sustainment: a narrative review of measures. Implement Sci Commun. 2020;1:76.
Article
Google Scholar
Birken SA, Haines ER, Hwang S, Chambers DA, Bunger AC, Nilsen P. Advancing understanding and indetifying strategies for sustaining evidence-based practices: a review of reviews. Implement Sci. 2020;15(88). https://doi.org/10.1186/s13012-020-01040-9.
Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The program sustainability assessment tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:E12.
Article
Google Scholar
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):1–11.
Article
Google Scholar
Hodge LM, Turner KMT. Sustained implementation of evidence-based programs in disadvantaged communities: a conceptual framework of supporting factors. Am J Community Psychol. 2016;58(1-2):192–210.
Article
Google Scholar
Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, et al. Public health programs capacity for sustainability: a new framework. Implement Sci. 2013;8:15.
Article
Google Scholar
Shoesmith A, Hall A, Wolfenden L, Shelton RC, Powell BJ, Brown H, et al. Barriers and facilitators influencing the sustainment of health behaviour interventions in schools and childcare services: a systematic review. Implement Sci. 2021;16(62). https://doi.org/10.1186/s13012-021-01134-y.
Boateng GO, Neilands TB, Frongillo EA, Melgar-Quinonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front. Public Health. 2018;6(149). https://doi.org/10.3389/fpubh.2018.00149.
Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a delphi study. Qual Life Res. 2018;27(5):115–1170.
Article
Google Scholar
Lewis CC, Mettert KD, Stanick CF, Halko HM, Nolen EA, Powell BJ, et al. The psychometric and pragmatic evidence rating scale (PAPERS) for measure development and evaluation. Implement Res Pract. 2021;2:1–6.
Google Scholar
Mettert K, Lewis C, Dorsey C, Halko H, Weiner B. Measuring implementation outcomes: an updated systematic review of measures’ psychometric properties. Implement Res Pract. 2020;1:1–29.
Google Scholar
Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21(4):651–7.
Article
Google Scholar
Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10:155.
Article
Google Scholar
Page MJ, et al. PRISMA 2020 explanation and elaboration: updated gudance and exemplars for reporting systematic reviews. BMJ. 2021;372(n160). https://doi.org/10.1136/bmj.n160.
Allen P, Pilar M, Walsh-Bailey C, Hooley C, Mazzucca S, Lewis CC, et al. Quantitative measures of health policy implementation determinants and outcomes: a systematic review. Implement Sci. 2020;15(1):47.
Article
Google Scholar
Weiner BJ, Mettert KD, Dorsey CN, Nolen EA, Stanick C, Powell BJ, et al. Measuring readiness for implementation: a systematic review of measures’ psychometric and pragmatic properties. Implement Res Pract. 2020;1:1–29.
Google Scholar
Clinton-McHarg T, Yoong SL, Tzelepis F, Regan T, Fielding A, Skelton E, et al. Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the consolidated framework for implementation research: a systematic review. Implement Sci. 2016;11(1):148.
Article
Google Scholar
Khadjesari Z, Boufkhed S, Vitoratou S, Schatte L, Ziemann A, Daskalopoulou C, et al. Implementation outcome instruments for use in physical healthcare settings: a systematic review. Implement Sci. 2020;15(1):66.
Article
Google Scholar
Khadjesari Z, Vitoratou S, Sevdalis N, Hull L. Implementation outcome assessment instruments used in physical healthcare settings and their measurement properties: a systematic review protocol. BMJ Open. 2017;7(10):e017972.
Article
Google Scholar
Lewis CC, Mettert KD, Dorsey CN, Martinez RG, Weiner BJ, Nolen E, et al. An updated protocol for a systematic review of implementation-related measures. Syst Rev. 2018;7(1):66.
Article
Google Scholar
Society for Implementation Research and Collaboration. Sustainability Instruments. Available from: https://societyforimplementationresearchcollaboration.org/sustainability-measures/.
Grid-Enabled Measures Database. GEM. Available from: https://www.gem-measures.org/Login.aspx?ReturnURL=Public/Measurelist.aspx?cat=2.
Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia. Available at www.covidence.org.
Stanick CF, Halko HM, Nolen EA, Powell BJ, Dorsey CN, Mettert KD, et al. Pragmatic measures for implementation research: development of the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). Transl Behav Med. 2021;11(1):11–20.
Article
Google Scholar
Moullin JC, Sklar M, Ehrhart MG, Green A, Aarons GA. Provider REport of sustainment Scale (PRESS): development and validation of a brief measure of inner context sustainment. Implement Sci. 2021;16(1):86.
Article
Google Scholar
Chamberlain P, Hendricks Brown C, Saldana L. Observational measure of implementation progress in community based settings: the Stages of Implementation Completion (SIC). Implement Sci. 2011;6:116.
Article
Google Scholar
Saldana L, Bennett I, Powers D, Vredevoogd M, Grover T, Schaper H, et al. Scaling implementation of collaborative care for depression: adaptation of the stages of implementation completion (SIC). Admin Pol Ment Health. 2020;47(2):188–96.
Article
Google Scholar
Saldana L, Chamberlain P, Wang W, Hendricks BC. Predicting program start-up using the stages of implementation measure. Admin Pol Ment Health. 2012;39(6):419–25.
Article
Google Scholar
Turri MG, Mercer SH, McIntosh K, Nese RNT, Strickland-Cohen MK, Hoselton R. Examining barriers to sustained implementation of school-wide prevention practices. Assess Eff Interv. 2016;42(1):6–17.
Google Scholar
Kittelman A, Mercer SH, McIntosh K, Nese RNT. Development and validation of a measure assessing sustainability of tier 2 and 3 behavior support systems. J Sch Psychol. 2021;85:140–54.
Article
Google Scholar
Slaghuis SS, Strating MMH, Bal RA, Nieboer AP. A measurement instrument for spread of quality improvement in healthcare. Int J Qual Health Care. 2013;25(2):125–31.
Article
CAS
Google Scholar
Solberg LI, Asche SE, Margolis KL, Whitebird RR. Measuring an organization's ability to manage change: the change process capability questionnaire and its use for improving depression care. Am J Med Qual. 2008;23(3):193–200.
Article
Google Scholar
Malone S, Prewitt K, Hackett R, Lin JC, McKay V, Walsh-Bailey C, et al. The clinical sustainability assessment tool: measuring organizational capacity to promote sustainability in healthcare. Implement Sci Commun. 2021;2(1):77.
Article
Google Scholar
Williams RM, Zhang J, Woodard N, Slade JL, Santos LZ, Knott CL. Development and validation of an instrument to assess institutionalization of health promotion in faith-based organizations. Eval Program Plann. 2020;79:101781.
Article
Google Scholar
Bond GR, Drake RE, Rapp CA, GJ MH, Xie H. Indivdualization and quality improvement: two new scales to complement measurement of program fidelity. Admin Pol Ment Health. 2009;36(5):349–57.
Article
Google Scholar
Heiervang KS, Egeland KM, Landers M, Ruud T, Joa I, Drake RE, et al. Psychometric properties of the General Organizational Index (GOI): a measure of indivualization and quality improvement to comlement program fidelity. Admin Pol Ment Health. 2020;47:920–6.
Article
Google Scholar
Barab SA, Redman BK, Froman RD. Measurement characteristics of the Levels of Institutionalization Scales: examining reliability and validity. J Nurs Meas. 1998;6(1):19–33.
Article
CAS
Google Scholar
Goodman RM, McLeroy KR, Steckler AB, Hoyle RH. Development of level of institutionalization scales for health promotion programs. Health Educ Q. 1993;20(2):161–78.
Article
CAS
Google Scholar
Goodman RM, Steckler A. A framework for assessing program institutionalization. Knowl Soc. 1989;2(1):57–71.
Google Scholar
Maher L, Gustafson DH, Evans A. Sustainability model and guide; 2010.
Google Scholar
Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):135.
Article
Google Scholar
Rapley T, Girling M, Mair FS, Murray E, Treweek S, McColl E, et al. Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):133.
Article
Google Scholar
Vis C, Ruwaard J, Finch T, Rapley T, de Beurs D, van Stel H, et al. Toward an objective assessment of implementation processes for innovations in health care: Psychometric evaluation of the Normalization Measure Development (NoMAD) Questionnaire among mental health care professionals. J Med Internet Res. 2019;21(2):e12376.
Article
Google Scholar
Davis S. Ready for prime time? Using normalization process theory to evaluate implementation success of personal health records designed for decision making. Front Digit Health. 2020;2:575951.
Article
Google Scholar
Loch AP, Finch T, Fonsi M, Soarez PC. Cross-cultural adaptation of the NoMAD questionnaire to Brazilian Portuguese. Rev Assoc Med Bras (1992). 2020;66(10):1383–90.
Article
Google Scholar
Elf M, Nordmark S, Lyhagen J, Lindberg I, Finch T, Aberg AC. The Swedish version of the normalization process theory measure S-NoMAD: translation, adaptation, and pilot testing. Implement Sci. 2018;13(146). https://doi.org/10.1186/s13012-018-0835-5.
May CR, Finch T, Ballini L, MacFarlane A, Mair F, Murray E, et al. Evaluating complex interventions and health technologies using normalization process theory: development of a simplified approach and web-enabled toolkit. BMC Health Serv Res. 2011;11(1):1–11.
Article
Google Scholar
Hawe P, King L, Noort M, Jordens C, Lloyd B. Indicators to help with capacity building in health promotion; 2000.
Google Scholar
Office of Adolescent Health. Building sustainable programs: the resource guide. 2014.
Google Scholar
Office of Population Affairs. Resource guide for building sustainable programs. 2019.
Google Scholar
Stamatakis KA, McQueen A, Filler C, Boland E, Dreisinger M, Brownson RC, et al. Measurement properties of a novel survey to assess stages of organizational readiness for evidence-based interventions in community chronic disease prevention settings. Implement Sci. 2012;7:65.
Article
Google Scholar
Hall A, Shoesmith A, Shelton RC, Lane C, Wolfenden L, Nathan N. Adaptation and Validation of the Program Sustainability Assessment Tool (PSAT) for use in the elementary school setting. Int J Environ Res Public Health. 2021;18(21):11414.
Article
Google Scholar
Mancini JA, Marek LI. Sustaining community-based programs for families: conceptualisation and measurement. Fam Relat. 2004;53(4):339–47.
Article
Google Scholar
McIntosh K, MacKay LD, Hume AE, Doolittle J, Vincent CG, Horner RH, et al. Development and initial validation of a measure to assess factors related to sustainability of school-wide positive behavior support. J Posit Behav Interv. 2010;13(4):208–18.
Article
Google Scholar
Hume A, McIntosh K. Construct validation of a measure to assess sustainability of school-wide behavior interventions. Psychol Sch. 2013;50(10):1003–14.
Article
Google Scholar
Kittelman A, Bromley KW, Mercer SH, McIntosh K. Validation of a measure of sustainability of school-wide behavior interventions and supports. Remedial Spec Educ. 2019;40(2):67–73.
Article
Google Scholar
McIntosh K, Mercer SH, Hume AE, Frank JL, Turri MG, Mathews S. Factors related to sustained implementation of schoolwide positive behavior support. Except Child. 2013;79(3):293–311.
Google Scholar
Mercer SH, McIntosh K, Strickland-Cohen MK, Horner RH. Measurement invariance of an instrument assessing sustainability of school-based universal behavior practices. Sch Psychol Q. 2014;29(2):125.
Article
Google Scholar
The Board of Regents of the University System of Georgia by and on behalf of Georgia State University and the Georgia Health Policy Center. Positioning for sustainability: a formative assessment tool – quick course. 2011.
Google Scholar
Markstrom U, Svensson B, Bergmark M, Hansson L, Bejerholm U. What influences a sustainable implementation of evidence-based interventions in community mental health services? Development and pilot testing of a tool for mapping corse components. J Ment Health. 2018;27(5):395–401.
Article
Google Scholar
Hodge LM, Turner KMT, Sanders MR, Filus A. Sustained Implementation Support Scale: validation of a measure of program characteristics and workplace functioning for sustained program implementation. J Behav Health Serv Res. 2017;44(3):442–64.
Article
Google Scholar
Askell-Williams H, Koh GA. Enhancing the sustainability of school improvement initiatives. Sch Eff Sch Improv. 2020;31(4):660–78.
Article
Google Scholar
Ehrhart MG, Torres EM, Green AE, Trott E, Willging CE, Moullin JC, et al. Leading for the long haul: a mixed-method evaluation of the Sustainment Leadership Scale (SLS). Implement Sci. 2018;13(17). https://doi.org/10.1186/s13012-018-0710-4.
Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Admin Pol Ment Health. 2011;38(1):4–23.
Article
Google Scholar
Palinkas LA, Chou CP, Spear SE, Mendon SJ, Villamar J, Brown CH. Measurement of sustainment of prevention programs and initiatives: the sustainment measurement system scale. Implement Sci. 2020;15(1):71.
Article
Google Scholar
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15.
Article
Google Scholar
Harris P, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. The REDCap consortium: building an international community of software partners. J Biomed Inform. 2019;95(2019):103208.
Harris P, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap) – a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.
Article
Google Scholar
Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci. 2013;8(1):1–20.
Article
Google Scholar
Slaghuis SS, Strating MMH, Bal RA, Nieboer AP. A framework and measurement instrument for sustainability of work practices in long-term care. BMC Health Serv Res. 2011;11(314). https://doi.org/10.1186/1472-6963-11-314.
Finnerty MT, Rapp CA, Bond GR, Lynde DW, Ganju V, Goldman HH. The State health authority yardstick (SHAY). Community Ment Health J. 2009;45:228–36.
Article
Google Scholar
Saldana L. The stages of implementation completion for evidence-based practice: protocol for a mixed methods study. Implement Sci. 2014;9(1):43.
Article
Google Scholar
Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25(24):3186–91.
Article
CAS
Google Scholar
Saldana L, editor Operationalizing sustainment activities for two evidence-based practices using the stages of implementation completion (SIC). 11 th Annual Conference on the Science of Dissemination and Implementation; 2018: AcademyHealth.
Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. 2020;8:134.
Article
Google Scholar
Measures and Checklists. 2019. Available from: http://www.re-aim.org/resources-and-tools/measures-and-checklists/.
Lennox L, Doyle C, Reed J, Bell D. What makes a sustainability tool valuable, practical and useful in realworld healthcare practice? A mixed methods study on the development of the long term success tool in Northwest London. BMJ Open. 2017;7:e014417.
Article
Google Scholar