A non-randomized controlled stepped wedge trial to evaluate the effectiveness of a multi-level mammography intervention in improving appointment adherence in underserved women
© Highfield et al. 2015
Received: 29 September 2015
Accepted: 6 October 2015
Published: 14 October 2015
Considerable racial and socio-economic disparities exist in breast cancer. In spite of the existence of numerous evidence-based interventions (EBIs) aimed at reducing breast cancer screening barriers among the underserved, there is a lack of uptake or sub-optimal uptake of EBIs in community and clinical settings. This study evaluates a theoretically based, systematically designed implementation strategy to support adoption and implementation of a patient navigation-based intervention, called Peace of Mind Program (PMP), aimed at improving breast cancer screening among underserved women.
The PMP will be offered to federally qualified health centers and charity clinics in the Greater Houston area using a non-randomized stepped wedge design. Due to practical constraints of implementing and adopting in the real-world, randomization of start times and blinding will not be used. Any potential confounding or bias will be controlled in the analysis. Outcomes such as appointment adherence, patient referral to diagnostics, time to diagnostic referral, patient referral to treatment, time to treatment referral, and budget impact of the intervention will be assessed. Assessment of constructs from the consolidated framework for implementation research (CFIR) will be assessed during implementation and at the end of the study (sustainment) from each participating clinic. Data will be analyzed using descriptive statistics (chi-square tests) and generalized estimating equations (GEE).
While parallel group randomized controlled trials (RCT) are considered the gold standard for evaluating EBI efficacy, withholding an effective EBI in practice can be both unethical and/or impractical. The stepped wedge design addresses this issue by enabling all clinics to eventually receive the EBI during the study and allowing each clinic to serve as its own control, while maintaining strong internal validity. We expect that the PMP will prove to be a feasible and successful strategy for reducing appointment no-shows in underserved women.
Clinical trials registration number: NCT02296177
Importance of mammography adherence in underserved women
Breast cancer is the most common cancer and the second leading cause of cancer mortality among women in the USA , with roughly one in eight women developing breast cancer during their lifetime [2, 3]. Substantial breast cancer-related morbidity and mortality disparities persist among the underserved, particularly among African American women [1–6]. Disparities in breast cancer outcomes are due to lower mammography screening rates, lack of timely follow-up of abnormal results, and lack of timely treatment initiation among women with breast cancer. Stage at diagnosis is an important predictor of survival [7, 8]. Women who are diagnosed at a late stage are at higher risk for mortality from breast cancer, and are at risk for complications and overall poorer health outcomes [7, 8]. Underserved populations have consistently been shown to be at higher risk for late-stage diagnosis due to a combination of lower screening rates, missed appointments, and lack of timely referral to diagnostic evaluation and treatment [7–11]. Additionally, research has found that past mammography screening behavior is strongly correlated with current screening behavior, meaning that women who are off-schedule with screening are less likely to become adherent . Appointment no-shows as they relate to the eventual completion of mammography screening have also been recently studied. Ontilo et al., 2013, found that women who missed a screening were more likely to be diagnosed at a later stage of cancer than women who attended . In a 2012 study by Fayanju et al. of federally qualified health center (FQHC) patients in St. Louis, women who missed a screening appointment were significantly more likely to never receive definitive diagnosis or needed follow-up care . A number of barriers prevent underserved women from obtaining timely screening, including patient-level socioeconomic differences like education, insurance, and income; differences in skills and qualifications of providers attending to different racial-ethnic groups; inadequacies in health care systems which provide care for the underserved; beliefs and misconceptions; and linguistic/communication barriers [8, 9, 14, 15]. A number of evidence-based intervention programs have been developed to address these barriers, yet disparities in breast cancer outcomes persist for underserved women. One reason for this may be the lack of uptake and use of evidence-based interventions in community and clinical settings that serve these women [16–20].
The need for adoption and implementation support in clinics serving underserved populations
Dissemination and implementation processes for the uptake of evidence-based interventions (EBIs) remain poorly understood. The expertise of practitioners is often not used effectively, and implemented programs often do not ultimately reflect the needs of underserved communities within the specific context [17, 21, 22]. Particularly for interventions with minority communities, EBIs should be adopted and tailored at the community level by partnerships that include both researchers and practitioners, adding an additional level of complexity to the translation of these EBIs [23, 24]. Efforts to increase adoption and use of EBIs in community and clinical settings serving underserved women should target the safety net health care delivery system . Within the safety net, primarily FQHCs and charity clinics provide comprehensive primary health care services for underserved communities regardless of ability to pay; this includes mammography screening [25, 26]. Additionally, the passage of the Affordable Care Act has significantly expanded the role of FQHCs who will be increasingly important in providing care for populations with a disproportionate burden of need . Due to their client base, FQHCs are uniquely situated to reach and serve women in the most need for breast cancer screening ; however, FQHCs need assistance in planning and implementing EBIs [27, 28]. There is currently a lack of theoretically informed, well-documented, and evaluated implementation strategies for the translation of EBIs particularly within safety net clinical settings [4, 5]. Systematically designed implementation strategies are particularly important for the implementation of complex interventions—meaning interventions consisting of multiple behavioral, technological, and organizational components .
This study aims to address the research to practice gap and uses a theoretically based, strongly designed D&I strategy to support implementation of a breast cancer screening intervention aimed at promoting mammography adherence in underserved populations. Our patient navigation-based breast cancer screening intervention for the underserved is called the Peace of Mind Program (PMP). PMP is an EBI which was adapted from a National Cancer Institute Research tested intervention program and uses a tailored telephone counseling reminder call based on the Transtheoretical Model of Change to counsel patients through barriers to appointment attendance [20, 29–31]. PMP builds on our previous intervention protocol [30, 31] by adding intervention components to specifically support implementation within safety net clinics.
We hypothesize that the PMP intervention will improve mammography appointment adherence as compared to usual practice.
To assess implementation of the intervention at the clinics.
To test the effectiveness of the PMP intervention in improving appointment adherence for screening mammograms.
To test the effectiveness of the PMP intervention in reducing the time it takes for a patient with abnormal results to receive follow-up and navigation to a diagnostic appointment and subsequent treatment initiation.
To assess the budget impact (cost savings) of the implementation of the PMP intervention from the perspective of the clinics.
Participants and settings
The PMP trial will be conducted in FQHCs and charity clinics who are members of the Breast Health Collaborative of Texas (BHCT) network. BHCT is a state-wide network of over 700 clinics and individuals with a shared mission to improve access to breast health services for all women in Texas.
Practice inclusion criteria
In order to participate, clinics must (1) be members of BHCT within the Houston service area, (2) have a designation as a FQHC by HRSA or be a charity clinic which provides free or reduced cost care to underserved populations in their service area, (3) be serving women between the ages of 40–64 who are at or below 200 % of the federal poverty level for a family of four and who lack health insurance, and (4) engage in provision of mammography screening services at least six times per year and (5) women at the clinic must be in need of mammography screening and be scheduled for an upcoming appointment. Patients must have completed a clinical breast exam prior to their screening appointment.
Practice exclusion criteria
There are no practice exclusion criteria if serving patients within the Houston service area.
The PMP intervention program consists of the following components: a financial incentive of $7500 to the clinic for participation; funding for screening mammograms for women with documented financial need; coordination of increased screening within each clinic (the research team has established partnerships with local mobile mammography providers to assist in increasing screening capacity); 8 h of state certified training for staff in the EBI’s reminder phone call protocol [20, 30, 31]; stakeholder review of program adaptation and implementation materials; stakeholder meetings to troubleshoot implementation issues as they arise and to develop and disseminate e-newsletters and reports which highlight success stories and re-inforce desired behaviors; technical support for implementation (implementation manuals, clinic manuals, stakeholder manuals); and assistance in data tracking and reporting and budget impact analysis.
Estimated sample sizes based on ICC reported values
Randomization and allocation
Clinic will be the unit of allocation in this study. Each clinic will be assigned a start date in groups of three (Fig. 1). Randomization of clinic start dates will not be done for practical reasons, including constraints in clinic operations (requirements for when they can or cannot start) and available screening dates (some clinics need additional time to reach the screening minimums required for study participation (n = 6)). The research team will assign a start date to each clinic based on these constraints.
Due to the nature of the intervention, it is not possible to blind the clinics or the research team in this study. However, outcome assessment is objective as we are using medical records to obtain screening outcomes. It is also not possible to blind the data analyst because the analysis is designed as a time series in which every clinic receives the intervention rather than a comparison between groups. The analysis will be pre-specified rather than blinded, and any analysis not pre-specified will be clearly delineated and considered hypothesis generating/exploratory rather than hypothesis testing/explanatory.
The primary outcome measure will be appointment adherence comparing the intervention period to the baseline period in each clinic. Appointment adherence will be considered using the following categories: no-show, not attended-called, late-screened, late-turned away, canceled/re-scheduled, and ineligible. A no-show will be considered a patient who makes no contact with the clinic prior to the appointment and does not attend on the day of mammography screening. A not attended-called is a patient who calls the clinic the day of the screening and asks to cancel or re-schedule (this may occur before or after their designated appointment time). Late-screened is a patient who arrives more than 30-min late for their appointment but is still seen by the mammography provided (e.g., worked in). Late-turned away is a patient who is more than 30-min late for their appointment and is turned away by the provider. A cancel or re-schedule is a patient who calls more than 24 h in advance to cancel or change their appointment date. An ineligible patient is a woman with symptoms who is ineligible to receive a screening mammogram and is turned away for a diagnostic evaluation or a woman who is turned away for incomplete paperwork. These categories are based on an assessment of the mammography providers’ and clinics’ behavior and the ways in which patient appointment outcomes are considered within a clinic system. Secondary measures will include patient referral to diagnostic (following screening) as a binary outcome and amount of time the referral took in days, patient referral to treatment as a binary outcome and the amount of time it took in days, and measures of implementation in each clinic based on the consolidated framework for implementation research (CFIR). A survey made up of scales assessing various CFIR constructs was developed by the Cancer Prevention and Control Research Network (CPCRN); we will use several scales from that survey in this study. These constructs will be measured at baseline (adoption), 8-week post-implementation, and following the end of the trial period (sustainment) [35–37]. Because randomization of start dates is not possible, we will control for potential confounders in the analysis of the data. Control variables may include time to intervention start date in each clinic, clinic size, staffing, and whether or not the clinic had a pre-existing patient navigation program for breast cancer screening, patient demographics, region served, and infrastructure characteristics (e.g., funding) (taken from the Health Resources and Services Administration Uniform Data System (UDS) reports for FQHCs and our economic survey).
Data collection protocols will be standardized across all clinics. In clinics with the capability to collect and report data from their electronic health record (EHR), the research team will seek to adapt the existing system for data collection. In clinics where the EHR is not easily modified or one is not available, data collection will be supported via the use of a customized Redcap database. The database will have a front-facing data entry interface which is user friendly and supports accurate data collection for statistical analysis. The patient navigator (or implementer) in each clinic will be responsible for tracking upcoming patient mammography appointments, reminder phone call attempts, patient consent for the trial, and the EBI screening questions and barriers counseled during the phone call. Appointment attendance, referral to diagnostic mammography, and treatment data will be abstracted from medical records in each clinic. For the budget impact analysis, two surveys will be used to assess the FQHC/clinic resource utilization before and after the implementation of the intervention. We are also collecting patient resource utilization data to understand what parts of patient costs are being subsidized by the FQHCs/clinics and to also ensure availability of cost information from patient perspective in case an economic evaluation from societal perspective is deemed important.
Descriptive statistics will be calculated for all primary and secondary measures. Following descriptive analysis, regression modeling will be used to evaluate outcomes while allowing for control of potential confounders (e.g., time, clinic size, staffing, and whether or not the clinic had a pre-existing patient navigation program for breast cancer screening). Due to the correlated nature of the data and repeated observations taken on each clinic, generalized estimating equations (GEE) will be used to model outcomes while allowing us to control for potential confounders [38, 39]. GEE is used to estimate the parameters from a generalized linear model when there is possible correlation between outcomes [38, 39]. GEE is a population averaged model that estimates the average response from a change in a covariate over the population rather than individual [38, 39]. GEE model estimates are less sensitive than generalized linear mixed models to the specification of the variance structure in the data making them a flexible choice [38, 39]. Data on appointment adherence will be analyzed using various sub-groups to evaluate the intervention effect. For example, no-shows and not attended-called may be combined to consider a more stringent definition of a no-show. They will also be analyzed separately. We expect the intervention to have an effect on the no-shows, not attended-called, late-screened, late-turned away and ineligible categories and will evaluate that effect separately. Sub-categories will be compared using chi-square tests to determine differences in outcomes between groups (control and intervention). Data from the CFIR survey will be analyzed using descriptive statistics and in the regression models to evaluate which constructs are related to implementation outcomes. Data analysis will be conducted in Stata (Stata Corp., College Station, TX).
Budget impact analysis
The pre-intervention/control period, which will give us a distribution of cost for the screening reminder or navigation systems currently used by the clinics. We expect to find a wide variety of patient navigation or screening reminder systems during this period. Many clinics might have no mechanism in place for navigation or screening reminders for mammogram eligible patients.
The intervention period, which will help estimate the initial investment and implementation cost associated with the EBI. Cost associated with this period will be the highest among the three periods and will provide an estimate of the upfront expenses for any new clinic establishing an EBI program. It will consists of the fixed cost of training the navigators on the EBI methods, cost associated with structural changes to the clinic’s EMR, which will help link the patient records with the appropriate EBI scripts, and cost of developing web-based scripts for patient navigation. Based on the volume of patients served, the intervention period cost will also include the variable cost of hiring additional navigators for implementing the EBI. The navigators will not only help improve the provision of mammograms but also facilitate diagnostic test administration and treatment initiation for patients who need these. The patient volume and amount of time required per patient will be obtained during the implementation period from each of the clinics. The initial time required to navigate a patient and improve adherence to the mammogram screen will be similar between patients. However, if the patient has a positive mammogram, the patient will require further navigation to obtain a diagnostic test (biopsy). Patients with positive diagnostic tests will also require navigation for treatment initiation. Although the study’s primary goal is to improve screening adherence and reduce screening “no-show” rates, the navigators are also involved with diagnostic and treatment navigation when required by the patient. Hence, cost of those tasks will also be included in the analysis.
The post-intervention period, which will help estimate the steady state cost for each clinic with no external support for the EBI. Post-intervention provision of EBI and associated costs will be obtained 3 months after the end of the study.
The trial has completed enrollment of the first wave of clinics (n = 9) and is currently collecting baseline measures at these locations. Baseline measures include appointment adherence outcomes as described above, the clinic and patient resource utilization survey for the budget impact analysis, and CFIR survey. All participating clinics have completed the baseline CFIR survey. Based on the number of mammography screening drives at each clinic, we anticipate enrolling approximately 2200 women into the first wave of the trial.
Rationale for the stepped wedge design
While parallel group randomized controlled trials (RCT) are considered the gold standard for evaluating EBI efficacy, withholding an effective EBI in practice can be seen as unethical and/or impractical [42–45] . The stepped wedge design addresses this issue by enabling all clinics to eventually receive the EBI during the study and allowing each clinic to serve as its own control, while maintaining strong internal validity [42–45]. Stepped wedge designs are particularly useful when the intervention is thought to do more good than harm and when an RCT is impractical, as is the case in this study (e.g., financial, logistical reasons) [45–47].
Despite the advantages to using a stepped wedge design for the PMP trial, we anticipate a number of challenges. The staggered entry into the intervention period means that clinics assigned to a later start date will start the intervention 48 weeks after their initial agreement to adopt the program. This may mean that the motivations for initially agreeing to adopt the program may no longer apply. We also anticipate that clinics may be frustrated by having to wait to start the program even if they stay motivated to participate. With that in mind, we defined a strategy to maintain clinic interest and engagement while they are waiting to start PMP. Each clinic will participate in a stakeholder committee where they will review implementation plans, handbooks, and undergo intervention training. The schedule for the stakeholder committee is staggered such that the later groups spend more time working in this venue in order to keep them engaged. A further potential problem in stepped wedge designs is contamination between clinics and those waiting for the intervention. It is possible that some clinics may change their procedures based on what they hear or see with other participating clinics. However, we expect such effects to be mild because meaningful and sustained improvements in mammography appointment adherence require systematic and prolonged effort that is unlikely to occur before the PMP training is completed.
This study has received approval from the Institutional Review Board at the University of Texas Health Science Center Houston. The protocol number is HSC-SPH-14-0269. Verbal consent will be obtained for patients participating in the EBI reminder phone calls. Written consent will be obtained for economic data collection from patients.
The authors would like to acknowledge the Cancer Prevention and Control Research Network for sharing their survey with us. We would also like to acknowledge our community partner, the Breast Health Collaborative of Texas for their role in the study.
Funding for this study is provided by the Agency for Healthcare Research and Quality grant number 1R18HS023255-01.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Bradley CJ, Given CW, Roberts C. Disparities in cancer diagnosis and survival. Cancer. 2001;91:178–88.PubMedGoogle Scholar
- Gordon NH, Crowe JP, Brumberg DJ, Berger NA. Socioeconomic factors and race in breast cancer recurrence and survival. Am J Epidemiol. 1992;135:609–18.PubMedGoogle Scholar
- Byers TE, Wolf HJ, Bauer KR, Bolick-Aldrich S, Chen VW, Finch JL, et al. The impact of socioeconomic status on survival after cancer in the United States: findings from the National Program of Cancer Registries Patterns of Care Study. Cancer. 2008;113:582–91.PubMedGoogle Scholar
- Farley TA, Flannery JT. Late-stage diagnosis of breast cancer in women of lower socioeconomic status: public health implications. Am J Public Health. 1989;79:1508–12.PubMedPubMed CentralGoogle Scholar
- Ayanian JZ, Kohler BA, Abe T, Epstein AM. The relation between health insurance coverage and clinical outcomes among women with breast cancer. N Engl J Med. 1993;329:326–31.PubMedGoogle Scholar
- Bradley CJ, Given CW, Roberts C. Re: race, socioeconomic status, and breast cancer treatment and survival. J Natl Cancer Inst. 2002;94:490–6.PubMedGoogle Scholar
- Schueler KM, Chu PW, Smith-Bindman R. Factors associated with mammography utilization: a systematic quantitative review of the literature. J Women’s Health. 2008;17(9):1477–98. doi:10.1089/jwh.2007.0603.Google Scholar
- Onitilo AA, Engel JM, Liang H, Stankowski RV, Miskowiak DA, et al. Mammography utilization: patient characteristics and breast cancer stage at diagnosis. Am J Roentgenol. 2013;201(5):1057–63.Google Scholar
- Fayanju OM, Jeffe DB, Tappenden JR, Beers CE, Drake BF, Gao F, et al. Breast cancer presentation in an urban health care safety net system. Mo Med. 2012;109(5):405.PubMedPubMed CentralGoogle Scholar
- Wang F, McLafferty S, Escamilla V, Luo L. Late-stage breast cancer diagnosis and health care access in illinois. Prof Geogr. 2008;60(1):54–69.PubMedPubMed CentralGoogle Scholar
- Rajan SS, Begley CE, Highfield LD, Kim B. Survival benefits of treatment access among underserved breast cancer patients diagnosed through the Texas breast and cervical cancer services program. J Public Health Manag Pract. 2015;21(5):477–86.PubMedGoogle Scholar
- Rajan SS, Begley CE, Kim B. Breast cancer stage at diagnosis among medically underserved women screened through the Texas breast and cervical cancer services. Popul Health Manag. 2014;17(4):202–10.PubMedGoogle Scholar
- Spitler HD, Mayo RM, Parker VG. Patterns of breast, cervical, colorectal, and prostate cancer in the Appalachian region of South Carolina. Ethnicity Disease. 2000;11(1):51–9.Google Scholar
- Davidson PL, Bastani R, Nakazono TT, Carreon DC. Role of community risk factors and resources on breast carcinoma stage at diagnosis. Cancer. 2005;103:922–30.PubMedGoogle Scholar
- Baicker K, Chandra A, Skinner JS, Wennberg JE. Who you are and where you live: how race and geography affect the treatment of medicare beneficiaries. Health Aff (Milwood). 2004;suppl:VAR33–44.
- Harris JR, Cheadle A, Hannon PA, Forehand M, Lichiello P, Mahoney E, et al. A framework for disseminating evidence-based health promotion practices. Prev Chronic Dis. 2012;9:E22.PubMedGoogle Scholar
- Kerner J, Rimer B, Emmons K. Introduction to the special section on dissemination: dissemination research and research dissemination: how can we close the gap? Health Psychol. 2005;24(5):443–6.PubMedGoogle Scholar
- Rogers EM. Diffusion of innovations. New York, NY, USA: Free Press; 2003.Google Scholar
- Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93(8):1261–7.PubMedPubMed CentralGoogle Scholar
- Highfield L, Bartholomew LK, Hartman MA, Ford MM, Balihe P. Grounding evidence-based approaches to cancer prevention in the community: a case study of mammography barriers in underserved African American women. Health Promot Pract. 2014;15(6):904–14.PubMedPubMed CentralGoogle Scholar
- Yano EM, Green LW, Glanz K, Ayanian JZ, Mittman BS, Chollette V, et al. Implementation and spread of interventions into the multilevel context of routine practice and policy: implications for the cancer care continuum. J Natl Cancer Inst Monogr. 2012;2012(44):86–99.PubMedPubMed CentralGoogle Scholar
- Stetler CB, McQueen L, Demakis J, Mittman BS. An organizational framework and strategic implementation for system-level change to enhance research-based practice: QUERI series. Implement Sci. 2008;3(1):1–11.Google Scholar
- Ahmed SM, Palermo AS. Community engagement in research: frameworks for education and peer review. Am J Public Health. 2010;100(8):1380–7.PubMedPubMed CentralGoogle Scholar
- Taplin SH, Price RA, Edwards HM, Foster MK, Breslau ES, Chollette V, et al. Introduction: understanding and influencing multilevel factors across the cancer care continuum. J Natl Cancer Inst Monogr. 2012;2012(44):2–10.PubMedPubMed CentralGoogle Scholar
- Allen CL, Harris JR, Hannon PA, Parrish AT, Hammerback K, Craft J, et al. Opportunities for improving cancer prevention at federally qualified health centers. J Cancer Educ. 2014;29(1):30–7.PubMedGoogle Scholar
- Shin P, Jones K, Rosenbaum S. Reducing racial and ethnic health disparities: estimating the impact of high health center penetration in low-income communities Available at: http://hsrc.himmelfarb.gwu.edu/cgi/viewcontent.cgi?article=1299&context=sphhs_policy_facpubs Accessed: September 20, 2013.
- Cialdella-Kam L, Parichart SM, Bispeck K, Silverman S, Bernstein L, Krawiec V, et al. Implementing cancer prevention into clinical practice. J Cancer Educ. 2012;27(2):136–43.Google Scholar
- Taplin SH, Haggstrom D, Jacobs T, Determan A, Granger J, Montalvo W, et al. Implementing colorectal cancer screening in community health centers: addressing cancer health disparities through a regional cancer collaborative. Med Care. 2008;46:S74–83.PubMedGoogle Scholar
- Lipkus IM, Kuchibhatla M, McBride CM, Bosworth HB, Pollak KI, Siegler IC, et al. Relationships among breast cancer perceived absolute risk, comparative risk, and worries. Cancer Epidemiol Biomark Prev. 2000;9(9):973–5.Google Scholar
- Highfield L, Hartman, MA, Mullen PD, Rodriquez S. Fernandez ME, Bartholomew LK. Intervention mapping to adapt evidence-based interventions for use in practice: increasing mammography among African American women in a practice setting. Biomed Res Int. 2015; Article ID 160103 in pressGoogle Scholar
- Highfield L, Hartman MA, Bartholomew LK, Balihe P, Ausborn VA. Evaluation of the effectiveness and implementation of an adapted evidence-based mammography intervention for African American women. Biomed Res Int. 2015; Article ID 240240, in pressGoogle Scholar
- Woertman W, de Hoop E, Moerbeek M, Zuidema SU, Gerritsen DL, Teerenstra S. Stepped wedge designs could reduce the required sample size in cluster randomized trials. J Clin Epidemiol. 2013;66(7):752–8.PubMedGoogle Scholar
- Hade EM, Murray DM, Pennell ML, Rhoda D, Paskett ED, Champion VL, et al. Intraclass correlation estimates for cancer screening outcomes: estimates and applications in the design of group-randomized cancer screening studies. JNCI Monographs. 2010;40:97–103.Google Scholar
- Wells KJ, Luque JS, Miladinovic B, Vargas N, Asvat Y, Roetzheim RG, et al. Do community health worker interventions improve rates of screening mammography in the United States? A systematic review. Cancer Epidemiol Biomark Prev. 2011;20(8):1580–98.Google Scholar
- Fernández ME, Melvin CL, Leeman J, Ribisl KM, Allen JD, Kegler MC, et al. The cancer prevention and control research network: an interactive systems approach to advancing cancer control implementation research and practice. Cancer Epidemiol Biomark Prev. 2014;23(11):2512–21.Google Scholar
- Liang S, Kegler M, Carvalho M, Fernandez M, Weiner B, Jacobs S, et al. Measuring constructs from the consolidated framework for implementation research in the context of increasing colorectal cancer screening at community health centers. Implement Sci. 2015;10 Suppl 1:A10.PubMed CentralGoogle Scholar
- Fernandez ME, Shuting L, Jacobs SR, Taplin SH, Weiner BJ. Pressing ahead: developing and testing of new measures in implementation science. Implement Sci. 2015;10 Suppl 1:A14.PubMed CentralGoogle Scholar
- Liang KY, Zeger S. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73(1):13–22.Google Scholar
- Hardin J, Hilbe J. Generalized estimating equations. London: Chapman and Hall/CRC; 2003.Google Scholar
- Mauskopf JA, Earnshaw S, Mullins CD. Budget impact analysis: review of the state of the art. Expert Rev Pharmacoecon Outcomes Res. 2005;5(1):65–79.PubMedGoogle Scholar
- Mauskopf JA, Sullivan SD, Annemans L, Caro J, Mullins CD, Nuijten M, et al. Principles of good practice for budget impact analysis: report of the ISPOR task force on good research practices—budget impact analysis. Value Health. 2007;10(5):336–47.PubMedGoogle Scholar
- Hawkins NG, Sanson-Fisher RW, Shakeshaft A, D’Este C, Green LW. The multiple baseline design for evaluating population-based research. Am J Prev Med. 2007;33(2):162–8.PubMedGoogle Scholar
- De Allegri M, Pokhrel S, Becher H, Dong H, Mansmann U, Kouyaté B, et al. Step-wedge cluster-randomised community-based trials: an application to the study of the impact of community health insurance. Health Res Policy Syst. 2008;6(10):6–10.Google Scholar
- Rhoda DA, Murray DM, Andridge RR, Pennell ML, Hade EM. Studies with staggered starts: multiple baseline designs and group-randomized trials. Am J Public Health. 2011;101(11):2164.PubMedPubMed CentralGoogle Scholar
- Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28:182e91.Google Scholar
- Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54.PubMedPubMed CentralGoogle Scholar
- Mdege ND, Man MS, Taylor Nee Brown CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64:936e48.Google Scholar