Skip to main content
Download PDF
Download ePub

Evaluating the impact of multilevel evidence-based implementation strategies to enhance provider recommendation on human papillomavirus vaccination rates among an empaneled primary care patient population: a study protocol for a stepped-wedge cluster randomized trial

Implementation Science volume 13, Article number: 96 (2018) Cite this article

Abstract

Background

Each year, human papillomavirus (HPV) causes 30,000 cancers in the USA despite the availability of effective and safe vaccines. Uptake of HPV vaccine has been low and lags behind other adolescent vaccines. This protocol describes a multilevel intervention to improve HPV vaccination rates.

Methods

Using a cluster randomized trial, we will evaluate the independent and combined impact of two evidence-based implementation strategies with innovative enhancements on HPV vaccination rates for female and male patients. The clusters are six primary care sites providing care to pediatric populations. We will use a stepped-wedge cluster randomized design, including process evaluation, to test the hypothesis that compared with the current course of care and a practice-level intervention using reminder-recall interventions coupled with provider-level audit and feedback with education increases HPV vaccination rates in exposed clusters. The factorial design allows us to use a single trial to test these two interventions and to assess each individually and in combination. Our design has four 12-month steps. The first step will be a baseline period; data collected during it will provide a within-practice control group for each cluster. Second, two clusters will be randomly assigned to receive intervention 1 (reminder and recall), and two clusters will be randomly selected to receive intervention 2 (audit and feedback with education). Third, the other two clusters will be randomly allocated to intervention 1 or 2. Clusters initially with intervention 1 will be randomly allocated to 1 + 2 or 1; clusters initially with intervention 2 will be randomly allocated to 1 + 2 or 2. Fourth, all clusters will receive both interventions. To ensure balance of patient numbers across interventions, we will use block randomization at the first step, with the six clusters grouped into three pairs according to volume. Our primary outcome will be vaccination rates.

Discussion

Results of our clinical trial and process evaluation will provide evidence showing whether practice- and provider-level interventions improve HPV vaccination rates and will offer insight into contextual factors associated with direction and magnitude of trial outcomes.

Trial registration

ClinicalTrials.gov, NCT03501992, registered April 18, 2018.

Peer Review reports

Background

From 2008 through 2012, on average, 38,793 new cases of human papillomavirus (HPV)-associated cancers occurred each year in the USA [1]. This represents an increase in the number of cases from 2004 through 2008, when the number calculated was 33,369 new cases [2]. The observed increase in HPV-associated cancers has occurred despite the recommendation for universal vaccination against HPV [3, 4].

Recommendations call for universal vaccination of children 11 to 12 years of age with the two-dose series (or three-dose series if immunocompromised) over 6 to 12 months, with universal catch-up for girls and women age 13 through 26 years and boys and men age 13 through 21 years [5]. However, nationally, HPV vaccine delivery rates continue to be low, have lagged in contrast to other vaccines for adolescents [6], and fail to approach Healthy People 2020 targets, which are 80% for girls and boys age 13 to 15 years [7]. HPV vaccination rates in our local population are similarly low [8]. Efforts to improve vaccination rates are urgently needed [9].

Study objectives

We propose to address this critical need through development, implementation, and evaluation of a multilevel intervention, leveraging evidence-based implementation strategies to increase HPV vaccination rates in an empaneled primary care patient population. Multilevel interventions offer not just the combination of incremental effects of individual interventions adopted simultaneously but also synergistic effects [10, 11]. Using a stepped-wedge cluster randomized trial, we will assess the independent and synergistic effect of a multilevel intervention on HPV vaccination rates. We also will conduct a process evaluation to assess the fidelity and quality of intervention implementation and evaluate potential factors that may be associated with observed outcomes.

Multilevel intervention framework

The Social Ecological Model (SEM) is a theoretical framework for understanding multiple and interactive personal and environmental factors that influence behavior [12,13,14]. Applying SEM to the clinical setting, we will implement two evidence-based interventions, each with novel components, to influence organizational, interpersonal, and individual levels of behavior.

Obstacles to HPV vaccination and strategies for improvement

A rich body of research has identified obstacles at the organizational, interpersonal, and individual levels that have impeded the USA from achieving national goals with HPV vaccination [15, 16].

Organizational barriers

Clinical visits among adolescents are infrequent. Yet, even when such visits occur, patients eligible for a dose of HPV vaccine often do not receive it. These missed opportunities occur without regard to race/ethnicity and spoken language in the USA [17, 18].

Organizational interventions

Reminders that the vaccination is due now or soon and recalls indicating a patient is late for a vaccination dose have been shown repeatedly to be effective in improving vaccination rates [19,20,21,22,23,24,25,26,27,28]. Reminders and recalls overcome major barriers created by the lack of adolescent health care visits [29] and of patient and parent awareness of the vaccine by offering the provider’s recommendation and providing the date when the next dose is due [30, 31]. Our reminder-recall communication will address message content and tone, adopting expectant (announcement-style or presumptive) language to make a strong recommendation for the vaccine, and it will describe benefits of early vaccination. Both these approaches have been shown to be effective in communication around vaccination [32,33,34,35,36,37,38].

Standing orders or nurse protocols also have proven effective for increasing vaccination rates [19]. These orders permit nurses to vaccinate patients without a provider’s order and support nurse visits without clinical examinations. Thus, standing orders improve access to vaccination and simplify the vaccination process. However, in our system, parents are not systematically informed of the availability and convenience of nurse visits for vaccination. Therefore, we will include this information in the reminder-recall communication, to bring attention to the convenience of scheduling nurse visits for HPV vaccination.

Point-of-care prompts can take the form of a nurse’s review of the current vaccination record at every patient visit and can flag recommended or past-due vaccines in the health record for the provider to address at that visit. Point-of-care prompts have been demonstrated to be effective for the HPV vaccine [21, 26, 39].

Audit and feedback interventions also have shown effectiveness for HPV vaccination [21, 26, 39,40,41,42]. These interventions involve the regularly presenting clinicians with their performance metrics. We propose to enhance this effective strategy by adopting a social pressure approach wherein providers will be given specific feedback on their performance and on the performance of their colleagues [43, 44].

Interpersonal barriers

The presence and quality of provider recommendation for HPV vaccination affect vaccination rates; provider recommendations have been shown across the spectrum of vaccines to result in increased vaccination rates [30, 45,46,47,48,49,50,51,52,53,54]. Furthermore, investigation has shown not only that the presence of the recommendation matters but also that the strength of the recommendation matters [52].

Interpersonal interventions

The Advisory Committee on Immunization Practices (ACIP) has identified the need for providers to strengthen their recommendations for HPV vaccinations [55, 56]. At the interpersonal level, our intervention includes a tool kit for providers that illustrates techniques for making a strong recommendation for HPV vaccination and for addressing questions commonly asked by patients and parents.

Individual barriers

An important individual-level barrier to HPV vaccination is pain, often expressed as a fear of needles or not wanting to receive a shot [49]. Injection-site pain is the most common adverse effect reported by recipients of the HPV vaccine [57]. Walter et al. [58] found modest increases in pain reported by girls age 9 to 18 years with HPV vaccine compared with concomitantly administered vaccines. Burns et al. [59] similarly found that approximately two thirds of the time, HPV vaccination was more painful than concomitant vaccinations given to children age 10 to 18 years.

Individual interventions

A comprehensive systematic review has demonstrated the ability through various measures to reduce pain associated with vaccination [60]. Two systematic reviews showed substantial evidentiary support for topical anesthetics and external cold combined with vibration to reduce vaccination pain [61, 62]. Despite the evidence supporting topical anesthetics with vaccination, they are rarely used [63]. Therefore, we will routinely offer pain-free vaccine administration and ensure that parents and patients learn of this practice through the reminder-recall communication.

Methods design

Research ethics approval

The protocol described herein was approved as a minimal risk study by the Mayo Clinic Institutional Review Board (IRB) (No. 17-010661).

Protocol amendments

This protocol was registered with ClinicalTrials.gov on April 18, 2018 (No. NCT03501992). Results of the study will be reported to ClinicalTrials.gov. The study team will meet weekly to monitor the study and record study progress in the minutes of those meetings. The team will regularly review protocol compliance. The corresponding principal investigator will submit annual reports on study progress to the Mayo Clinic IRB. The principal investigator will report any unexpected and serious adverse events immediately to the IRB and will follow IRB policy and procedures regarding any unanticipated problem involving risk to participants or other persons or any protocol violations.

Trial design

We will test the individual and synergistic effects of novel, multilevel HPV vaccine delivery implementation interventions on rates of initiation and completion of HPV vaccinations in six clinical practices, using a stepped-wedge cluster randomized trial with process evaluation [64,65,66,67]. This will enable the evaluation of both main and synergistic effects. We will also collect rich process information to inform dissemination and implementation efforts that in turn will inform interpretation of the success of the interventions.

The stepped-wedge design permits us to test the presence of each intervention in each primary care practice, allowing each practice to serve as its own control, thereby reducing bias due to imbalanced risk factors across practices. The incorporation of a factorial design allows us to use a single trial to test two interventions and assess them individually and in combination [68, 69]. This design also provides opportunity to conserve sample size while maintaining statistical power [70]. In addition, we will collect data from parents and providers about their experiences with the intervention components. We will use these process measures along with provider- and clinic-level characteristics to analyze outcomes and make adjustments necessary for future dissemination.

Study aims

Over 5 years, we will evaluate two evidence-based implementation interventions with innovative enhancements in six primary care practices to evaluate their individual and combined effect on rates of HPV vaccination among female and male patients. Specifically, we propose to address the following aims and hypotheses.

Aim 1

Test the hypothesis that, compared with no intervention (current practice), a practice-level intervention using reminder-recall communication featuring the availability of nonmedication and medication anesthetics, the convenience of nurse-only visits, and the use of persuasive language for early on-time vaccinations will improve the odds of a child receiving an HPV vaccine dose by at least 20%.

Aim 2

Test the hypothesis that, compared with no intervention (current practice), a provider-level intervention using missed opportunities audit and feedback applying social pressure (peer-performance comparisons) and equipping providers with a strong-recommendation tool kit will improve the odds of a child receiving an HPV vaccine dose by at least 20%.

Aim 3

Test the hypothesis that simultaneous implementation of interventions targeting individual, interpersonal, and organizational factors will have a synergistic effect on HPV vaccine delivery rates, more than doubling the odds of a child receiving an HPV vaccine dose.

Research setting

The study will be conducted in six primary care practices at Mayo Clinic in Rochester, Minnesota, and Kasson, Minnesota (Appendix A). These practices are part of Mayo Clinic’s Employee and Community Health primary care initiative and employ salaried pediatricians, pediatric nurse practitioners, family physicians, and family medicine nurse practitioners (Appendix B). All six practices provide care to children, including those in the trial’s target age group of 11 to 12 years. Table 1 reports the recent incidence rates of initiation and completion of the HPV vaccine series in the six participating practices. These reflect the numbers used in our power calculations. Table 2 reports the recent prevalence rates of the same. Uniformity in critical features across the six clinical practices will enhance the success and efficiency of the study and contribute to the fidelity of the intervention delivery across time, providers, and clinics, as well as the interpretability of the results.

Table 1 Rates of incidence of human papillomavirus vaccine Initiation and of completion in participating practices
Full size table
Table 2 Rates of prevalence of human papillomavirus vaccine initiation and of completion in participating practices
Full size table

Eligibility

Eligibility at the practice level was determined by two criteria: primary care practice in the Mayo Clinic Employee and Community Health practice and provision of care to age-eligible children and adolescents. The Consolidated Standards of Randomized Trials flow diagram is shown in Additional file 1.

Eligible patients are children age 11 and 12 years who in the previous calendar month became eligible for one dose of the HPV vaccine. Review of the patient’s electronic health record at Mayo Clinic will determine whether the patient is due for a dose of the vaccine. The six practices currently conduct such scans for the point-of-care prompts to determine whether the patient presenting for care is due for an HPV vaccine dose. Patients eligible for this study would be age 11 or 12 years at the first day of each 12-month step. Thus, patients eligible for the study and due for the vaccination comprise three groups: those who received no valid HPV vaccine, those who received one valid HPV dose and five calendar months or more have passed since that dose, and those who received two valid doses, but the second dose was given less than 5 months after the first. A valid dose includes a dose administered when the patient was age 9 years or older (the minimum age per ACIP) and that meets the minimum interval [5].

Allocation

Participating practices will be allocated as illustrated in Table 3. Our design has four 12-month steps, for a total of 48 months. To ensure that patient numbers balance across the interventions, we will group the six practices into three pairs according to volume and will block randomize these pairs.

Table 3 Factorial design used in the proposed stepped-wedge cluster randomized trial
Full size table

Interventions

Reminder and recall intervention

During step 1 (current care), we will obtain key stakeholder feedback on our intervention materials and will use this feedback to optimize the overall acceptability of the materials before they are finalized. Specifically, parents of eligible children will be invited to review and provide feedback regarding the reminder-recall communication. Households of the girls and boys in the target age group of 11 to 12 years will be identified from each of the clinical practices. In randomly assigned groups of 75, parents will be invited by mailed letter to participate in individual or group interviews to evaluate the reminder-recall communication. Using the same approach as the National Immunization Survey and National Immunization Survey-Teen, the invitation will be directed toward the person in the household who is the most knowledgeable about that child’s immunizations [71]. Through individual interviews and focus groups, formative and evaluative data will be obtained on a prototype of the reminder-recall communication, to finalize the intervention for study implementation. Central features important to effective health communication will be evaluated qualitatively, including clarity, understandability, believability and convincingness, suggestions for improvement, and overall reactions.

For practices randomly assigned to the reminder-recall intervention, communication to patients due for vaccination will go out at the beginning of each calendar month. In addition, in each month, the practice will send a secure electronic communication to the parent or legal guardian of the patient through the patient’s electronic health record portal. The communication will use expectant (announcement-style or presumptive) language [33, 38]. The reminder-recall intervention will note the child’s vaccination status and state that the child’s provider strongly recommends that the child receive the HPV vaccination now. The communication also will describe the availability of topical anesthetics, indicate the availability and convenience of nurse visits for HPV vaccination, and describe the immunologic and logistic benefits of early vaccination.

This communication will encourage the parent and the patient to take advantage of the pain-reducing measures, including the nonmedication and medication approaches available at the practice. The communication will point out the availability of nurse visits that would permit vaccination without the more difficult-to-schedule provider encounter or examination. The communication will state that younger children have a higher level of antibody response to the HPV vaccine, and early vaccination takes advantage of the less busy and less complicated schedules of younger children relative to teenagers.

In this study, we will use the patient portal as the primary modality for our adolescent patients’ reminder-recall communications. We will send mailed letters to parents or guardians of patients who either have opted out of the portal messaging or do not access the portal within 1 week of delivering the reminder or recall through the portal. To support the reminder-recall intervention, we will conduct a broad education of practice staff—front desk personnel, nurses, and providers—regarding the nature of the intervention, its goals, and its likely impact on the practice.

Audit feedback reports and provider tool kit

The provider-facing intervention consists of two components. The first component is the missed-opportunities audit and feedback. During the steps for which the providers’ practices are randomized to this intervention, we will mail providers a monthly missed-opportunities audit and feedback report. The feedback will indicate their personal HPV vaccine missed-opportunities rate compared with the median rate of their peers and will provide the names of the top performers in their practice for the previous month. The rate will be constructed from a denominator that includes the number of visits during which they saw an 11- or 12-year-old patient eligible for a dose of HPV vaccine. The numerator will include the counts of an HPV vaccine dose ordered and given. The missed-opportunities report will undergo feedback and evaluation with a subset of providers through formative work conducted during step 1 of the protocol.

The second component will include the provision of a strong-recommendation provider tool kit. The tool kit will support providers in making strong recommendations using expectant (announcement-style or presumptive) language to express a strong recommendation [33, 38] and to address any hesitancy by engaging the parent and patient with the Corroborate, About Me, Science, Explain (CASE) approach [72, 73]. The tool kit will be an online resource available only to the staff in practices allocated to the intervention at the beginning of the step. Supervisors and research staff will communicate the availability of the tool kit through emails. To support the provider-facing intervention, we will conduct broad education of the practice staff—front desk personnel, medical secretaries, appointment secretaries, and nurses—in preparation of the intervention about the nature of the intervention, its goals, and its likely impact on the practice.

Process evaluation

We will assess the fidelity and quality of implementation of the interventions and evaluate at the individual, provider, and clinic levels the potential causal and contextual factors that may be associated with observed outcomes. After completion of steps 2, 3, and 4, we will survey samples of parents to evaluate their awareness of and response to the reminder-recall intervention. In addition, we will assess their attitudes and beliefs about HPV vaccination through the use of a modified version of the Carolina HPV Immunization and Attitudes and Beliefs Scale [74,75,76]. We will document the time to respond to the reminder-recall letter and the type of visit scheduled. Finally, we will analyze clinical records’ data to evaluate the use of pain-reducing measures and nurse-only visits. We will also survey all providers to assess their exposure to and experience with receipt of the monthly audit and feedback report and use of and experience with the strong-recommendation tool kit. We will use Web analytics to track access to the online tool kit.

After completion of steps 2, 3, and 4, we will send all providers randomly assigned to the intervention a questionnaire to assess their exposure to and experience with the audit and feedback report, use of and experience with the strong-recommendation tool kit in each step, and use of presumptive language, as well as the CASE approach in their clinical practices. Adapting items from a previously validated instrument used to survey clinicians [77], we will assess provider knowledge of HPV and HPV vaccination, perceived parental barriers to HPV vaccination, provider self-reported perceived strength of HPV vaccination recommendations, and the type of visits during which they offer the vaccine, as well as clinical practice and personal characteristics, to ascertain potential mechanisms of impact. These process measures will be used to conduct secondary analyses of the interventions’ impact on the outcomes of interest.

Data collection

For each 12-month period, we will electronically obtain demographic data (race/ethnicity, sex, and age) for all empaneled patients age 11 and 12 years. Our administrative database will be searched electronically to identify the occurrence and dates of all HPV vaccination of these children to the end of the study with the use of Current Procedural Terminology codes 90649, 90650, and 90651.

Outcomes

Our primary outcome variables are the rates of at least one dose of HPV vaccine for empaneled vaccine-eligible boys and girls, measured at the end of the last day of each 12-month step. Secondary outcome variables include HPV vaccine initiation and completion rates for the eligible patients empaneled to a given practice. Specifically, we will measure at the end of the last day of each 12-month step the initiation and completion rates for the eligible patients empaneled to a given primary care practice who presented for care of any type to that practice during the 12 months of that step.

To qualify for eligibility for initiation, the patient must have been 11 or 12 years of age on the beginning day of the 12-month step. In addition, they must be due for the first dose of HPV vaccine, having never received a previous dose of the vaccine. To qualify for eligibility for completion of the HPV vaccine series, the patient must have been 11 or 12 years of age at the beginning day of the 12-month step and able to complete the series in the following 12 months, having had 0, 1, or 2 doses previously. We also will obtain rates of missed opportunities from the electronic health records. In these rates, the denominator will be the number of encounters that took place during a given step at a particular primary care practice of the empaneled patients eligible for a dose of HPV vaccine. The numerator will be the number of encounters in which an HPV vaccine was not given.

We will further categorize outcomes by patient sex, age, and vaccine dose, as well as by provider type. We will calculate the missed-opportunity rates for those providers in practices randomly assigned to receive the second intervention for aim 2 during those steps (Table 3). We also will examine the association of outcomes with variables ascertained through the process evaluation.

Statistical analyses

We will summarize patient characteristics (sex and age) by intervention status (baseline, intervention 1, intervention 2, or interventions 1 + 2). All patients will be analyzed on an intention-to-treat status; this principle will be extended to the practice status so that delays in the implementation of an intervention will not affect the intervention status of patients [78]. We will use generalized linear mixed models to assess the effects of the interventions [78]. All models will be assessed both overall and stratified by sex in recognition of differential rates of HPV vaccination among male and female patients. Our main model will be a mixed-effects logistic regression model specified as follows.

Let Yijt indicate whether the ith eligible patient at the jth practice during step t (t = 0, ..., 3) receives a vaccine during period t. Then, as recommended by Lyons et al. [68, 69], let

$$ {Y}_{ijt}\sim \mathrm{Binomial}\left({\pi}_{ijt}\right) $$
$$ {\pi}_{ijt}=\mu +{a}_j+{\beta}_t+{\beta}_1X{1}_{j1}+{\beta}_2X{2}_{jt}+{\beta}_3X{1}_{jt}\times X{2}_{jt} $$
(1)

where αj is a random effect for clinical practice (cluster) j, j = 1, 2, …, 6, αj ~ N(0, τ2), βt is a fixed effect for time (t = 0, 1, 2, 3), and Xjt indicates the intervention m at the ith clinical practice at time t. β1 and β2 capture the main effects for the two interventions. β3 captures any interaction (synergy) effect between the two interventions. In the event of a large imbalance in any patient characteristics between baseline and intervention periods, those characteristics and potential interactions will be included in model (1). We then can test the primary aims of the study by testing β1 = 0, β2 = 0, and β3 = 0.

For secondary analyses, we will estimate models similar to the main model, using only the initiation and completion-eligible patients. For all models, we will report C-statistics and between-practice variance. Because of the small number of practices, estimation of between-practice variance may be computationally difficult or give biased results, or both. Thus, we will, through a sensitivity analysis, replicate the main model for all outcomes using a generalized estimating equation. The generalized estimating equation models account appropriately but conservatively for clustering of outcomes and are thus appropriate for stepped-wedge cluster randomized trials [78]. Although they have less power to detect effects, they do not require making assumptions about or estimates of the between-practice variance.

We will incorporate into our secondary analyses the measures from the process evaluation. We also will incorporate other moderating factors to determine their roles in the explanation of the magnitude and direction of the results.

Sample size and power

>Statistical power was estimated using simulation, which accounted for potential differences in baseline rates across sites [79]. On the basis of pilot data from a 12-month period, we have assumed an average cluster size of 800 children (400 boys and 400 girls), a baseline probability of requiring any vaccination of 25% (log odds of − 1.1), and a conservative estimate of the standard deviation of the random effect of 0.5, assuming a secular treatment effect (Table 4). These estimates are conservative. We have sufficient power to detect meaningful patient-centered and provider-level effects in both the sex-stratified and the full cohorts.

Table 4 Power scenarios through mixed-model simulation (no interaction)
Full size table

For the interaction between patient-centered and provider-level effects (with the same assumptions described above) and based on the simulation, we have at least 80% power to detect an odds ratio (OR) of 1.5 or greater when evaluating boys and girls separately and of 1.34 or greater in the full cohort (Table 5). Although our power estimates are conservative, power is somewhat marginal (70% to detect an OR of 1.45) for the synergistic effect in the sex-stratified cohorts (aim 3). However, we have sufficient power to detect an effect using the combined sample.

Table 5 Power scenarios through mixed-model simulation (with interaction)
Full size table

We conducted several sensitivity analyses with adjustment of the terms in our model (baseline probability, random effect standard deviation, and secular trend) and found small changes in our already-conservative power estimates. We used average cluster instead of actual size, which was reasonable given our pilot data indicate only a slight imbalance across the clusters [80]. Our average observed total cluster size was 895, with close to 50% male sex and 50% female sex. We assumed an average cluster size of 800 (400 for each sex) to allow for decreases in eligible participants over time as a result of the effect of our interventions.

Data management and monitoring

For the proposed behavioral intervention, the posed risks do not exceed the threshold of minimal risk. Thus, an independent data management and monitoring committee are not required. Multiple principal investigators will monitor the safety of the participants and the integrity of the data. We will maintain patient safety through the adherence of our clinical staff to the standards of clinical care. Mayo Clinic has established uniform clinical policies and practices for all of the six practice sites, including the adoption of a standardized children and adolescent immunization schedule for southeast Minnesota and an Ambulatory Specialty Procedural Guideline for Vaccine Administration. In addition, Mayo Clinic has adopted standard nursing vaccine protocols for all routine and risk-based vaccinations. Adverse reactions to vaccinations will be managed in the clinical setting by the clinical staff as part of routine care abiding by clinic standards and federal law regarding Vaccine Adverse Events Reporting.

Data integrity

We will use the data integrity best practices promulgated by the Mayo Clinic Division of Biomedical Statistics and Informatics in the Department of Health Sciences Research. These include the use of real-time data validation and checklists developed and standardized for best practices.

Subject privacy

Institutional clinical policies and procedures maintain patient privacy with respect to communications, paperwork, and electronic health record documentation and access. Because patients will participate in the study only through their receipt of routine care, these provisions will ensure subject privacy.

Data confidentiality

We will ensure that all patient data are secure and confidential through adherence to institutional policies and procedures for securely storing, maintaining, and updating health record information. All health record and patient data are securely stored behind the institutional electronic firewall, and these data will not be released externally except under specific data sharing agreements. Additionally, we will store study data in separate, secure servers. Only study personnel will be able to access those servers through appropriate logins and passwords. All study results will be presented in aggregate, and no individual patient or provider will be identifiable.

Product accountability

All practices in this trial use a central pharmacy, which will purchase and receive all 9-valent human papillomavirus (9vHPV) vaccine. These six practices are routinely monitored by the Minnesota Department of Health to ensure adherence to the policies and procedures of vaccine storage and handling, documentation, and administration for the Minnesota Vaccines for Children Program. Mayo Clinic adheres to the Centers for Disease Control and Prevention recommendations and guidelines for vaccine storage and handling, documentation, and administration.

Discussion

Timeline

This is a 5-year proposal. Each design step will involve 12 months, which allows us to balance seasonality of HPV administration across the four steps [81, 82]. Step 1 involves measurement of current practice during which we will engage samples of parents and providers in optimizing the interventions. The next steps will take 36 months, leaving 12 months in year 5 for our analysis.

Limitations and related considerations

We note potential limitations with our approach. First, the factorial design requires all steps to be completed to achieve the full power. Midtrial analyses are likely to be underpowered and at risk for type II errors. Thus, we plan no midtrial analyses. Because the study involves interventions that pose minimal risks, we do not need to conduct ongoing safety and data monitoring for this clinical trial or specify stopping rules. Second, we chose to measure the outcomes of the eligible patients’ status at each of the steps. This is opposed to following either a closed or open cohort from step to step, which may be too mathematically complex to analyze rigorously [83, 84]. Therefore, we ignore whether patients contributed to previous steps in assessing their status for a particular step. We permit patients to drop out of the provider panels, switch providers, or enter the provider panels on the basis of the primary care practices’ policies and practices regarding panel membership. Our choice simplifies eligibility at each step, as well as the mathematical analysis and results, in a conservative measure of the outcome. Third, we recognize that our interventions involve voluntary behaviors. Thus, we will use the process evaluation measures in secondary analyses to understand how intervention exposure and fidelity may impact study outcomes.

Potential for impact and implications and plans for dissemination

The rigor, design, feasibility, and high likelihood of success of this trial will provide important evidence regarding practice and provider interventions to improve HPV vaccination. We will disseminate these findings through peer-reviewed publications, presentations, and professional meetings.

Throughout this effort, we will explore our options for sharing data in accordance with National Institutes of Health policies, with a mind toward the unique challenges around confidentiality and consent waiver of our embedded pragmatic design. Mayo Clinic is a widely dispersed organization with locations in five states. Mayo Clinic also maintains the Mayo Clinic Care Network, which consists of 45 health care organizations with a close working relationship with Mayo. Given this ready resource for dissemination, the interventions being tested in our study (if successful) can be disseminated readily to various clinical settings throughout the USA.

Abbreviations

4vHPV:

4-Valent human papillomavirus

9vHPV:

9-Valent human papillomavirus

ACIP:

Advisory Committee on Immunization Practices

CASE:

Corroborate, About Me, Science, Explain

HPV:

Human papillomavirus

IRB:

Institutional review board

OR:

Odds ratio

SEM:

Social ecological model

References

  1. Viens LJ, Henley SJ, Watson M, Markowitz LE, Thomas CC, Thompson TD, et al. Human papillomavirus-associated cancers—United States, 2008–2012. MMWR Morb Mortal Wkly Rep. 2016;65(26):661–6.

    Article  PubMed  Google Scholar 

  2. Wu X, Watson M, Wilson R, Saraiya M, Cleveland JL, Markowitz L. Human papillomavirus–associated cancers—United States, 2004–2008. JAMA. 2012;308(5):445–7.

    CAS  Google Scholar 

  3. Markowitz LE, Dunne EF, Saraiya M, Lawson HW, Chesson H, Unger ER, et al. Quadrivalent human papillomavirus vaccine: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep. 2007;56(RR-2):1–24.

    PubMed  Google Scholar 

  4. Centers for Disease Control and Prevention. Recommendations on the use of quadrivalent human papillomavirus vaccine in males—Advisory Committee on Immunization Practices (ACIP), 2011. MMWR Morb Mortal Wkly Rep. 2011;60(50):1705–8.

    Google Scholar 

  5. Meites E, Kempe A, Markowitz LE. Use of a 2-dose schedule for human papillomavirus vaccination—updated recommendations of the Advisory Committee on Immunization Practices. MMWR Morb Mortal Wkly Rep. 2016;65(49):1405–8.

    Article  PubMed  Google Scholar 

  6. Reagan-Steiner S, Yankey D, Jeyarajah J, Elam-Evans LD, Singleton JA, Curtis CR, et al. National, regional, state, and selected local area vaccination coverage among adolescents aged 13–17 years—United States, 2014. MMWR Morb Mortal Wkly Rep. 2015;64(29):784–92.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Healthy People 2020. Immunization and infectious diseases: objectives. Washington, DC: Office of Disease Prevention and Health Promotion; 2018. [updated 2018 Apr 17; cited 2018 Apr 17]; Available from: https://www.healthypeople.gov/2020/topics-objectives/topic/immunization-and-infectious-diseases/objectives

    Google Scholar 

  8. Finney Rutten LJ, Wilson PM, Jacobson DJ, Agunwamba AA, Radecki Breitkopf C, Jacobson RM, et al. A population-based study of sociodemographic and geographic variation in HPV vaccination. Cancer Epidemiol Biomark Prev. 2017;26(4):533–40.

    Article  Google Scholar 

  9. US Department of Health and Human Services, National Institutes of Health, National Cancer Institute. Accelerating HPV vaccine uptake: urgency for action to prevent cancer: a report to the president of the United States from the President’s Cancer Panel 2014: Available from: https://deainfo.nci.nih.gov/advisory/pcp/annualreports/HPV/PDF/PCP_Annual_Report_2012-2013.pdf.

    Google Scholar 

  10. Sorensen G, Emmons K, Hunt MK, Johnston D. Implications of the results of community intervention trials. Annu Rev Public Health. 1998;19:379–416.

    Article  PubMed  CAS  Google Scholar 

  11. Stange KC, Breslau ES, Dietrich AJ, Glasgow RE. State-of-the-art and future directions in multilevel interventions across the cancer control continuum. J Natl Cancer Inst Monogr. 2012;2012(44):20–31.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Stokols D. Translating social ecological theory into guidelines for community health promotion. Am J Health Promot. 1996;10(4):282–98.

    Article  PubMed  CAS  Google Scholar 

  13. Sallis JF, Owen N, Fisher EB. Ecological models of health behavior. In: Glanz K, Rimer BK, Viswanath K, editors. Health behavior and health education: theory, research, and practice. 4th ed. San Francisco: Jossey-Bass; 2008. p. 465–85.

    Google Scholar 

  14. Institute of Medicine. Committee on health and behavior: research practice and policy. Health and behavior: the interplay of biological, behavioral, and societal influences. Washington (DC): National Academies Press; 2001.

    Google Scholar 

  15. Hofstetter AM, Rosenthal SL. Factors impacting HPV vaccination: lessons for health care professionals. Expert Rev Vaccines. 2014;13(8):1013–26.

    Article  PubMed  CAS  Google Scholar 

  16. Jacobson RM, Agunwamba AA, St Sauver JL, Finney Rutten LJ. The most effective and promising population health strategies to advance human papillomavirus vaccination. Expert Rev Vaccines. 2016;15(2):257–69.

    Article  PubMed  CAS  Google Scholar 

  17. Mayne SL, duRivage NE, Feemster KA, Localio AR, Grundmeier RW, Fiks AG. Effect of decision support on missed opportunities for human papillomavirus vaccination. Am J Prev Med. 2014;47(6):734–44.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Dorton BJ, Vitonis AF, Feldman S. Identification of potential missed opportunities for human papillomavirus (HPV) vaccination in women referred to an academic colposcopy clinic for evaluation of abnormal cervical cytology. Hum Vaccin Immunother. 2016;12(6):1639–45.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Briss PA, Rodewald LE, Hinman AR, Shefer AM, Strikas RA, Bernier RR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18(1 Suppl):97–140.

    Article  PubMed  CAS  Google Scholar 

  20. Szilagyi P, Vann J, Bordley C, Chelminski A, Kraus R, Margolis P, et al. Interventions aimed at improving immunization rates. Cochrane Database Syst Rev. 2002;4:CD003941.

    Google Scholar 

  21. Fiks AG, Grundmeier RW, Mayne S, Song L, Feemster K, Karavite D, et al. Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt. Pediatrics. 2013;131(6):1114–24.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Cassidy B, Braxter B, Charron-Prochownik D, Schlenk EA. A quality improvement initiative to increase HPV vaccine rates using an educational and reminder strategy with parents of preteen girls. J Pediatr Health Care. 2014;28(2):155–64.

    Article  PubMed  Google Scholar 

  23. Matheson EC, Derouin A, Gagliano M, Thompson JA, Blood-Siegfried J. Increasing HPV vaccination series completion rates via text message reminders. J Pediatr Health Care. 2014;28(4):e35–9.

    Article  PubMed  Google Scholar 

  24. Szilagyi PG, Albertin C, Humiston SG, Rand CM, Schaffer S, Brill H, et al. A randomized trial of the effect of centralized reminder/recall on immunizations and preventive care visits for adolescents. Acad Pediatr. 2013;13(3):204–13.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Kharbanda EO, Stockwell MS, Fox HW, Andres R, Lara M, Rickert VI. Text message reminders to promote human papillomavirus vaccination. Vaccine. 2011;29(14):2537–41.

    Article  PubMed  Google Scholar 

  26. Niccolai LM, Hansen CE. Practice- and community-based interventions to increase human papillomavirus vaccine coverage: a systematic review. JAMA Pediatr. 2015;169(7):686–92.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Szilagyi PG, Humiston SG, Gallivan S, Albertin C, Sandler M, Blumkin A. Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates. Arch Pediatr Adolesc Med. 2011;165(6):547–53.

    Article  PubMed  Google Scholar 

  28. Suh CA, Saville A, Daley MF, Glazner JE, Barrow J, Stokley S, et al. Effectiveness and net cost of reminder/recall for adolescent immunizations. Pediatrics. 2012;129(6):e1437–45.

    Article  PubMed  Google Scholar 

  29. Nordin JD, Solberg LI, Parker ED. Adolescent primary care visit patterns. Ann Fam Med. 2010;8(6):511–6.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Darden PM, Thompson DM, Roberts JR, Hale JJ, Pope C, Naifeh M, et al. Reasons for not vaccinating adolescents: National Immunization Survey of Teens, 2008–2010. Pediatrics. 2013;131(4):645–51.

    Article  PubMed  Google Scholar 

  31. Perkins RB, Chigurupati NL, Apte G, Vercruysse J, Wall-Haas C, Rosenquist A, et al. Why don’t adolescents finish the HPV vaccine series? A qualitative study of parents and providers. Hum Vaccin Immunother. 2016;12(6):1528–35.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Opel DJ, Robinson JD, Heritage J, Korfiatis C, Taylor JA, Mangione-Smith R. Characterizing providers’ immunization communication practices during health supervision visits with vaccine-hesitant parents: a pilot study. Vaccine. 2012;30(7):1269–75.

    Article  PubMed  Google Scholar 

  33. Opel DJ, Heritage J, Taylor JA, Mangione-Smith R, Salas HS, Devere V, et al. The architecture of provider-parent vaccine discussions at health supervision visits. Pediatrics. 2013;132(6):1037–46.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Opel DJ, Mangione-Smith R, Robinson JD, Heritage J, DeVere V, Salas HS, et al. The influence of provider communication behaviors on parental vaccine acceptance and visit experience. Am J Public Health. 2015;105(10):1998–2004.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Opel DJ, Omer SB. Measles, mandates, and making vaccination the default option. JAMA Pediatr. 2015;169(4):303–4.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Nyhan B, Reifler J, Richey S, Freed GL. Effective messages in vaccine promotion: a randomized trial. Pediatrics. 2014;133(4):e835–42.

    Article  PubMed  Google Scholar 

  37. Nyhan B, Reifler J. Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine. 2015;33(3):459–64.

    Article  PubMed  Google Scholar 

  38. Brewer NT, Hall ME, Malo TL, Gilkey MB, Quinn B, Lathren C. Announcements versus conversations to improve HPV vaccination coverage: a randomized trial. Pediatrics. 2017;139(1). https://doi.org/10.1542/peds.2016-1764

  39. Bundy DG, Persing NM, Solomon BS, King TM, Murakami PN, Thompson RE, et al. Improving immunization delivery using an electronic health record: the ImmProve project. Acad Pediatr. 2013;13(5):458–65.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Elam-Evans LD, Yankey D, Jeyarajah J, Singleton JA, Curtis RC, MacNeil J, et al. National, regional, state, and selected local area vaccination coverage among adolescents aged 13–17 years—United States, 2013. MMWR Morb Mortal Wkly Rep. 2014;63(29):625–33.

    PubMed  PubMed Central  Google Scholar 

  41. Moss JL, Reiter PL, Dayton A, Brewer NT. Increasing adolescent immunization by webinar: a brief provider intervention at federally qualified health centers. Vaccine. 2012;30(33):4960–3.

    Article  PubMed  Google Scholar 

  42. Gilkey MB, Dayton AM, Moss JL, Sparks AC, Grimshaw AH, Bowling JM, et al. Increasing provision of adolescent vaccines in primary care: a randomized controlled trial. Pediatrics. 2014;134(2):e346–53.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Kallgren CA, Reno RR, Cialdini RB. A focus theory of normative conduct: when norms do and do not affect behavior. Personal Soc Psychol Bull. 2000;26(8):1002–12.

    Article  Google Scholar 

  44. Cialdini RB, Goldstein NJ. Social influence: compliance and conformity. Annu Rev Psychol. 2004;55:591–621.

    Article  PubMed  Google Scholar 

  45. Ylitalo KR, Lee H, Mehta NK. Health care provider recommendation, human papillomavirus vaccination, and race/ethnicity in the US National Immunization Survey. Am J Public Health. 2013;103(1):164–9.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Darden PM, Jacobson RM. Impact of a physician recommendation. Hum Vaccin Immunother. 2014;10(9):2632–5.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Freeman VA, Freed GL. Parental knowledge, attitudes, and demand regarding a vaccine to prevent varicella. Am J Prev Med. 1999;17(2):153–5.

    Article  PubMed  CAS  Google Scholar 

  48. Holman DM, Benard V, Roland KB, Watson M, Liddon N, Stokley S. Barriers to human papillomavirus vaccination among US adolescents: a systematic review of the literature. JAMA Pediatr. 2014;168(1):76–82.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Rambout L, Tashkandi M, Hopkins L, Tricco AC. Self-reported barriers and facilitators to preventive human papillomavirus vaccination among adolescent girls and young women: a systematic review. Prev Med. 2014;58:22–32.

    Article  PubMed  Google Scholar 

  50. Staras SA, Vadaparampil ST, Patel RP, Shenkman EA. Parent perceptions important for HPV vaccine initiation among low income adolescent girls. Vaccine. 2014;32(46):6163–9.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Perkins RB, Lin M, Silliman RA, Clark JA, Hanchate A. Why are U.S. girls getting meningococcal but not human papilloma virus vaccines? Comparison of factors associated with human papilloma virus and meningococcal vaccination among adolescent girls 2008 to 2012. Womens Health Issues. 2015;25(2):97–104.

    Article  PubMed  Google Scholar 

  52. Rosenthal SL, Weiss TW, Zimet GD, Ma L, Good MB, Vichnin MD. Predictors of HPV vaccine uptake among women aged 19–26: importance of a physician’s recommendation. Vaccine. 2011;29(5):890–5.

    Article  PubMed  CAS  Google Scholar 

  53. Dorell C, Yankey D, Strasser S. Parent-reported reasons for nonreceipt of recommended adolescent vaccinations, national immunization survey: teen, 2009. Clin Pediatr (Phila). 2011;50(12):1116–24.

    Article  Google Scholar 

  54. Stokley S, Jeyarajah J, Yankey D, Cano M, Gee J, Roark J, et al. Human papillomavirus vaccination coverage among adolescents, 2007–2013, and postlicensure vaccine safety monitoring, 2006–2014—United States. MMWR Morb Mortal Wkly Rep. 2014;63(29):620–4.

    PubMed  PubMed Central  Google Scholar 

  55. Markowitz LE, Dunne EF, Saraiya M, Chesson HW, Curtis CR, Gee J, et al. Human papillomavirus vaccination: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep. 2014;63(RR-05):1–30.

    PubMed  Google Scholar 

  56. Centers for Disease Control and Prevention. Vaccines for your children: protect your child at every age. 2016 [updated Apr 15; cited 2018 Apr 17]; Available from: https://www.cdc.gov/vaccines/parents/protecting-children/index.html

    Google Scholar 

  57. Moreira ED Jr, Block SL, Ferris D, Giuliano AR, Iversen OE, Joura EA, et al. Safety profile of the 9-valent HPV vaccine: a combined analysis of 7 phase III clinical trials. Pediatrics. 2016;138(2). https://doi.org/10.1542/peds.2015-4387

  58. Walter EB, Kemper AR, Dolor RJ, Dunne EF. Pain in adolescent girls receiving human papillomavirus vaccine with concomitantly administered vaccines. Pediatr Infect Dis J. 2015;34(2):200–2.

    Article  PubMed  Google Scholar 

  59. Joseph Burns J, Flock N, Lopez M, Hobby L, Lopez-Beyer A, Amin RA. Blinded comparison of injection pain with quadrivalent human papilloma virus vaccine versus other vaccines. J Pediatric Infect Dis Soc. 2015;4(4):396–7.

    PubMed  Google Scholar 

  60. Taddio A, Ilersich AL, Ipp M, Kikuta A, Shah V, Team HE. Physical interventions and injection techniques for reducing injection pain during routine childhood immunizations: systematic review of randomized controlled trials and quasi-randomized controlled trials. Clin Ther. 2009;31(Suppl 2):S48–76.

    Article  PubMed  Google Scholar 

  61. Shah V, Taddio A, McMurtry CM, Halperin SA, Noel M, Pillai Riddell R, et al. Pharmacological and combined interventions to reduce vaccine injection pain in children and adults: systematic review and meta-analysis. Clin J Pain. 2015;31(10 Suppl):S38–63.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Taddio A, Shah V, McMurtry CM, MacDonald NE, Ipp M, Riddell RP, et al. Procedural and physical interventions for vaccine injections: systematic review of randomized controlled trials and quasi-randomized controlled trials. Clin J Pain. 2015;31(10 Suppl):S20–37.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Taddio A, Manley J, Potash L, Ipp M, Sgro M, Shah V. Routine immunization practices: use of topical anesthetics and oral analgesics. Pediatrics. 2007;120(3):e637–43.

    Article  PubMed  Google Scholar 

  64. Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006 Nov 8;6:54.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Herrin J, da Graca B, Nicewander D, Fullerton C, Aponte P, Stanek G, et al. The effectiveness of implementing an electronic health record on diabetes care and outcomes. Health Serv Res. 2012;47(4):1522–40.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Fatemi Y, Jacobson RM. The stepped wedge cluster randomized trial and its potential for child health services research: a narrative review. Acad Pediatr. 2015;15(2):128–33.

    Article  PubMed  Google Scholar 

  67. Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ. 2015;350:h391.

    Article  PubMed  CAS  Google Scholar 

  68. Lyons VH, Li L, Hughes J, Rowhani-Rahbar A. Evaluation of multiple interventions using a stepped wedge design. UW Biostatistics Working Paper Series. 2017: Available from: https://biostats.bepress.com/uwbiostat/paper411.

    Google Scholar 

  69. Lyons VH, Li L, Hughes JP, Rowhani-Rahbar A. Proposed variations of the stepped-wedge design can be used to accommodate multiple interventions. J Clin Epidemiol. 2017;86:160–7.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Woertman W, de Hoop E, Moerbeek M, Zuidema SU, Gerritsen DL, Teerenstra S. Stepped wedge designs could reduce the required sample size in cluster randomized trials. J Clin Epidemiol. 2013;66(7):752–8.

    Article  PubMed  Google Scholar 

  71. Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases. About the National Immunization Surveys (NIS). 2018 [updated 2018 Apr 11; cited 2018 Apr 17]; Available from: https://www.cdc.gov/vaccines/imz-managers/nis/about.html

    Google Scholar 

  72. Jacobson RM. Making the C.A.S.E. for the human papillomavirus vaccine: how to talk to parents and adolescents. Minn Med. 2014;97(2):38–42.

    PubMed  Google Scholar 

  73. Jacobson RM, Van Etta L, Bahta L. The C.A.S.E. approach: guidance for talking to vaccine-hesitant parents. Minn Med. 2013;96(4):49–50.

    PubMed  Google Scholar 

  74. McRee AL, Brewer NT, Reiter PL, Gottlieb SL, Smith JS. The Carolina HPV immunization attitudes and beliefs scale (CHIAS): scale development and associations with intentions to vaccinate. Sex Transm Dis. 2010;37(4):234–9.

    PubMed  Google Scholar 

  75. Gowda C, Carlos RC, Butchart AT, Singer DC, Davis MM, Clark SJ, et al. CHIAS: a standardized measure of parental HPV immunization attitudes and beliefs and its associations with vaccine uptake. Sex Transm Dis. 2012;39(6):475–81.

    Article  PubMed  Google Scholar 

  76. Dempsey AF, Fuhrel-Forbis A, Konrath S. Use of the Carolina HPV Immunization Attitudes and Beliefs Scale (CHIAS) in young adult women. PLoS One. 2014;9(6):e100193.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  77. Rutten LJ, St Sauver JL, Beebe TJ, Wilson PM, Jacobson DJ, Fan C, et al. Clinician knowledge, clinician barriers, and perceived parental barriers regarding human papillomavirus vaccination: association with initiation and completion rates. Vaccine. 2017;35(1):164–9.

    Article  PubMed  Google Scholar 

  78. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–91.

    Article  PubMed  Google Scholar 

  79. Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Sample size calculation for a stepped wedge trial. Trials. 2015;16:354.

    Article  PubMed  PubMed Central  Google Scholar 

  80. Guittet L, Ravaud P, Giraudeau B. Planning a cluster randomized trial with unequal cluster sizes: practical issues involving continuous outcomes. BMC Med Res Methodol. 2006;6:17.

    Article  PubMed  PubMed Central  Google Scholar 

  81. Moss JL, Reiter PL, Rimer BK, Ribisl KM, Brewer NT. Summer peaks in uptake of human papillomavirus and other adolescent vaccines in the United States. Cancer Epidemiol Biomark Prev. 2016;25(2):274–81.

    Article  Google Scholar 

  82. Lorick SA, Fishbein D, Weintraub E, Wortley PM, Lee GM, Zhou F, et al. Uptake of meningococcal conjugate vaccine among adolescents in large managed care organizations, United States, 2005: demand, supply and seasonality. BMC Infect Dis. 2009;9:175.

    Article  PubMed  PubMed Central  Google Scholar 

  83. Davey C, Hargreaves J, Thompson JA, Copas AJ, Beard E, Lewis JJ, et al. Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials. 2015;16:358.

    Article  PubMed  PubMed Central  Google Scholar 

  84. Hughes JP, Granston TS, Heagerty PJ. Current issues in the design and analysis of stepped wedge trials. Contemp Clin Trials. 2015;45(Pt A):55–60.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We acknowledge the following individuals for their assistance in developing this protocol: Stephanie M. Pagel, MA; Jennifer L. Brickley, RN; and Katy M. Bos, APRN, CNS, MS.

Mayo Clinic does not endorse specific products or services included in this article.

Funding

This research is supported by the National Cancer Institute of the National Institutes of Health under award number R01CA217889. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Availability of data and materials

The datasets generated during the current study are not publicly available because of the concerns about confidentiality and waiver of consent and assent but will be made available from the corresponding author on reasonable request after primary trial results are published. We will explore the development of a private data enclave; with this approach, we would develop parameters around appropriate data use, maintain authority over data use, and, when deemed appropriate, permit investigators to perform analyses without taking possession of the data.

Author information

Authors and Affiliations

  1. Division of Health Care Policy and Research, Mayo Clinic, Rochester, MN, USA

    Lila J. Finney Rutten, Carmen Radecki Breitkopf & Robert M. Jacobson

  2. Robert D. and Patricia E. Kern Center for Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA

    Lila J. Finney Rutten, Jennifer L. St. Sauver, Ivana T. Croghan & Robert M. Jacobson

  3. Division of Epidemiology, Mayo Clinic, Rochester, MN, USA

    Jennifer L. St. Sauver & Ivana T. Croghan

  4. Nicotine Dependence Center, Mayo Clinic, Rochester, MN, USA

    Ivana T. Croghan

  5. Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN, USA

    Debra J. Jacobson & Patrick M. Wilson

  6. Yale University, New Haven, CT, USA

    Jeph Herrin

  7. Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, MN, USA

    Robert M. Jacobson

Authors
  1. Lila J. Finney Rutten
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carmen Radecki Breitkopf
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jennifer L. St. Sauver
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ivana T. Croghan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Debra J. Jacobson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Patrick M. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jeph Herrin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Robert M. Jacobson
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LJFR, CRB, JLSS, RMJ and ITC developed the study design and drafted the study protocol. DJJ, PMW, and JH developed the analytic and statistical plan. All authors will participate in the study implementation and will have read and approved the final manuscript.

Corresponding author

Correspondence to Lila J. Finney Rutten.

Ethics declarations

Ethics approval and consent to participate

The protocol described herein was approved as a minimal risk study by the Mayo Clinic IRB on February 21, 2018, as IRB No. 17-010661. Persons who participate in the planned focus groups will be asked to provide written consent.

Consent for publication

Not applicable.

Competing interests

LJFR, ITC, CRB, JH, PMW, DJJ, and JLSS have no competing interests to declare. RMJ serves as a member of a safety review committee for a postlicensure safety study of Gardasil (4vHPV) and one for a postlicensure safety study of Gardasil 9 (9vHPV). He also serves on a data-monitoring committee for a series of prelicensure trials of a 15-valent pneumococcal vaccine. All of these studies are funded by Merck & Co, Inc.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

The consolidated standards of reporting trials diagram for the stepped-wedge cluster randomized trial. (TIF 1392 kb)

Appendices

Appendix A

Clinical sites participating in the trial

We will test our hypotheses in the six primary care practices at Mayo Clinic in Rochester, Minnesota, and Kasson, Minnesota, summarized in this table.

Table 6 Clinical sites participating in the stepped-wedge randomized trial
Full size table

Appendix B

Description of trial setting

Mayo Clinic Employee and Community Health (ECH) serves as the practice setting for the proposed trial interventions. The ECH initiative, for which Sarah J. Crane, MD, is the practice chair, brings together Mayo Clinic’s primary care departments of Family Medicine, Primary Care Internal Medicine, and Community Pediatric and Adolescent Medicine, which include the six clinical sites (called clusters) identified for our trial. It embraces the Mayo Clinic Model of Community Care to provide the right care, delivered by the right provider in the right place at the right time. The goal is to deliver the highest-value care to an identified population of patients. The financial model under which ECH operates is an approach of the total cost of care. The primary focus is to deliver the highest quality and best patient care experience at the most appropriate cost to the system. To achieve this goal, ECH concentrates on core principles frequently associated with patient-centered medical homes. Care should be provided by a physician-led multidisciplinary team. All members of the team should function at the top of their licensure. Appropriate care is delivered at the right time from the right provider in the right place. Care is provided both in the office and outside of the office. ECH maintains a continuous relationship with patients through active care management. The chronic care model also serves as an important philosophy to guide practice redesign within ECH.

The six practices in our trial provide primary care for more than 151,000 empaneled patients who are cared for at seven full-service clinical practices and two express care sites. These are located in Dodge and Olmsted counties in Minnesota. There are 41 care teams of 116 physicians, including the principal investigator, Robert M. Jacobson, MD; 166 resident physicians; and 42 nurse practitioner and physician assistant providers. Patients receive reminders about necessary preventive services at the time of visits through an electronic alert system and also through mail or electronic communication on a secure patient portal that is compliant with the Health Insurance Portability and Accountability Act. Care management of chronic diseases such as asthma, diabetes mellitus, and depression is accomplished through a team approach that uses care coordinators, registered nurses, protocols, and patient education. High-use patients with multiple, often poorly controlled, chronic diseases are enrolled in a structured care management program.

In 2016, the six practices in our trial empaneled 151,322 patients, of whom 41,388 were 18 years old or younger. The 151,322 patients had a total of 985,535 unique patient encounters with 411,799 outpatient office visits from August 1, 2015, through August 1, 2016. During this time, 66,847 outpatient visits occurred for children younger than age 11 years and 32,799 outpatient visits for adolescents age 11 through 18 years. In addition, 98,112 vaccinations were given to children younger than 11 years and 49,769 vaccinations given to adolescents age 11 through 18 years. The following table shows the current distribution of 11- and 12-year-old children (the cohort of interest for this study) for sex and race/ethnicity.

Table 7 Distribution of 11- and 12-year-old children in the six clusters in the trial by sex and race/ethnicity
Full size table

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Finney Rutten, L.J., Radecki Breitkopf, C., St. Sauver, J.L. et al. Evaluating the impact of multilevel evidence-based implementation strategies to enhance provider recommendation on human papillomavirus vaccination rates among an empaneled primary care patient population: a study protocol for a stepped-wedge cluster randomized trial. Implementation Sci 13, 96 (2018). https://doi.org/10.1186/s13012-018-0778-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-018-0778-x

Keywords

Implementation Science

ISSN: 1748-5908