Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Implementing school nursing strategies to reduce LGBTQ adolescent suicide: a randomized cluster trial study protocol

Implementation Science201611:145

https://doi.org/10.1186/s13012-016-0507-2

Received: 19 August 2016

Accepted: 8 October 2016

Published: 22 October 2016

Abstract

Background

Reducing youth suicide in the United States (U.S.) is a national public health priority, and lesbian, gay, bisexual, transgender, and queer or questioning (LGBTQ) youth are at elevated risk. The Centers for Disease Control and Prevention (CDC) endorses six evidence-based (EB) strategies that center on meeting the needs of LGBTQ youth in schools; however, fewer than 6 % of U.S. schools implement all of them. The proposed intervention model, “RLAS” (Implementing School Nursing Strategies to Reduce LGBTQ Adolescent Suicide), builds on the Exploration, Preparation, Implementation, and Sustainment (EPIS) conceptual framework and the Dynamic Adaptation Process (DAP) to implement EB strategies in U.S. high schools. The DAP accounts for the multilevel context of school settings and uses Implementation Resource Teams (IRTs) to facilitate appropriate expertise, advise on acceptable adaptations, and provide data feedback to make schools implementation ready and prepared to sustain changes.

Methods/Design

Mixed methods will be used to examine individual, school, and community factors influencing both implementation process and youth outcomes. A cluster randomized controlled trial will assess whether LGBTQ students and their peers in RLAS intervention schools (n = 20) report reductions in suicidality, depression, substance use, bullying, and truancy related to safety concerns compared to those in usual care schools (n = 20). Implementation progress and fidelity for each EB strategy in RLAS intervention schools will be examined using a modified version of the Stages of Implementation Completion checklist. During the implementation and sustainment phases, annual focus groups will be conducted with the 20 IRTs to document their experiences identifying and advancing adaptation supports to facilitate use of EB strategies and their perceptions of the DAP.

Discussion

The DAP represents a data-informed, collaborative, multiple stakeholder approach to progress from exploration to sustainment and obtain fidelity during the implementation of EB strategies in school settings. This study is designed to address the real-world implications of enabling the use of EB strategies by school nurses with the goal of decreasing suicide and youth risk behaviors among LGBTQ youth. Through its participatory processes to refine and sustain EB strategies in high schools, the RLAS represents a novel contribution to implementation science.

Trial registration

ClinicalTrials.gov, NCT02875535

Keywords

Evidence-based practice Implementation Sexual and gender minority School nurses—suicide

Background

In 2012, the United States (U.S.) Surgeon General identified lesbian, gay, bisexual, transgender, and queer or questioning (LGBTQ) youth as at heightened risk for suicide [1]. Risk factors include depression, substance use, inadequate social support, and not feeling safe at school [17]. Large population-based studies over the past 15 years have found that lesbian, gay, and bisexual (LGB) youth are at two to four times increased risk for suicidal ideation and attempts when compared to their cisgender, heterosexual peers [13, 5, 816]. Within LGB populations, suicidality disproportionately affects racial and ethnic minorities, including American Indian and Hispanic people [1, 17, 18]. In numerous surveys and qualitative studies, transgender youth report elevated risk for suicide, depression, and substance use [1924].

In general, LGBTQ youth report high levels of rejection, harassment, victimization, violence, and sexual abuse that can contribute to mental health problems and suicide behaviors [3, 9, 13, 18, 25]. These behaviors relate to “minority stress,” i.e., chronic stress from stigmatization, prejudice, and discrimination [26]. National school climate surveys have found that LGBTQ youth are often exposed to minority stress in schools [25, 27]. Youth describe being victimized because of their known, or perceived, sexual orientation or gender identity [811, 13, 19, 27]. LGBTQ youth are also more likely to experience high school victimization when they disclose their orientation, self-identify as a sexual minority, recognize same-sex feelings at a younger age, or demonstrate gender-atypical behavior [14, 28].

LGBTQ youth with greater school connectedness and safety report lower suicidal ideation and attempts [29]. Gay-Straight Alliances (GSAs)—peer-to-peer support groups—protect against suicide and depression for LGBTQ students [25, 3034]. School policies are also pivotal to the mental health of LGBTQ youth. LGBTQ students at schools with anti-harassment policies may feel safer and are less likely to be harassed [35]. Those at schools with supportive staff, anti-bullying policies, and GSA clubs are less likely to be victimized, skip school because of safety concerns, or attempt suicide compared with those in other schools [32]. LGBTQ youth may be at lower risk for attempting suicide if they attend school in districts with anti-bullying policies covering sexual orientation and gender identity [36]. LGBTQ students in settings with more protective school climates report fewer suicidal thoughts than those in places with less protective climates [37].

A supportive, safe school environment is key to a comprehensive public health strategy to prevent youth suicide [38]. As a protective factor against suicidal ideation and attempts, school connectedness is second in importance only to family connectedness [39]. The Centers for Disease Control and Prevention (CDC) endorses six EB strategies for schools to meet the needs of LGBTQ youth (see Table 1) [40]. However, the 2012 School Health Profiles Report based on data from 44 states found that only 5.5 % of secondary schools implement all six [40].
Table 1

Description of the school-based evidence-based strategies to meet the needs of LGBTQ youth

1. Identify “safe spaces” such as a counselor’s office, designated classroom, or student organization where LGBTQ youth can receive support from administrators, teachers, other school staff, or other students.

2. Prohibit harassment and bullying based on a student’s perceived or actual sexual orientation or gender expression.

3. Facilitate access to providers not on school property who have experience delivering health services, including human immunodeficiency virus (HIV)/sexually transmitted infection (STI) testing, counseling, and reproductive healthcare, to LGBTQ youth.

4. Facilitate access to providers not on school property who have experience in providing social and psychological services to LGBTQ youth.

5. Encourage staff members to attend professional development on safe and supportive school environments for all students, regardless of sexual orientation, gender identity, or gender expression.

6. Provide health education curricula or supplemental materials, i.e., HIV, STI, or pregnancy prevention information relevant, to LGBTQ youth (e.g., curricula or materials that use inclusive language or terminology).

Several factors within an organization can impact the success of implementation efforts. For example, organizational culture and climate influence staff willingness to engage in new practices [41], as do job tenure and level of professional development [42]. Leadership is also important [43]. Implementation leaders require capacity to be successful change agents and local champions, and their ability to motivate and interact effectively shapes staff attitudes toward adopting an EB strategy [44]. Personal innovativeness or ability to adapt or change can also impact attitudes toward new ways of working in a team or organization [45]. Provider attitudes toward adopting EB practices are also associated with actual utilization [46].

Schools are vital but largely untapped venues for intervention research on LGBTQ youth [2]. However, efforts to address the unmet mental health needs of youth in general and LGBTQ students specifically may not reach their full potential due to implementation challenges in school settings. The proposed intervention model, “RLAS” (Implementing School Nursing Strategies to Reduce LGBTQ Adolescent Suicide), builds on the Exploration, Preparation, Implementation, and Sustainment (EPIS) conceptual framework and the Dynamic Adaptation Process (DAP) to implement EB strategies in U.S. high schools [47, 48].

The EPIS framework segments implementation into four phases: exploration (considering new approaches to carry out EB strategies); preparation (planning to apply EB strategies); implementation (ongoing planning, training, coaching, and use of EB strategies); and sustainment (maintaining EB strategies over time) [47]. Per Fig. 1, the model emphasizes three levels of influence: system (or school), provider (or school nurse), and client (LGBTQ student). The model also attends to factors pertinent to both the “inner context” (e.g., schools, school nurses) and the larger “outer context” (e.g., policies, funding, resources) of EB strategy implementation and sustainment.
Fig. 1

Key Dynamic Adaptation Process components to support evidence-based strategy implementation per the Exploration, Preparation, Implementation, and Sustainment framework

The DAP is an implementation strategy guided by the EPIS framework that provides direction for activities to undertake during each EPIS phase and a continuously iterative data-informed approach to support EB strategy implementation [48]. The DAP has four key components: initial assessment; stakeholder engagement and training; problem solving; and outcomes feedback to address challenges. Another core feature of the DAP is the development of an Implementation Resource Team (IRT), a collaboration comprised of multiple stakeholders to assist with implementation, interpret data, and address adaptation as an explicit part of the implementation process.

Interventions to promote mental health in schools often fail because they want for leadership and staffing [49]. Leaders can positively or negatively impact the capacity to foster change and thus are instrumental in facilitating a positive climate and attitudes for implementing EB strategies [43, 44]. The Institute of Medicine has recommended that nurses in the U.S. should “practice to the full extent of their education and training” and be “full partners” in “redesigning healthcare” [50]. In the current study, school nursing expertise is applied in a distinctive way to lead IRTs in implementing the six EB strategies in their schools. This study combines qualitative methods (e.g., in-depth interviews and focus groups) with population-based surveys and other quantitative measures to pursue three specific aims:
  1. 1.

    Utilize the DAP to enable specially-trained school nurse champions and IRTs to implement and sustain EB strategies to address the needs of LGBTQ high school students.

     
  2. 2.

    Conduct a cluster randomized controlled trial to assess whether LGB youth and peers in RLAS intervention schools report reduced suicidality, depression, substance use, and bullying, and increased safety compared to control schools.

     
  3. 3.

    Examine the individual, school, and community factors influencing implementation and outcomes.

     

Methods/Design

Study context

This work will occur in New Mexico (NM) high schools. Sociodemographic and minority stress factors contributing to adverse psychosocial outcomes are common in NM, which ranks 43rd in personal income per capita ($34,133) and second in persons below the poverty level (20.2 %) [51]. Of NM’s 33 counties, 32 are medically underserved [52]. Suicide rates for NM youth, ages 15 to 24, are nearly twice the national rate (19.7/100,000) [53, 54]. American Indian and Hispanic people comprise almost 56 % of 2,085,287 NM residents [55] and, like LGBTQ people, are disproportionately affected by health disparities [56, 57].

The New Mexico Youth Risk and Resiliency Survey (NM YRRS) is part of the nationwide CDC Youth Risk Behavior Surveillance System. The NM YRRS is administered to high school students in odd-numbered years and uses two discrete population-based sampling designs. A smaller sample provides data representative of the state and a larger sample furnishes data on specific populations not well represented in the smaller state sample. In 2013, data from 5451 students were obtained for the statewide sample and data from 19,080 students were obtained for the larger sample. A minority of students, 26.1 %, were White, non-Hispanic. Hispanics comprised 59.2 % of the population, African Americans, 0.9 %, and other races/ethnicities, 13.7 %. Half (48.8 %) of students were female [58]. Suicidality rates among NM students far exceeded national averages; 15.6 % seriously considered suicide, 13.7 % planned an attempt, and almost 1 in 10 (9.4 %) attempted it in the last year [59].

The 2013 NM YRRS was the first to include questions about same-sex sexual contact and sexual identity of NM high school students, which enables the NM Department of Health and schools to better understand the health and safety risks of students. Almost 11 % of the sample identified as LGB, with 3.0 % identifying as lesbian or gay and 7.5 % as bisexual [60]. In NM, LGB students experienced a relative lack of safety at schools: 35.5 % reported being bullied on school property in the last year, compared with 17.0 % of their straight peers; 17.3 % skipped school because of safety concerns, compared with 5.6 % of their peers [61]. Suicidality rates among NM LGB students were strikingly high with 38.3 % considering suicide, 35.4 % reporting making a suicide plan, and 28.5 % attempting suicide in the past year [61].

Study design

This 5-year cluster randomized trial will include 40 NM high schools that employ a nurse on staff (n = 64). The schools will be randomized into the RLAS intervention condition (IC; n = 20) or the control condition (CC; n = 20). Cluster randomization was chosen because the intervention will be delivered at the school level. Schools will be eligible if they express interest in the full intervention and the school nurse and a lead administrator agree to participate and support the project. To minimize confounding effects, eligible schools will share the following characteristics: (1) public high school; (2) school nurse willingness to convene an IRT if randomized to the IC group; and (3) informed written consent from the school nurse to participate for four years. Additional file 1 contains confirmation of ethical approval.

From the IC schools, there will be three study populations: (1) school nurses; (2) school administrators; and (3) IRT members. From CC schools, there will be two study populations: (1) school nurses and (2) school administrators. School nurses and IRTs in the IC schools will receive innovation supports to encourage greater implementation of the six EB strategies (see Table 1). Nurses in the IC will recruit IRT members (4–6 people) willing to work with them on implementing the six EB strategies. These nurse champions will function as coordinators who keep the IRTs motivated, cohesive, and focused on goals; gain administrative buy-in during the Preparation Phase; and facilitate IRT planning in the Implementation Phase.

Aim 1: utilize the DAP to enable specially-trained school nurse champions and IRTs to implement and sustain EB strategies to address the needs of LGBTQ high school students

Exploration phase

For the exploration phase, the DAP centers on recruiting, enrolling, and randomizing schools, followed by initial assessments of system, provider, and client data to identify school needs, strengths, barriers, and readiness to implement the EB strategies. Results of the initial assessments will be shared with the IC schools specifically via comprehensive summaries to be vetted by the IRTs and integrated into their respective planning for implementation.

Qualitative data collection

Readiness interviews will consist of open-ended questions that will be asked of IC stakeholders at the system, provider, and client levels. These include 20 administrators, 20 school nurses, and 20 individuals randomly selected from the IRTs (n = 60). Interviews will explore if the necessary conditions exist for implementation and sustained use of the EB strategies and how best to optimize them. Questions will center on attitudes toward, access to, and availability of school and community supports for (a) suicide prevention, (b) LGBTQ youth, and (c) school nursing staff. Verbal and written information regarding school policies and practices pertinent to EB strategies will be collected. Specific questions will focus on transgender and intersex youth, i.e., perceptions and knowledge related to their risks for suicide, substance use, bullying, and truancy within school settings, and extant support. Finally, interviews will address pragmatic concerns (e.g., staffing and resources) and organizational factors (e.g., leadership and supportive culture/climate) [43, 47, 48, 62]. Results of these first assessments will inform IRT efforts to build implementation capacity and subsequent plans to implement and sustain EB strategies. For comparison purposes, a complementary set of interviews will be completed with the 20 nurses and the 20 administrators in the CC schools.

Qualitative data analysis

All qualitative interviews will be digitally recorded and professionally transcribed. Hand written notes will be organized according to a standard format, with information on date, time, and length of the interaction, setting, and participants involved. Typed notes and transcripts will be imported into a password protected NVivo database for efficient organization and analysis via iterative readings or codings [63]. Responses will first be analyzed by “open coding” to discover themes, ideas, and issues; a coded index of topics addressed in the data will next be developed. “Focused coding” will then be used to determine which topics arise often and which represent unusual or particular concerns [64, 65]. NVivo search functions can quickly locate and cross-reference statements of interest from participants (e.g., all text coded with “bullying at school,” “nonjudgmental teachers,” and “implementation barriers”) to examine relationships in the data within and across participant groups (e.g., school nurse, administrator, and IRT member).

Interview findings will be triangulated across participant types, creating a matrix (a) describing specific themes related to key study issues outlined in the EPIS model, and (b) supporting data by participant type. A side-by-side comparison of the various perspectives will be used to identify points of convergence and divergence in all participant statements related to the themes under consideration. In this staged approach to analysis, sets of notes and transcripts will be coded and detailed memos that describe and link codes to each theme and issue will be created. Discrepancies in coding and analysis will be identified during this process and resolved during regular team meetings.

Quantitative data collection

The 40 administrators and 40 school nurses in both the IC and CC schools will complete the first of an annual 30-min web-based survey that assesses organizational characteristics of schools and administrator/nurse work attitudes.

Measures. Demographics include age, gender, sexual orientation, race/ethnicity, education, employment, experience, geographical area of work, and activities performed in schools. Attitudes toward sexual and gender minorities will be assessed through two validated and reliable instruments, the Attitudes Toward Lesbians and Gays Scale [66] and the Attitudes Toward Transgender Individuals Scale [67]. Organizational culture and climate will be assessed via the Organizational Social Context Scale, which has good psychometric properties with Cronbach’s alphas ranging from 0.78 to 0.94 [42]. Climate includes three domains that affect use of EB strategies: (1) stress; (2) engagement; and (3) functionality. Culture includes (1) proficiency (responsiveness, competence) and (2) resistance (apathy, suppression). Work attitudes consist of two measures, job satisfaction (ten items) and organizational commitment (13 items), with good reliability and validity [6870]. Attitudes toward adopting EB strategies will be assessed using the Evidence-Based Practice Attitude Scale, which examines: (1) intuitive appeal of EB practices; (2) likelihood of adopting if required; (3) openness; and (4) perceived divergence between EB practices and current practice [71]. Leadership will be measured via the Multifactor Leadership Questionnaire (MLQ) [72]. The MLQ assesses transformational leadership, transactional leadership, passive/avoidant leadership (laissez-faire), and perceived outcomes of leadership (extra effort, effectiveness, satisfaction). The MLQ has good psychometric properties with Cronbach’s alphas ranging from 0.76 to 0.90 [41, 69]. School-community linkages will be measured through an LGBTQ-specific modification of the Collaborating with Community Subscale, an eight-item tool based on the Measure of School, Family, and Community Partnerships [73]. This tool rates school partnership practices for integrating resources and services from the community to strengthen school programs, learning, and well-being for LGBTQ students on a scale of 1 (not occurring) to 5 (extensively occurring) and offers insight into improvement areas and future directions. An adapted Assessment of Climate Embedding Mechanisms (ACEM) measure will elicit Likert scale ratings on how frequently each EB strategy is used, EB strategy adherence, and school leadership support [43]. In year 1, it will be completed by nurses and administrators to serve as a baseline assessment for each school.

School administrative data. Publicly-available administrative data will be used to assess faculty and staff retention, school truancy, graduation rates, drop-out rates, extracurricular student activities, and overall student characteristics, e.g., socioeconomic status and race/ethnicity, at participating schools.

Quantitative data analysis

Quantitative analysis of the web-based surveys will involve descriptive analyses, aggregated at the school level, for each subscale in the organizational web-based survey. Data will be compared across schools to determine areas of relative strength and weakness regarding factors known to impact implementation. Data will be shared with the IC IRTs in the form of feedback reports comparing each school’s scores on each subscale to the average scores across other schools. The raw data, feedback, and recommended changes will help ensure that the IRTs recognize barriers that could affect implementation and will provide impetus for the IRTs to enhance deficit areas. For example, if the data reveal that a school has a defensive climate, an IRT can then determine ways to work under this constraint or, alternatively, to improve the climate. The analysis of school administrative data will also involve descriptive analyses of staffing, faculty/staff retention, truancy, graduation and drop-out rates, extracurricular student activities, and overall student characteristics, which will be compared across schools to identify potential confounders.

Feedback reports

After completing qualitative and quantitative analyses, the research team will prepare summaries that describe issues, ideas, and concerns raised by the school nurses, administrators, and IRTs at the IC schools, implementation readiness at the school and staff levels, and the encompassing school context. These summaries will include contextual detail for nuanced implementation planning through the DAP and outline strengths and weaknesses in the school settings that may influence implementation and uptake of each of the EB strategies. No feedback will be provided to the CC schools until the final year of the project.

Preparation phase

The IRTs review data from the exploration phase in the preparation phase to determine (a) adaptations needed in the school context and its workforce to ensure uptake; and (b) how to accomplish such adaptations. The school-specific feedback reports made available to the IRTs will provide information on relative strengths and weaknesses and key areas for the IRTs to address when implementing the strategies. Each school nurse-led IRT will review their relevant report, analyzing the extent to which their schools possess the requisite conditions for implementing each strategy. The nurse champion will guide the IRT through a process for prioritizing and addressing recommendations [74], weighing each based on five dimensions (importance, cost, time, commitment, and feasibility). The IRT would then outline the steps needed to follow through with actions. If a potential action is deemed “infeasible” at that time, the nurse would encourage the IRT to consider more “workable,” short-term strategies to create conditions sufficient for implementation.

Coaching for IC schools begins in the preparation phase. Coaches will engage in monthly conference calls with each nurse-led IRT supplemented with bi-monthly visits to the IC schools to identify additional training needs and create Resource Guides and Referral Lists to support practices related to the EB strategies. The guides will contain written material about advancing EB strategies, and the lists will identify community health and mental health providers and resources for LGBTQ youth and their peers. Coaching staff have specialized expertise in team building, school climates, LGBTQ youth intervention, suicide prevention, and transgender populations. Their role is to improve school nurse and IRT performance through both training and technical assistance (including development and delivery of webinar content), enabling them to hone the knowledge and skills for meeting EB strategy implementation goals.

Implementation phase

Training with adaptation support begins in the implementation phase. Twenty nurse-led IRTs will annually implement or strengthen a minimum of two of the six EB strategies per year, building on lessons accumulated over time. Ongoing assessment will occur to optimize local adaptations to enhance implementation. Fidelity and adherence data will be collected and provided as feedback to coaches and IRTs.

The overall training will emphasize explicit inclusion and discussion of adaptation, i.e., why and what to adapt, when to seek guidance on adaptation, and how to use the coaches and IRT. Further augmentation may be accomplished by including materials and training in response to unique school level barriers or populations (e.g., lack of LGBTQ-inclusive anti-bullying policies or negative teacher attitudes about LGBTQ youth). School nurses will be provided with a one-day in-person training that coincides with an annual conference sponsored by the NM School Nurses Association. The training will cover leadership that supports effective EB strategy implementation. To prepare nurses to be strong change agents in schools and to negotiate interdisciplinary IRTs, adapted modules (e.g. development of positive IRT climates and organizational support for EB strategies) of an implementation leadership curriculum that features traditional didactics and role-play activities are included in the training. The modules will be reinforced through regularly scheduled coaching and annual in-person booster trainings for school nurses in the implementation phase [75]. The IRTs will also benefit from eight 1-h webinars on LGBTQ people, youth suicide prevention, and school safety and supports. Trainees will receive monetary incentives or continuing medical education (CME) credits.

Coaching in deployment of the strategies will occur during the implementation phase. Coaches will continue to organize monthly conference calls with each IRT and conduct bi-monthly visits to IC schools to provide technical assistance. These efforts are intended to enhance the likelihood that EB strategies are implemented in the IC schools, and will contribute to adjustments to school-specific implementation protocols, if warranted. Coaches will compile logs of interactions with nurses and IRTs, which will be analyzed for process evaluation.

Sustainment phase

During the sustainment phase, the IRT analyzes implementation successes and challenges, focusing on school nurse and member satisfaction, adherence and fidelity to action plans, use of EB strategies, and the NM YRRS outcomes over time.

Aim 2: conduct a cluster randomized controlled trial to assess whether LGB youth and peers in RLAS intervention schools report reduced suicidality, depression, substance use, and bullying, and increased safety compared to control schools

A cluster randomized controlled trial will assess whether LGBTQ students and their peers in RLAS intervention schools report reductions in suicidality, depression, substance use, bullying, and truancy related to safety concerns compared to those in usual care schools. Additional file 2 provides a study-specific checklist of information to include when reporting a cluster randomized trial. Additional file 3 contains a flow diagram of the phases of the randomized trial for the IC and CC schools. Randomization will take place in the exploration phase, as noted previously. Mahalanobis distance metric matching will be used to create pairs of similar schools [76], where one school in each pair will be randomly assigned to the IC.

The 2015 NM YRRS will offer baseline data for all schools and the 2017 and 2019 NM YRRS will yield follow-up data for the implementation and sustainment phases. From each school an average of 150 students complete the NM YRRS, for a sample size of 6000 per survey administration. Demographic data include gender, age, race/ethnicity, and sexual identity. Primary outcome measures assess suicide-related outcomes (“During the past 12 months, did you ever seriously consider attempting suicide?” “During the past 12 months, did you make a plan about how you would attempt suicide?” “During the past 12 months, how many times did you actually attempt suicide?”). Secondary measures assess depression (“During the past 12 months, did you ever feel so sad or hopeless almost every day for two weeks or more in a row that you stopped doing some usual activities?”), substance use (“During the past 30 days, on how many days did you have at least one drink of alcohol?” “During the past 30 days, how many times did you use marijuana?”), bullying (“During the past 12 months, have you ever been bullied on school property?”), and school safety (“During the past 30 days, on how many days did you not go to school because you felt you would be unsafe at school or on your way to or from school?”).

Analysis will examine five hypotheses using NM YRRS data: (1) IC schools will have a greater reduction in suicide-related outcomes than CC schools; (2) IC schools will have a greater reduction in depression than CC schools; (3) IC schools will have a greater reduction in substance use outcomes than CC schools; (4) IC schools will have a greater reduction in bullying than CC schools; and (5) IC schools will have a greater improvement in school safety than CC schools. Each analysis assesses these effects for the entire population and whether they are more pronounced for LGB students. The five outcomes predicted in hypotheses 1–5 will likely be more pronounced for LGB students, relative to other students. As the NM YRRS does not include items addressing gender identity, interviews and focus groups with school nurses, administrators, and IRT members will elucidate perceived impacts on transgender and intersex youth.

Data analysis

Students are nested within school by time period groups (e.g., 2015 observations of one school), and these repeated observations of a school are each nested within one school. Hence, all quantitative models examining hypotheses will use multilevel modeling (i.e., hierarchical linear modelling) to conservatively adjust estimates for variability arising in outcomes due to student (level 1) nesting within school by time groups (level 2) and these repeated observations at the school level being nested within schools (level 3). Thus, all models will be run as random intercept regressions. Sampling weight and design information will be used to appropriately weight cases in analyses via multilevel pseudo maximum likelihood methods [77, 78] and Mplus© [79]. Models for continuous variables will be run assuming normally distributed, continuous outcomes (unless descriptive analyses indicate zero-inflated Poisson models for count outcomes). Binary outcomes will be treated as having a binomial distribution and use a logit link function. Regression models (one for each measured outcome in all hypotheses) will regress the outcome on a dummy contrast representing study condition; a dummy contrast representing LGB; two contrasts representing linear and quadratic, i.e., u-shaped, change; the orthogonal interactions between condition and change contrasts; the orthogonal interactions between LGB, condition, and change contrasts; and all other component interactions not mentioned. These models represent whether (a) there was differential change as a function of condition and (b) whether this differential change was more pronounced among LGB students.

Aim 3: examine the individual, school, and community factors influencing implementation and outcomes

For the RLAS process evaluation, school health policies, practices, and community linkages, and the status of EB strategies will be assessed and compared across IC and CC groups. This will be done annually each spring via semi-structured interviews with all school administrators (n = 40) and nurses (n = 40) in years 1–4. To assess individual and school influences, the web-based surveys will be repeated with IC and CC groups each year. All qualitative interviews and surveys will follow the data collection and analysis procedures used in the preparation phase. Using these same qualitative analysis procedures, the content of coaching logs that describe in-depth implementation issues requiring technical assistance in the IC schools over the intervention period will also be assessed. Analysis of the web-based surveys aggregated across IC and CC schools will help identify and compare differences in the organizational characteristics of schools and administrator/nurse work attitudes over time.

Data collection and analysis

Many schools may already utilize one or more EB strategy; yet use of all six in accordance with CDC specifications is uncommon [40]. There is also a need to assess potential diffusion of the RLAS to CC schools. The EB strategy implementation in all 40 schools will be tracked using the adapted ACEM incorporated into the annual web-based surveys completed by nurses and administrators [43]. This instrument will yield data on frequency of EB strategy usage, EB strategy adherence, and school leadership support. These data will be supplemented with descriptive information from qualitative interviews concerning the extent to which the RLAS may stimulate changes in implementation, diffusion, and overall implementation status of each EB strategy at each school.

Implementation progress and fidelity for each EB strategy in IC schools will be determined via a modified version of the Stages of Implementation Completion (SIC) checklist, an eight-stage measure that will assess progress for phases of pre-implementation (exploration/preparation), implementation, and sustainment using activity completion dates and duration of activities and does not require additional effort from the school nurse champions and IRTs beyond participating in the usual implementation process [80]. The research team will complete the SIC by coding data from interviews, surveys, action plans, and coaching logs concerning markers or milestones. These data sources offer insight into individual, school, and community factors affecting progress. The SIC will be adapted to capture key stages of implementation progress for the six EB strategies. These may include the following: (1) engagement; (2) consideration of feasibility; (3) readiness planning; (4) staff trained; (5) fidelity monitoring process in place; (6) implementation begins; (7) ongoing implementation, coaching, and feedback; and (8) competency. The SIC scores will indicate the level of RLAS implementation per IC school over time, allowing categorization of the IC schools into “low” versus “high” implementation sites. These categories will be used to organize analyses of qualitative data to determine characteristics shared among IC schools assigned particular scores and to describe common barriers and facilitators that may have affected their implementation progress.

During the implementation and sustainment phases, the research team will conduct annual focus groups with the 20 IRTs to document their common and particular experiences identifying and advancing adaptation supports to facilitate use of EB strategies and their overall progress and satisfaction with the DAP. Each focus group will consist of at least five IRT members (including the school nurse), take place on site when IRTs regularly meet, and last between 90 to 120 min. Members will complete demographic forms (recording age, gender, sexual orientation, race/ethnicity, education, employment, socioeconomic status, religion, etc.) and a 10-min survey on individual perceptions and experiences related to the IRTs and the DAP. The focus group guides will consist of eight to ten open-ended questions. Participants will be asked to reflect on satisfaction, EB strategy implementation (including contextual relevance, feasibility, strategies to overcome barriers, and processes for forging and maintaining local linkages with health, mental health, and LGBTQ resources), and perceived outcomes on students, schools, and communities. These collective discussions [81, 82] will illuminate factors influencing the ability of IRTs to maintain their original action plans, reasons for possible divergence from these plans (e.g., community resistance) and subsequent adaptations (community education), and provide insight into possible changes to school climate, safety, support, and minority stress for LGBTQ students in IC schools. Using iterative coding, responses on perceived strengths, weaknesses, adaptations, and needed improvements will be clustered.

Convergences and divergences in quantitative and qualitative data will be identified [8386]. Analyses of survey, NM YRRS, ACEM, and SIC data will be summarized, and findings from different data sources will be compared to create a complete picture of DAP and EB strategy implementation in the IC schools over time. Results will be integrated through an inclusive process that values the input of all stakeholders (nurses, administrators, and IRT members). Data will be merged by the following: (a) linking qualitative and quantitative databases; and (b) embedding one within the other so that each plays a supportive role for the other. Results of each dataset will be placed side-by-side to examine: (1) convergence (do results provide the same answer to the same question, e.g., do interview data concur with NM YRRS data regarding impact of RLAS on suicide risk among LGBTQ students?); (2) expansion (are unanticipated findings of one dataset explained by another, e.g., can web-based survey data that suggest a disempowering school climate be explained by qualitative interview data?); and (3) complementarity (does embedding results of the qualitative analysis in the quantitative dataset help contextualize results, e.g., does it explain variability represented by confidence intervals or variance estimates in statistical analyses on “bullying on school grounds” and “fear-based bullying”?). Triangulated results will be shared annually with IRTs and schools.

Discussion

The RLAS keeps with national priorities to (a) improve school-based services for pediatric populations, (b) focus on LGBTQ youth mental health, and (c) revolutionize the role of nurses in U.S. healthcare. In addition to LGBTQ youth suicide, the conceptual framework and methods for this novel nurse-led intervention are applicable to addressing the health-related concerns of other pediatric populations in schools as well. Through its collaborative processes to refine, improve, and sustain EB strategies in these systems, the RLAS represents an innovative contribution to implementation science that also addresses a pressing public health challenge.

Abbreviations

ACEM: 

Assessment of Climate Embedding Mechanisms

CC: 

Control condition

CDC: 

Centers for Disease Control and Prevention

DAP: 

Dynamic Adaptation Process

EB: 

Evidence-based

GSA: 

Gay-Straight Alliance

IC: 

Intervention condition

IRT: 

Implementation Resource Team

LGB: 

Lesbian, gay, and bisexual

LGBTQ: 

Lesbian, gay, bisexual, transgender, and queer or questioning

MLQ: 

Multifactor Leadership Questionnaire

NM: 

New Mexico

RLAS: 

Implementing School Nursing Strategies to Reduce LGBTQ Adolescent Suicide

SIC: 

Stages of Implementation Completion

YRRS: 

Youth Risk and Resiliency Survey

Declarations

Acknowledgements

We are thankful for the support provided by the New Mexico School Nurses Association and the New Mexico Department of Health. We thank Patricia Hokanson for her generous assistance with formatting this manuscript.

Funding

This study is supported by Eunice Kennedy Shriver National Institute of Child Health and Human Development Grant R01HD083399.

Availability of data and materials

Not applicable.

Authors’ contributions

CEW is the co-principal investigator for the described study. She conceptualized and designed the study, drafted the manuscript, and approved the final version. AEG, co-investigator, contributed to the conceptualization and design of the study, drafted the manuscript, and approved the final version. MMR is the co-principal investigator for this study. She conceptualized and designed the study, revised the manuscript, and approved the final version. All authors read and approved the final manuscript.

Competing interests

CEW is on the Editorial Board and Gregory A. Aarons (a co-investigator of this study) is an Associate Editor of Implementation Science; all decisions regarding this paper were made by another editor. The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Ethics approval including consent to participate procedures were approved by the Pacific Institute for Research and Evaluation Institutional Review Board.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Behavioral Health Research Center of the Southwest, Pacific Institute for Research and Evaluation
(2)
Department of Psychiatry, University of California, San Diego
(3)
Child and Adolescent Services Research Center
(4)
Department of Pediatrics, University of New Mexico, MSC10 5590, 1 University of New Mexico

References

  1. U.S. Department of Health and Human Services Office of the Surgeon General, National Action Alliance for Suicide Prevention. 2012 national strategy for suicide prevention: goals and objectives for action. Washington, DC: U.S. Department of Health and Human Services; 2012. http://www.surgeongeneral.gov/library/reports/national-strategy-suicide-prevention/overview.pdf. Accessed 21 July 2016.Google Scholar
  2. Institute of Medicine. The health of lesbian, gay, bisexual, and transgender people: building a foundation for better understanding. Washington, DC: National Academies Press; 2011.Google Scholar
  3. Garofalo R, Wolf RC, Wissow LS, Woods ER, Goodman E. Sexual orientation and risk of suicide attempts among a representative sample of youth. Arch Pediatr Adolesc Med. 1999;153:487–93. http://archpedi.jamanetwork.com/article.aspx?articleid=346930.View ArticlePubMedGoogle Scholar
  4. Russell ST, Joyner K. Adolescent sexual orientation and suicide risk: evidence from a national study. Am J Public Health. 2001;91:1276–81. doi:https://doi.org/10.2105/AJPH.91.8.1276.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Stone DM, Luo F, Ouyang L, Lippy C, Hertz MF, Crosby AE. Sexual orientation and suicide ideation, plans, attempts, and medically serious attempts: evidence from local youth risk behavior surveys, 2001-2009. Am J Public Health. 2014;104:262–71. doi:https://doi.org/10.2105/AJPH.2013.301383.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Mereish EH, O’Cleirigh C, Bradford JB. Interrelationships between LGBT-based victimization, suicide, and substance use problems in a diverse sample of sexual and gender minorities. Psychol Health Med. 2013;19:1–13. doi:https://doi.org/10.1080/13548506.2013.780129.View ArticlePubMedGoogle Scholar
  7. Talley AE, Hughes TL, Aranda F, Birkett M, Marshal MP. Exploring alcohol-use behaviors among heterosexual and sexual minority adolescents: intersections with sex, age, and race/ethnicity. Am J Public Health. 2013;104:295–303. doi:https://doi.org/10.2105/AJPH.2013.301627.View ArticlePubMedGoogle Scholar
  8. Almeida J, Johnson RM, Corliss HL, Molnar BE, Azrael D. Emotional distress among LGBT youth: the influence of perceived discrimination based on sexual orientation. J Youth Adolesc. 2009;38:1001–14. doi:https://doi.org/10.1007/s10964-009-9397-9.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Birkett M, Espelage D, Koenig B. LGB and questioning students in schools: the moderating effects of homophobic bullying and school climate on negative outcomes. J Youth Adolesc. 2009;38:989–1000. doi:https://doi.org/10.1007/s10964-008-9389-1.View ArticlePubMedGoogle Scholar
  10. Bontempo DE, D’Augelli AR. Effects of at-school victimization and sexual orientation on lesbian, gay, or bisexual youths’ health risk behavior. J Adolesc Health. 2002;30:364–74. doi:https://doi.org/10.1016/S1054-139X(01)00415-3.View ArticlePubMedGoogle Scholar
  11. Jiang Y, Perry DK, Hesser JE. Adolescent suicide and health risk behaviors: Rhode Island’s 2007 Youth Risk Behavior Survey. Am J Prev Med. 2010;38:551–5. doi:https://doi.org/10.1016/j.amepre.2010.01.019.View ArticlePubMedGoogle Scholar
  12. Saewyc EM, Skay CL, Pettingell SL, Reis EA, Bearinger L, Resnick M, et al. Hazards of stigma: the sexual and physical abuse of gay, lesbian, and bisexual adolescents in the United States and Canada. Child Welfare. 2006;85:195–213.PubMedGoogle Scholar
  13. Shields JP, Whitaker K, Glassman J, Franks HM, Howard K. Impact of victimization on risk of suicide among lesbian, gay, and bisexual high school students in San Francisco. J Adolesc Health. 2012;50:418–20. doi:https://doi.org/10.1016/j.jadohealth.2011.07.009.View ArticlePubMedGoogle Scholar
  14. Kann L, Olsen EO, McManus T, Kinchen S, Chyen D, Harris WA, et al. Sexual identity, sex of sexual contacts, and health-risk behaviors among students in grades 9–12—youth risk behavior surveillance, selected sites, United States, 2001–2009. MMWR Survelliance Summaries. 2011;60:1–133. http://www.cdc.gov/mmwr/preview/mmwrhtml/ss6007a1.htm.Google Scholar
  15. Faulkner AH, Cranston K. Correlates of same-sex sexual behavior in a random sample of Massachusetts high school students. Am J Public Health. 1998;88:262–6. doi:https://doi.org/10.2105/AJPH.88.2.262.View ArticlePubMedPubMed CentralGoogle Scholar
  16. Garofalo R, Wolf RC, Kessel S, Palfrey SJ, Durant RH. The association between health risk behaviors and sexual orientation among a school-based sample of adolescents. Pediatrics. 1998;101:895–902. doi:https://doi.org/10.1542/peds.101.5.895.View ArticlePubMedGoogle Scholar
  17. Paul JP, Catania J, Pollack L, Moskowitz J, Canchola J, Mills T, et al. Suicide attempts among gay and bisexual men: lifetime prevalence and antecedents. Am J Public Health. 2002;92:1338–45. doi:https://doi.org/10.2105/AJPH.92.8.1338.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Meyer IH, Dietrich J, Schwartz S. Lifetime prevalence of mental disorders and suicide attempts in diverse lesbian, gay, and bisexual populations. Am J Public Health. 2008;98:1004–6. doi:https://doi.org/10.2105/AJPH.2006.096826.View ArticlePubMedPubMed CentralGoogle Scholar
  19. Grant JM, Mottet LA, Tanis J, Harrison J, Herman JL, Keisling M. Injustice at every turn: a report of the National Transgender Discrimination Survey. Washington, DC: National Center for Transgender Equality and National Gay and Lesbian Task Force; 2011. http://www.thetaskforce.org/static_html/downloads/reports/reports/ntds_full.pdf. Accessed 21 July 2016.Google Scholar
  20. Clements-Nolle K, Marx R, Katz M. Attempted suicide among transgender persons: the influence of gender-based discrimination and victimization. J Homosex. 2006;51:53–69. doi:https://doi.org/10.1300/J082v51n03_04.View ArticlePubMedGoogle Scholar
  21. Grossman AH, D’Augelli AR. Transgender youth and life-threatening behaviors. Suicide Life Threat Behav. 2007;37:527–37. doi:https://doi.org/10.1521/suli.2007.37.5.527.View ArticlePubMedGoogle Scholar
  22. Spack NP, Edwards-Leeper L, Feldman HA, Leibowitz S, Mandel F, Diamond DA, et al. Children and adolescents with gender identity disorder referred to a pediatric medical center. Pediatrics. 2012;129:418–25. doi:https://doi.org/10.1542/peds.2011-0907. http://pediatrics.aappublications.org/content/early/2012/02/15/peds.2011-0907.abstract.View ArticlePubMedGoogle Scholar
  23. Rowe C, Santos GM, McFarland W, Wilson EC. Prevalence and correlates of substance use among transfemale youth ages 16–24 years in the San Francisco Bay area. Drug Alcohol Depend. 2015;147:160–6. doi:https://doi.org/10.1016/j.drugalcdep.2014.11.023.View ArticlePubMedGoogle Scholar
  24. Keuroghlian AS, Shtasel D, Bassuk EL. Out on the street: a public health and policy agenda for lesbian, gay, bisexual, and transgender youth who are homeless. Am J Orthopsychiatry. 2014;84:66–72. doi:https://doi.org/10.1037/h0098852.View ArticlePubMedPubMed CentralGoogle Scholar
  25. Kosciw JG, Diaz EM, Greytak EA. 2007 National School Climate Survey: the experiences of lesbian gay, bisexual and transgender youth in our nation’s schools. New York, NY: Gay, Lesbian and Straight Education Network; 2008. https://www.glsen.org/download/file/NDIyMg==. Accessed 21 July 2016.Google Scholar
  26. Meyer IH. Minority stress and mental health in gay men. J Health Soc Behav. 1995;36:38–56.View ArticlePubMedGoogle Scholar
  27. Kosciw JG, Greytak EA, Diaz EM. Who, what, where, when, and why: demographic and ecological factors contributing to hostile school climate for lesbian, gay, bisexual, and transgender youth. J Youth Adolesc. 2009;38:976–88. doi:https://doi.org/10.1007/s10964-009-9412-1.View ArticlePubMedGoogle Scholar
  28. D’Augelli AR. Coming out, visibility, and creating change: empowering lesbian, gay, and bisexual people in a rural university community. Am J Community Psychol. 2006;37:203–10. doi:https://doi.org/10.1007/s10464-006-9043-6.View ArticlePubMedGoogle Scholar
  29. Eisenberg ME, Resnick MD. Suicidality among gay, lesbian and bisexual youth: the role of protective factors. J Adolesc Health. 2006;39:662–8. doi:https://doi.org/10.1016/j.jadohealth.2006.04.024.View ArticlePubMedGoogle Scholar
  30. Walls NE, Kane SB, Wisneski H. Gay-straight alliances and school experiences of sexual minority youth. Youth and Society. 2010;41:307–32. doi:https://doi.org/10.1177/0044118X09334957.View ArticleGoogle Scholar
  31. Walls NE, Freedenthal S, Wisneski H. Suicidal ideation and attempts among sexual minority youths receiving social services. Soc Work. 2008;53:21–9. doi:https://doi.org/10.1093/sw/53.1.21.View ArticlePubMedGoogle Scholar
  32. Goodenow C, Szalacha L, Westheimer K. School support groups, other school factors, and the safety of sexual minority adolescents. Psychol Sch. 2006;43:573–89. doi:https://doi.org/10.1002/pits.20173.View ArticleGoogle Scholar
  33. Poteat VP, Sinclair KO, DiGiovanni CD, Koenig BW, Russell ST. Gay-straight alliances are associated with student health: a multischool comparison of LGBTQ and heterosexual youth. J Res Adolesc. 2013;23:319–30. doi:https://doi.org/10.1111/j.1532-7795.2012.00832.x.View ArticleGoogle Scholar
  34. Toomey RB, Ryan C, Diaz RM, Russell ST. High school Gay-Straight Alliances (GSAs) and young adult well-being: an examination of GSA presence, participation, and perceived effectiveness. Appl Dev Sci. 2011;15:175–85. doi:https://doi.org/10.1080/10888691.2011.607378.View ArticlePubMedPubMed CentralGoogle Scholar
  35. O’Shaughnessy M, Russell S, Heck K, Calhoun C, Laub C. Safe place to learn: consequences of harassment based on actual or perceived sexual orientation and gender non-conformity and steps for making schools safer. San Francisco, CA: California Safe School Coalition; 2004. https://gsanetwork.org/files/resources/SafePlacetoLearn.pdf. Accessed 21 Jul 2016.Google Scholar
  36. Hatzenbuehler ML, Keyes KM. Inclusive anti-bullying policies and reduced risk of suicide attempts in lesbian and gay youth. J Adolesc Health. 2013;53:S21–6. doi:https://doi.org/10.1016/j.jadohealth.2012.08.010.View ArticlePubMedPubMed CentralGoogle Scholar
  37. Hatzenbuehler ML, Birkett M, Van WA, Meyer IH. Protective school climates and reduced risk for suicide ideation in sexual minority youths. Am J Public Health. 2014;104:279–86. doi:https://doi.org/10.2105/AJPH.2013.301508.View ArticlePubMedPubMed CentralGoogle Scholar
  38. Hertz MF, Donato I, Wright J. Bullying and suicide: a public health approach. J Adolesc Health. 2013;53:S1–3. doi:https://doi.org/10.1016/j.jadohealth.2013.05.002.View ArticlePubMedPubMed CentralGoogle Scholar
  39. Centers for Disease Control and Prevention. School connectedness: strategies for increasing protective factors among youth. Atlanta, GA: U.S. Department of Health and Human Services; 2009. http://www.cdc.gov/healthyyouth/protective/pdf/connectedness.pdf. Accessed 21 July 2016.Google Scholar
  40. Demissie Z, Brener ND, McManus T, Shanklin SL, Hawkins J, Kann L. School health profiles 2012: characteristics of health programs among secondary schools. Atlanta, GA: Centers for Disease Control and Prevention; 2013. http://www.cdc.gov/healthyyouth/profiles/2012/profiles_report.pdf. Accessed 21 July 2016.
  41. Aarons GA, Sawitzky AC. Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Adm Policy Ment Health. 2006;33:289–301. doi:https://doi.org/10.1007/s10488-006-0039-1.View ArticlePubMedPubMed CentralGoogle Scholar
  42. Aarons GA, Glisson C, Green PD, Hoagwood K, Kelleher KJ, Landsverk JA, et al. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implement Sci. 2012;7:56. doi:https://doi.org/10.1186/1748-5908-7-56. http://implementationscience.biomedcentral.com/articles/https://doi.org/10.1186/1748-5908-7-56.View ArticlePubMedPubMed CentralGoogle Scholar
  43. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255–74. doi:https://doi.org/10.1146/annurev-publhealth-032013-182447.View ArticlePubMedPubMed CentralGoogle Scholar
  44. Aarons GA. Transformational and transactional leadership: association with attitudes toward evidence-based practice. Psychiatr Serv. 2006;57:1162–9. doi:https://doi.org/10.1176/appi.ps.57.8.1162.View ArticlePubMedPubMed CentralGoogle Scholar
  45. Aarons GA. Clinician personality and attitudes toward adoption of empirically supported treatment. Presented at the 15th International Conference on Mental Health Services Research: Evidence in Mental Health Services Research: What Types, How Much, and Then What? Washington, DC: National Institute of Mental Health; 2002.Google Scholar
  46. Aarons GA, Sommerfeld DH, Walrath-Greene CM. Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implement Sci. 2009;4:83. doi:https://doi.org/10.1186/1748-5908-4-83.View ArticlePubMedPubMed CentralGoogle Scholar
  47. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38:4–23. doi:https://doi.org/10.1007/s10488-010-0327-7.View ArticlePubMedGoogle Scholar
  48. Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, et al. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implement Sci. 2012;7:32. doi:https://doi.org/10.1186/1748-5908-7-32. http://www.implementationscience.com/content/7/1/32.View ArticlePubMedPubMed CentralGoogle Scholar
  49. Stephan SH, Sugai G, Lever N, Connors E. Strategies for integrating mental health into schools via a multitiered system of support. Child Adolesc Psychiatr Clin N Am. 2015;24:211–31. doi:https://doi.org/10.1016/j.chc.2014.12.002.View ArticlePubMedGoogle Scholar
  50. Institute of Medicine. The future of nursing: leading change, advancing health. Washington, DC: The National Academies Press; 2011.Google Scholar
  51. U.S. Census Bureau American FactFinder. American Community Survey selected population profile in the United States 2009–2011: 3-year estimates. 2012. http://factfinder2.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_11_3YR_S0201&prodType=table. Accessed 21 July 2016.Google Scholar
  52. U.S. Health Resources and Services Administration. Health professional shortage areas: medically underserved areas/medically underserved populations. 2016. http://muafind.hrsa.gov/. Accessed 21 July 2016.
  53. New Mexico Department of Health. State of New Mexico Comprehensive Strategic Health Plan 2006. Santa Fe, NM: State of New Mexico Department of Health, New Mexico Health Policy Commission; 2012. http://nmcompplan.pbworks.com/f/NMCSHP+Final.pdf.Google Scholar
  54. New Mexico Department of Health. The state of health in New Mexico 2011. Santa Fe, NM: Author; 2012. http://www.astho.org/accreditation/new-mexico-state-health-assessment/. Accessed 21 Jul 2016.Google Scholar
  55. U.S. Census Bureau. American Fact Finder. n.d. http://factfinder2.census.gov/faces/nav/jsf/pages/index.xhtml. Accessed 21 July 2016.
  56. New Mexico Department of Health. New Mexico Substance Abuse Epidemiology Profile. Santa Fe, NM: New Mexico Department of Health Epidemiology and Response Division; 2013. https://nmhealth.org/about/erd/ibeb/sap/. Accessed 21 July 2016.Google Scholar
  57. New Mexico Department of Health. The state of health in New Mexico, 2013. Santa Fe, NM: Author; 2013. http://nmhealth.org/publication/view/data/160/. Accessed 21 July 2016.Google Scholar
  58. Kann L, Kinchen S, Shanklin SL, Flint KH, Hawkins J, Harris WA, et al. Youth Risk Behavior Surveillance—United States, 2013. MMWR Survelliance Summaries. 2014;63:1–168. http://www.cdc.gov/mmwr/pdf/ss/ss6304.pdf.Google Scholar
  59. Fitzgerald C, Reno J, Green D, Penaloza LJ. New Mexico Youth Risk and Resiliency 2013 Survey results report: mental health and related behaviors. Santa Fe, NM: Epidemiology and Response Division, New Mexico Department of Health, School and Family Support Bureau, New Mexico Public Education Department, and the University of New Mexico Prevention Research Center; 2015. http://www.youthrisk.org/pdf/YRRS_2013_MentalHealthReport.pdf. Accessed 21 July 2016.Google Scholar
  60. New Mexico Department of Health and the Public Education Department, University of New Mexico Prevention Research Center. 2013 NM-YRRS results: sexual minority youth. YRRS Connections. 2016;3:2. http://youthrisk.org/pdf/YRRS_Connections_13.pdf.Google Scholar
  61. Padilla J. Overview of New Mexico LGBT health data for the Legislative Health and Human Service Committee. Santa Fe, NM: New Mexico Department of Health; 2014. http://www.nmlegis.gov/lcs/handouts/LHHS%20112414%20Item%205%20James%20Padilla,%20Epidemiologist%20DOH,%20Overview%20of%20NM%20LGBT%20Health%20Data.pdf. Accessed 21 Jul 2016.Google Scholar
  62. Aarons GA, Sommerfeld DH. Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Psy. 2012;51:423–31. doi:https://doi.org/10.1016/j.jaac.2012.01.018.View ArticleGoogle Scholar
  63. QSR International. NVivo qualitative data analysis software (version 11). 2016.Google Scholar
  64. Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. Aldine de Gruyter: New York, NY; 1967.Google Scholar
  65. Corbin J, Strauss A. Basics of qualitative research: techniques and procedures for developing grounded theory (3rd ed). Thousand Oaks, CA: Sage Publications; 2008.View ArticleGoogle Scholar
  66. Herek GM. Attitudes toward lesbians and gay men scale. In: Davis CM, Yaber WL, Bauserman RL, Schreer G, Davis SL, editors. Handbook of sexuality-related measures. Thousand Oaks, CA: Sage Publications, Inc; 1998. p. 392–3.Google Scholar
  67. Walch SE, Ngamake ST, Francisco J, Stitt RL, Shingler KA. The attitudes toward transgendered individuals scale: psychometric properties. Arch Sex Behav. 2012;41:1283–91. doi:https://doi.org/10.1007/s10508-012-9995-6.View ArticlePubMedGoogle Scholar
  68. Glisson C. The organizational context of children’s mental health services. Clin Child Fam Psychol Rev. 2002;5:233–53.View ArticlePubMedGoogle Scholar
  69. Aarons GA, Sawitzky AC. Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychol Serv. 2006;3:61–72. doi:https://doi.org/10.1037/1541-1559.3.1.61.View ArticlePubMedPubMed CentralGoogle Scholar
  70. Aarons GA, Fettes DL, Flores Jr LE, Sommerfeld DH. Evidence-based practice implementation and staff emotional exhaustion in children’s services. Behav Res Ther. 2009;47:954–60. doi:https://doi.org/10.1016/j.brat.2009.07.006.View ArticlePubMedPubMed CentralGoogle Scholar
  71. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the evidence-based practice attitude scale (EBPAS). Ment Health Serv Res. 2004;6:61–74. doi:https://doi.org/10.1023/B%3AMHSR.0000024351.12294.65.View ArticlePubMedPubMed CentralGoogle Scholar
  72. Bass BM, Avolio BJ. Improving organizational effectiveness through transformational leadership. Thousand Oaks, CA: Sage Publications; 1994.Google Scholar
  73. Salinas KC, Epstein JL, Sanders MG, Davis D, Douglas I. Measure of school, family, and community partnership. Baltimore, MD & Portland, OR: Johns Hopkins University & Northwest Regional Educational Laboratory; 1999.Google Scholar
  74. Centers for Disease Control and Prevention. School Health Index: a self-assessment and planning guide. Middle school/high school version. Atlanta, GA: Author; 2012. http://www.cdc.gov/healthyyouth/shi/pdf/MiddleHigh.pdf. Accessed 21 Jul 2016.Google Scholar
  75. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implement Sci. 2015;10. http://dx.doi.org/https://doi.org/10.1186/s13012-014-0192-y.
  76. Rubin DB. Bias reduction using Mahalanobis-metric matching. Biometrics. 1980;36:293–8. http://www.jstor.org/stable/2529981.View ArticleGoogle Scholar
  77. Asparouhov T. General multilevel modeling with sampling weights. Commun Stat Theory Methods. 2006;35:439–60. doi:https://doi.org/10.1080/03610920500476598.View ArticleGoogle Scholar
  78. Asparouhov T, Muthén BO. Multilevel modeling of complex survey data. Presented at the Joint Statistical Meeting of the American Statistical Association: Seattle, WA; 2006. https://www.statmodel.com/download/SurveyJSM1.pdf
  79. Muthen LK, Muthen BO. Mplus user’s guide 1998–2008. Los Angeles, CA: Muthen & Muthen; 2008.Google Scholar
  80. Chamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: the stages of implementation completion (SIC). Implement Sci. 2011;6:116. doi:https://doi.org/10.1186/1748-5908-6-116.View ArticlePubMedPubMed CentralGoogle Scholar
  81. Patton MQ. Qualitative research and evaluation methods (3rd ed). Thousand Oaks, CA: Sage Publications; 2002.Google Scholar
  82. Krueger RA, Casey MA. Focus groups: a practical guide for applied research (4th ed). Thousand Oaks, CA: Sage; 2009.Google Scholar
  83. Aarons GA, Fettes DL, Sommerfeld DH, Palinkas LA. Mixed methods for implementation research: application to evidence-based practice implementation and staff turnover in community-based organizations providing child welfare services. Child Maltreat. 2012;17:67–79. doi:https://doi.org/10.1177/1077559511426908.View ArticlePubMedGoogle Scholar
  84. Semansky RM, Goodkind J, Sommerfeld DH, Willging CE. Culturally competent services within a statewide behavioral healthcare transformation: a mixed-method assessment. J Comm Psychol. 2013;41:378–93. doi:https://doi.org/10.1002/jcop.21544.View ArticleGoogle Scholar
  85. Willging CE, Sommerfeld D, Aarons GA, Waitzkin H. The effects of behavioral health reform on safety-net institutions: a mixed-method assessment in a rural state. Adm Policy Ment Health. 2013;41:276–91. doi:https://doi.org/10.1007/s10488-012-0465-1.View ArticleGoogle Scholar
  86. Creswell JW, Klassen AC, Plano Clark VL, Smith KC. Best practices for mixed methods research in health sciences. Bethesda, MD: National Institutes of Health, Office of Behavioral and Social Sciences Research; 2011. https://obssr.od.nih.gov/training/mixed-methods-research/. Accessed 21 July 2016.View ArticleGoogle Scholar

Copyright

© The Author(s). 2016

Advertisement