- Open Access
Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS)
Implementation Science volume 17, Article number: 48 (2022)
Strategic implementation leadership is a critical determinant of successful implementation, hypothesized to create a more supportive implementation climate conducive to the adoption and use of evidence-based practices. Implementation leadership behaviors may vary significantly across contexts, necessitating studies that examine the validity of established measurement tools in novel health service delivery sectors. The education sector is the most common site for delivering mental health services to children and adolescents in the USA, but research focused on implementation leadership in schools is in the early phases, and there is a need for adaptation and expansion of instruments in order to tailor to the school context. The current study adapted and validated the School Implementation Leadership Scale (SILS) (based on the Implementation Leadership Scale) in a sample of elementary school personnel from six school districts who were implementing one of two well-established prevention programs for supporting children’s mental health.
Participants were 441 public school teachers from 52 elementary schools in the Midwest and West Coast of the USA. Participants completed a survey that contained: (1) an adapted and expanded version of the SILS with additional items generated for four existing subscales as well as three new subscales (communication, vision/mission, and availability), and (2) additional tools to evaluate convergent and divergent validity (i.e., measures of general/molar leadership and teaching attitudes). Data underwent (1) examination of item characteristic curves to reduce items and ensure a pragmatic instrument, (2) confirmatory factor analyses to establish structural validity, and (3) evaluation of convergent and divergent validity.
Item reduction analyses resulted in seven subscales of three items each. Results indicated acceptable fit for a seven-factor structural model (CFI = .995, TLI = .99, RMSEA = .07, SRMR = 0.02). Second-order factor loadings were high (λ = .89 to .96), suggesting that the SILS subscales comprise a higher-order implementation leadership factor. All subscales demonstrated good inter-item reliability (α = .91–.96). Convergent and divergent validity results were generally as hypothesized, with moderate to high correlations between SILS subscales and general leadership, moderate correlations with teaching attitudes, and low correlations with school demographics.
Overall, results provided strong structural, convergent, and divergent validity evidence for the 21-item, 7-factor SILS instrument. Implications for the measurement of implementation leadership in schools are discussed, as well as strategies to support leaders to enhance their strategic behaviors related to the implementation of mental health prevention programs (e.g., adaptation of existing leadership-focused implementation strategies).
Measurement of organizational influences on successful implementation of evidence-based practices (EBP) is a clear priority for implementation research, given consistent findings regarding their impact on implementation outcomes [1,2,3]. Organizational structures and processes—such as leadership behaviors—can vary widely across contexts, necessitating studies that examine the application and validity of established measurement tools in novel health service delivery sectors. Among these, the education sector is the most common site for delivering prevention and intervention services targeting child and adolescent mental health . However, EBP are inconsistently adopted, delivered with fidelity, or sustained over time in schools [5, 6]. Barriers related to support from school leadership are commonly cited as reasons why EBP implementation fails in the school setting [7, 8]. Despite the availability of a psychometrically sound measure of implementation leadership (Implementation Leadership Scale [ILS])  in other health and mental health delivery contexts, no research has systematically validated an instrument to assess strategic implementation leadership in schools.
Organizational leadership for implementation
Research suggests that the inner context—characteristics of the immediate organizational setting in which implementation occurs [10, 11]—is critical to the successful implementation of EBP [12,13,14,15,16]. As a key driver of inner context organizational functioning, the skills and behaviors of leaders are frequently highlighted across organizational types, including in schools [17,18,19]. Research on organizational leadership can take a general (i.e., molar) approach, indicating the typical behaviors performed when interacting with their subordinates (e.g., transformational leadership ), or a strategic approach, focusing specifically on those leadership behaviors that support or inhibit a strategic goal (such as customer service  or safety ). Leaders who accomplish their strategic goals engage in routine interactions and communications with staff that support those goals, protect time during meetings to discuss related content, hold staff accountable, and provide performance-based feedback related to strategic goals [23, 24]. Meta-analytic findings indicate that such approaches are useful and often have strong relationships with strategic climates (e.g., service climate) and quality outcomes, such as customer satisfaction and financial outcomes , because of their focus on the specific aspects of leadership that are most relevant to and predictive of particular outcomes of interest (e.g., high-quality customer service).
Within implementation research, strategic implementation leadership has been hypothesized to be a critical precursor of strategic implementation climate  and to be more directly related to implementation outcomes (e.g., feasibility, fidelity, reach) than general/molar leadership. Strategic implementation leadership is made up of behaviors (e.g., strategic communication and direct support) that serve an embedding function (i.e., what leaders do to achieve a strategic objective) for new practices and programs . Rigorous assessment of implementation leadership is critical to inform implementation research and practice in this domain as recent studies have shown that implementation leadership is a malleable organizational characteristic . Aarons, Ehrhart, and Farahnak  developed the Implementation Leadership Scale (ILS) to capture strategic leadership behaviors that drive successful EBP implementation. Designed to be brief and pragmatic , the original ILS contained 12 items loading onto 4 subscales: Proactive Leadership (anticipating and addressing implementation challenges), Knowledgeable Leadership (deep understanding of EBP and implementation issues), Supportive Leadership (support for EBP adoption/use), and Perseverant Leadership (consistent and responsive to challenges). Subscale internal consistencies ranged from 0.95 to 0.98. Although the ILS was originally developed in specialty mental health, the authors called for research examining its utility in other service settings . Subsequent studies have begun this work, including successful validation of the instrument in child welfare , substance use treatment , and acute care ; as well as translation into other languages (e.g., [33, 34]) and adaptation to measure sustainment-focused leadership . The ILS has recently been identified as one of only a few leadership instruments with adequate evidence for its use in healthcare .
School-based mental health services
A wealth of research has highlighted the central role that the education sector plays in mental health services for children and adolescents. Internationally, school-based mental health programs have grown markedly over the past two decades [37,38,39,40]. In the USA, studies consistently indicate that the education sector is the most heavily accessed youth mental health service setting, responsible for 50–80% of all mental healthcare [4, 41,42,43,44,45]. Increasingly, school-based mental health programs are organized via a multi-tiered system of support (MTSS) framework. MTSS is grounded in the public health model of prevention and conceptualized as a three-tier framework that provides a data-driven continuum of intensifying supports [46, 47]. The foundation of MTSS is the universal level in which EBPs are delivered to all students to prevent the emergence of student mental health problems [48, 49]. Unfortunately, despite strong evidence for a wide variety of universal prevention programs for student mental health, findings suggest that inconsistent implementation is common and that problems with delivery attenuate their impact [50,51,52,53]. Organizational leadership is a critical determinant of the successful implementation of these programs.
Implementation leadership in schools
While implementation research in schools has recently increased, the contributions of leadership and other organizational influences on successful implementation of EBPs in that context remain understudied . Although leadership models vary widely across schools [54, 55], all schools have a site-based principal or headmaster who has the authority to make decisions and hold staff accountable for achieving implementation objectives. Prior organizational research in the education sector has focused on principal managerial or instructional leadership  and has shown positive leadership to increase staff productivity , and established its links to school climate and student outcomes . Related to mental health, research has established that principal buy-in and school leadership are critically important to the implementation of universal prevention programs [58,59,60,61]. Other studies are examining leadership in relation to implementation of interventions for autism spectrum disorders [8, 17, 18, 62, 63] and characterizing leadership styles conducive to implementation .
Despite strong interest, research focused on implementation leadership in schools has been significantly hampered by instrumentation limitations. Existing measures of principal leadership (e.g., [64, 65]) tend to assess global leadership qualities and are too broad to track the specific leadership behaviors most associated with the adoption, high-fidelity use, and sustainment of EBPs. Many specific programs have developed their own “readiness” assessments, but they are neither specific to leadership nor generalizable across EBPs. Pragmatic and brief instruments are needed to assess implementation leadership in schools. In pursuit of this objective, Lyon and colleagues conducted an initial adaptation of the ILS for use with mental and behavioral health consultants in the education sector . These consultants functioned primarily as implementation intermediaries to support the installation of school-based programs. Items underwent minor surface-level adaptations (e.g., replacing “agency” with “school,” “clinician” with “school personnel,” etc.), but efforts were made to preserve the integrity of the original items and constructs. Findings provided strong support for the original ILS factor structure with this new population. However, because it was administered to implementation consultants/intermediaries, the instrument was not designed for teachers, the primary deliverers of mental health prevention programming in schools. It also did not include the full range of relevant school leader behaviors that can support implementation. As a result, the ILS was further revised through a series of focus groups with educators (central administrators, principals, teachers ;) and pilot tests to understand the applicability and conceptual boundaries of the concept of implementation leadership and ensure the instrument’s alignment with the realities of educational settings implementing universal prevention programs. This process produced an initial School Implementation Leadership Scale (SILS, see “Method” section), which was refined and tested in the current study.
In light of the need for reliable, valid, and pragmatic instruments to measure strategic implementation leadership in the education sector, the current study administered an adapted SILS to teachers who were implementing one of two different universal EBPs (Schoolwide Positive Behavioral Interventions and Supports (SWPBIS ;) and Promoting Alternative Thinking Strategies (PATHS ;) across six elementary school districts. The evidence for universal prevention programming in mental health is strongest at the elementary level, making elementary schools a priority for increasing the public health impact of evidence-based mental health prevention practices. The resulting data were used to (1) conduct item reduction analyses to ensure a pragmatic instrument , (2) complete a confirmatory factor analysis to establish structural validity, and (3) evaluate convergent and divergent validity with measures of molar leadership, staff attitudes, and school demographic variables. We hypothesized that the SILS would demonstrate moderate to high associations with molar leadership (convergent and divergent validity) and lower associations with attitudes and demographics (divergent validity).
Setting and participants
Schools implementing one of two evidence-based interventions (n = 39 SW-PBIS; n = 13 PATHS) were eligible and recruited for participation, resulting in 441 teachers from 52 elementary schools in 6 school districts in Washington, Ohio, and Illinois. The average racial/ethnic and socioeconomic composition of students across schools was 66% Non-White (range 21 to 100%) and 57% low-income status (range 4 to 100%), respectively. Across all participating schools, an average of 88% of teachers who were contacted to participate (n = 500) completed the study’s online survey (see “Procedures” section below).
On average, 9 teachers per school were recruited to complete measures. Most teachers were female, had at least a master’s degree, had an average of 11.6 years of experience, and were predominately White (see Table 1 for complete demographic information). The number of participants included in analyses was sometimes less than 441 due to missing data (< 5% overall).
This study was part of a large-scale, federally funded measure adaptation and development project with the aim of creating school-adapted organizational assessments. Prior to conducting the current study, the original SILS was adapted for use in schools through (1) input from research and practice experts during a structured in-person convening and (2) mixed-methods focus group sessions with key educator stakeholder groups (central district administrators, principals, teachers) . Adaptations included changing item wording to ensure construct equivalence for the target respondents (i.e., school-based practitioners) and deleting or expanding items and item content to ensure contextual appropriateness to schools . An effort was made to preserve the integrity of the original items and constructs as much as possible . Expansion included developing items to address additional constructs in subscales focused on leaders’ (a) communication, (b) organizational vision/mission, and (c) availability to support EBP implementation.
Human subjects’ approval was obtained from the University of Washington Institutional Review Board and participating school districts’ research and evaluation departments, when applicable. Study investigators first reached out to school district central administrators to discuss the project and secure participation. School recruitment was done in collaboration with central administrators who identified eligible school buildings and facilitated the distribution of information surrounding project benefits and data collection procedures to site-based administrators. Teachers from each school (n = 4–10) were then recruited by school administrators or a site-based liaison who typically presented the opportunity either during standing staff meetings or via email communications. Contact information for interested teachers was provided to research staff and used to establish and maintain project communications (e.g., sending survey links).
Data were collected between September and November of the 2017 academic year. In November, teachers were sent an initial email to provide a project overview, obtain informed consent, and provide a link to the online survey. Upon receiving the initial email, teachers had one-month to complete the online survey. Weekly email reminders were sent to increase the response rates at each school.
School Implementation Leadership Scales (SILS)
The original ILS  and original SILS adaptation  are 12-item instruments developed to assess strategic leadership for EBP implementation. All ILS items are scored on a four-point Likert scale ranging from 0 (“not at all”) to 4 (“to a very great extent”). Both versions have previously supported a factor structure with four first-order factors (proactive leadership, knowledgeable leadership, supportive leadership, perseverant leadership)—each with three items—loading onto an overarching implementation leadership latent factor [26, 66]. As described above, the present study adapted the original SILS based on expert feedback, adding items for three new subscales (communication, vision, available). Eighteen additional items were initially developed for the new subscales and to augment the existing subscales with contextually appropriate items. This resulted in an initial 30-item revised SILS measure. Item reduction procedures along with reliability and validity data are reported in the Results. In addition, two versions of the adapted SILS were created, which included different referents. In one version, items referenced EBP generally (e.g., “Our principal is knowledgeable about evidence-based practice”). In the other, items referenced the specific EBP being implemented (e.g., “Our principal is knowledgeable about SW-PBIS”). Multigroup models were examined to determine whether the underlying factor structure was invariant across these two referents (see “Results” section).
Multifactor Leadership Questionnaire (MLQ)
The MLQ, a widely used measure of organizational leadership , was included to assess SILS convergent validity. Only the transformational and transactional leadership subscales were used in the present study. Transformational leadership is measured via five subscales: intellectual stimulation, inspirational motivation, individualized consideration, idealized behaviors, and idealized attributes. Two subscales comprise transactional leadership (contingent reward, management-by-exception active). The MLQ and its subscales have previously demonstrated strong psychometric properties [72, 73]. Internal consistency for subscales and scale scores in the current study were acceptable and as follows: intellectual stimulation (α = .88), inspirational motivation (α = .89), individualized consideration (α = .80), idealized behaviors (α = .84), idealized attributes (α = .84), transformational leadership total score (α = .91), contingent rewards (α = .78), management-by-exception active (α = .79).
Public School Teacher Questionnaire
The Public School Teacher Questionnaire (PSTQ), included for decades as part of the Schools and Staff Survey conducted by the National Center on Educational Statistics , was prioritized in the present study for purposes of divergent validity as a measure of teachers’ general attitudes toward teaching. Respondents used a 4-point Likert scale ranging from strongly disagree to strongly agree to rate 9 items that assess different attitudes toward the teaching profession (e.g., “The teaching profession is something that I enjoy and feel competent doing”). The PTSQ has demonstrated acceptable psychometric properties in extant research , as well as in the present study (α = .81).
Data analytic approach
Several methodological approaches were employed to establish construct validity. Although this study did not have sufficient higher-level units (i.e., schools) to examine a multi-level confirmatory factor analysis (CFA), ICCs for SILS subscales provide evidence that 30–45% of the variability existed between schools, which is the level at which the construct theoretically resides. A series of CFAs were examined in Mplus v8  specifying robust standard errors to account for clustering of teachers within schools and weighted least squares means and variances (WLSMV) estimation with delta parameterization for the order-categorical scale items. Model fit was assessed using several indices including chi-square test statistics, comparative fit index (CFI) , the Tucker-Lewis index (TLI) , the root mean square error of approximation (RMSEA) [79, 80], and the standardized root mean square residual (SRMR) . CFI and TLI values greater than .95, RMSEA less than or equal to .05, and SRMR less that or equal to .08 indicate a model well fit to the data. Standardized factor loadings (β) less than .55 were considered low and flagged for further examination .
Two measurement models were examined. The first included only first-order factors modeling exogenous, but correlated SILS subscales. The second model tested a second-order factor structure in which all first-order factors were then assumed to load onto the higher-order Implementation Leadership factor. Each of these models were tested twice—once prior to and once post item reduction (see description below). If the first-order factors appreciably load onto the higher-order factor, the second-order factor structure would be prioritized in alignment with this study’s driving theory, measurement development process, and goal of producing a brief yet comprehensive measure of a school’s strategic implementation leadership supportive of EBP implementation.
The initial CFAs were intended to provide evidence of the underlying measurement structure of the SILS. Once established, item characteristics curves were evaluated to narrow SILS items to those most representative of each subscale . Item coverage and redundancy of information were assessed to reduce the number of items for each subscale to three, as the fewest items necessary is a recommended criterion for pragmatic measures . Note that one subscale (proactive) included only three items and so was not subjected to item reduction. Using the reduced item version of SILS, we then tested both CFA models again and recalculated internal consistency estimates. Next, multigroup modeling was used to determine whether the underlying factor structure of SILS was invariant across versions of the scale employing general versus specific EBP item referents. Because the chi-square difference test is heavily influenced by sample size , two additional statistics were used to examine invariance across survey type. Cochran’s Q statistic  was used to determine the difference in magnitude between factor loadings of the two survey types, whereas d (Cox) was used to assess the difference in magnitude between thresholds. Q statistics that cluster around 0 indicate no substantive difference between factor loadings. There are not agreed upon cutpoints for d(Cox). Because d(Cox) ranges from 0 to 1, we employed a decision rule in line with similar effect sizes  such that values greater than .50 would be flagged as a moderate difference between thresholds of the two survey types that would require more thorough investigation.
Convergent and divergent validity were assessed via correlations between SILS subscales and select measures that were theoretically hypothesized to yield small-to-moderate (convergent) or no (divergent) association. Specifically, correlations between SILS subscales and correlations between SILS and MLQ subscales were examined to establish convergent validity. The SILS subscales theoretically measure a unitary construct and as such the inter-subscale correlations were anticipated to be moderate-to-large. Correlations between SILS and all MLQ subscales except for management-by-exception were also expected to be moderate-to-large, but smaller than the SILS inter-subscale correlations. Management-by-exception was anticipated to either be minimally or uncorrelated with SILS subscales. Divergent validity was similarly assessed via correlations, but between SILS subscales and both the PSTQ total score and school-level demographic characteristics. While the SILS and PTSQ are intended to measure different traits, they share the same assessment method (teacher reports) which makes it likely the two measures would share low-to-moderate correlations . Some school-level demographic characteristics might influence teachers’ views of, experience with, and implementation of EBPs. As such, we hypothesized null-to-low correlations between SILS subscales and school-level demographics.
Preliminary confirmatory factor analyses
To establish evidence of the hypothesized measurement model, two preliminary CFA models were examined using all 30 items of the adapted SILS. Results indicated acceptable and identical fit for both the seven-factor correlated model and the second-order factor in which the seven first-order factors loaded onto one higher order factor (CFI = .98, TLI = .98, RMSEA = .07, SRMR = .03). First-order factor loadings were appreciable (λ = .88 to λ = .99) and inter-factor correlations were high (r = .82 to .95). Second-order factor loadings were also high, ranging from λ = .92 to λ = .97.
Figure 1 presents item characteristics curves for all items by SILS subscale. The seventh item on the knowledgeable subscale (panel A, row 1) provided substantially less information (fewer and less pronounced peaks) than the other three items and had the lowest factor loading within the subscale. Item 11 on the supportive subscale (panel A, row 2) contributed less information than items 8, 9, and 10. However, item 11 is the only Supportive subscale item about direct use of an EBP—a key aspect of overall support of EBP implementation. Items 8 and 10 both elicit feedback regarding learning about an EBP, which duplicates content coverage. Further, item 10 showed a similar pattern while contributing less overall information than item 8. Items 12 and 16 on the perseverant subscale (panel A, row 3) provided a similar pattern, but less information than items 13 and 15. Item 17 provided little information and had the lowest factor loading within the subscale. For these reasons, items 7, 10, 12, 16, and 17 were all dropped from their respective subscales.
Item 21 on the communication subscale (panel B, row 1) provided less information and had a lower factor loading than the other three items in the subscale. Item 22 on the vision subscale (panel B, row 2) provided similar patterns of information to item 24, though had the lowest factor loading in the subscale. Items 26 and 27 on the availability subscale (panel B, row 3) provided substantially less information than the other items in the subscale as evidenced by relatively flat lines compared to the other items. For these reasons, items 21, 22, 26, and 27 were dropped from their respective subscales. Table 2 displays summary statistics and inter-item reliabilities for each SILS subscale and Table 3 displays individual item response frequencies.
Confirmatory factor analyses post-item reduction
To confirm that the measurement structure observed using all 30 adapted SILS items held post-item reduction, first- and second-order CFAs were examined using the reduced 21-item scale (3 items per subscale). Results again indicated acceptable and identical fit for both the seven-factor correlated and the second-order factor models (CFI = .99, TLI = .99, RMSEA = .07, SRMR = .02). Figure 2 shows that first-order factor loadings were appreciable (λ = .89 to λ = .98), and inter-factor correlations were high (r = .77 to .95). Second-order factor loadings were also high, ranging from λ=.89 to λ=.96 (see Fig. 3), providing evidence supportive of the theoretical model that SILS subscales comprise a higher-order Implementation Leadership factor. School-specific means, medians, modes, standard deviations, and ranges for each final SILS subscale are provided in Additional file 1.
Multigroup model to test invariance
Results from multigroup models testing invariance between all paths freely estimated versus all paths constrained to be equal indicated invariance across the two survey types administered (χ2(105) = 127.33, p ≤ .07). The chi-square statistic is sensitive to sample size so invariance was also examined using q and d(Cox) statistics, which probe where invariance might be located (factor loadings, thresholds) and the magnitude of the invariance. Results supported invariance of factor loadings as evidenced by q statistics clustered around zero (q = − .03–.02). The vast majority of d(Cox) statistics revealed no difference in thresholds between the two survey types, and no values greater than .50 were observed. There were five thresholds (items 9, 23, 29, and two thresholds for item 30) that approached (.40–.48) though did not reach the .50 cutpoint. The pattern of results indicated that respondents to the general version of the survey may have been more likely to endorse items 29 and 30 than respondents to the specific version. Though future refinement and replication may be needed (see “Discussion” section), the preponderance of evidence showed SILS factor loadings (i.e., the amount of variance each item contributes to a latent factor) and thresholds (i.e., the intercept for each categorical response option) to be invariant across survey type.
Convergent and divergent validity
Table 4 displays bivariate correlations between the means for each of the three-item SILS subscales and other measures included to evidence convergent or divergent validity. Measures of convergent validity are included closer to the top of the table so correlations should decrease as one scans down. As hypothesized and aligned with the inter-factor correlations observed in CFA results, the bivariate correlations between SILS subscales were high (r = .71–.93). Also aligned with hypotheses, all MLQ subscale and scale scores except for management-by-exception were moderately to highly correlated with SILS subscales (r = .56–.80), and less so than the SILS inter-subscale correlations. In line with expectations, Management-by-Exception shared low to no correlation with SILS subscales (r = .05–.12). Correlations with the PTSQ provide preliminary evidence of divergent validity, with moderate correlations that are lower than most subscales of the MLQ (r = .36 – .44). Finally, and providing additional evidence of divergent validity, correlations between SILS subscales and school demographics were low (r = − .31 – .22).
The objectives of the current study were to develop and test a revised version of the SILS for use in the education sector by completing item reduction, confirmatory factor analyses, and examinations of convergent and divergent validity in the context of universal, evidence-based mental health prevention program implementation. Results provided strong evidence of structural validity for the 21-item, 7-factor SILS following item reduction, including for three newly developed subscales that represent salient implementation leadership characteristics in schools: communication, vision, and availability. Communication involves concrete efforts to engage in bidirectional communication surrounding EBP implementation and often is a foundation on which other implementation leadership behaviors are built and maintained. Vision reflects how a leader integrates EBP implementation with the core objectives of a school. Finally, Availability is the extent to which leaders are accessible and responsive to staff needs or problems surrounding implementation. Convergent and divergent validity results for all SILS subscales confirmed study hypotheses, including higher correlations with general leadership scales relative to teacher attitudes and school demographics. Measuring these aspects of implementation leadership provides additional avenues for tracking and supporting the behaviors of school building-level leaders interested in improving the availability of EBP in their systems.
Scores on the SILS suggested moderate levels of most implementation leadership constructs in the current sample (Table 2). All values on the original subscales (i.e., proactive, knowledgeable, supportive, perseverant) were consistently higher than the original ILS development sample in outpatient mental health  and validations in substance use treatment settings  and acute care , but generally comparable to a validation study in child welfare . Scores also were higher than those observed on the earlier version of the ILS adapted for schools . This could be due to the fact that the items and subscales in the new SILS had been specifically tailored to reflect the implementation experiences of school personnel, potentially improving their likelihood of endorsement.
We also observed evidence of invariance in the factor structures between the general (“evidence-based practice” referent) and intervention-specific (SWPBIS or PATHS referent) versions of the SILS. This suggests that the instrument can likely be used to assess the implementation supports delivered by leaders for individual interventions as well as across multiple EBPs, though future replication and refinement are prudent given the moderate to large effect sizes observed for five thresholds (out of 120). Importantly, the invariance established for the SILS factor loadings provides compelling evidence that the interpretation of the underlying construct is synonymous across the different referents used. Differences in thresholds, which were minimal, indicate a shift in the response curves to the right or left of a distribution, but have no bearing on the interpretation of the underlying construct. Such invariance may support the use of the general version of the SILS prior to the selection of an EBP to implement, such as during the Exploration phase of implementation , and the intervention-specific version in later phases. However, future research is needed to evaluate whether the predictive validity of the SILS for variables such as implementation climate  and implementation outcomes  is equivalent for the general and specific versions.
Implications for supporting implementation leadership in schools
Leadership has been found to be a significant predictor of organizational climate . In schools, aspects of leadership and climate also are associated with student wellbeing and success [58, 89]. Measuring implementation leadership in the education sector can be useful in supporting leadership behaviors that create a conducive implementation climate across phases of implementation, including prior to or during active EBP implementation and sustainment. However, little research has developed and evaluated specific implementation strategies that focus on changing aspects of the school organizational context to cultivate an environment that influences educators’ adoption, use, and sustainment of EBPs. Principals and educational leadership teams typically receive little explicit guidance or support surrounding EBP implementation. To address this, the SILS could form the foundation of a leadership-oriented action planning process in schools to improve organizational readiness (i.e., an organization’s commitment to change and implement new practices) . Action planning involves determining who is going to do what and along what sequence and timeline in order for an organization to advance its strategic goals . Low initial readiness accounts for over half of all unsuccessful organizational change efforts  and is heavily influenced by leadership. Since the SILS is pragmatic, brief, and has been designed for repeated administration, resulting data could be used in the context of action planning to drive deployment of novel implementation strategies based on context-specific needs identified prior to, or over the course of, implementation. Future studies should investigate the relative utility of different methods of data presentation—and different indicators of central tendency (e.g., mean vs. median)—for feedback and action planning processes.
Existing leadership-focused implementation strategies, such as Leadership and Organizational Change for Implementation (LOCI)  or iLead  also likely have utility for promoting implementation leadership behaviors among principals and other school building-level leaders. For instance, our research team is currently leveraging the SILS in an adaptation of LOCI for building-level leaders who are implementing mental health prevention programs in schools (Institute of Education Sciences award number R305A200023; https://ies.ed.gov/funding/grantsearch/details.asp?ID=4471). Components of the strategy are being modified to fit with contextual factors such as the school academic calendar, existing professional development needs and supports for leaders, and policies surrounding the design and execution of school improvement plans.
Limitations and future directions
The current study provides strong evidence for the structural, convergent, and divergent validity of the SILS among a sample of elementary school teachers delivering universal prevention programming focused on children’s mental health. Nonetheless, there are important limitations and opportunities for future research surrounding the evaluation of implementation leadership in schools. First, although data were collected during the implementation of two different prevention programs, a larger number of participants were implementing SWPBIS. Additional studies should continue to expand the application of the general and specific versions of the SILS to other programs. Second, as noted above, additional research is needed to evaluate the predictive validity of the SILS, as it relates to variables such as implementation climate and implementation outcomes (e.g., EBP fidelity). Third, the current study focused on principals as the primary formal leaders in school buildings who are ideal targets to promote strategic implementation leadership, given their accountability and central role in decision-making. However, other informal leaders often play important roles in the implementation of EBPs in educational settings. Future studies with the SILS might incorporate additional informal, building-level leaders into data collection efforts in schools that support distributed leadership models . Fourth, further research should examine the degree to which the additional implementation leadership dimensions in the SILS (i.e., communication, vision, and availability) generalize to other settings such as medical care, behavioral health, addiction, or child welfare. Fifth, the number of respondents per organizational unit used to assess organizational constructs such as leadership and climate has been found to vary in the implementation literature . To promote efficient evaluation and feedback processes, future research with the SILS and could explicitly assess the minimum number of responses needed to produce a reliable and valid assessment. Finally, the current study conceptualized implementation leadership as an organizational construct, similar to prevailing characterizations of other constructs such as organizational climate. Nevertheless, although we evaluated ICCs to examine between-school variability, the study was not sufficiently powered to rigorously examine the measurement model at the school level.
The current study adapted and expanded a leading instrument for measuring strategic implementation leadership to ensure its relevance to the implementation of universal prevention programs in schools. The resulting SILS demonstrated structural, convergent, and divergent validity in the context of two distinct interventions designed to prevent student mental health problems. As the education sector continues to be the most common location in the USA for the delivery of mental health services to children and adolescents , opportunities to understand and support building-level leaders in promoting the use of EBP in their systems is critical to ensuring public health impact.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.
Confirmatory factor analysis
Comparative fit index
Implementation Leadership Scale
Leadership and Organizational Change for Implementation
Multifactor Leadership Questionnaire
Multi-tiered systems of support
Promoting Alternative Thinking Strategies
Public School Teacher Questionnaire
Root mean square error of approximation
School Implementation Leadership Scale
Standardized root mean square residual
Schoolwide Positive Behavioral Interventions and Supports
Weighted least squares means and variances
Dodek P, Cahill NE, Heyland DK. The relationship between organizational culture and implementation of clinical practice guidelines. JPEN. 2010;34(6):669–74 Available from: http://onlinelibrary.wiley.com/doi/abs/10.1177/0148607110361905.
Li SA, Jeffs L, Barwick M, Stevens B. Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: a systematic integrative review. Syst Rev. 2018;7(1):72. https://doi.org/10.1186/s13643-018-0734-5.
Scholl I, LaRussa A, Hahlweg P, Kobrin S, Elwyn G. Organizational- and system-level characteristics that influence implementation of shared decision-making and strategies to address them — A scoping review. Implement Sci. 2018;13(1):40. https://doi.org/10.1186/s13012-018-0731-z.
Duong MT, Bruns EJ, Lee K, Cox S, Coifman J, Mayworm A, et al. Rates of mental health service utilization by children and adolescents in schools and other common service settings: a systematic review and meta-analysis. Admin Pol Ment Health. 2021;48(3):420–39. https://doi.org/10.1007/s10488-020-01080-9.
Lyon AR, Bruns EJ. From evidence to impact: Joining our best school mental health practices with our best implementation strategies. Sch Ment Heal. 2019;11(1):106–14. https://doi.org/10.1007/s12310-018-09306-w.
Owens JS, Lyon AR, Brandt NE, Masia Warner C, Nadeem E, Spiel C, et al. Implementation science in school mental health: Key constructs in a developing research agenda. Sch Ment Heal. 2014;6(2):99–111. https://doi.org/10.1007/s12310-013-9115-3.
Langley AK, Nadeem E, Kataoka SH, Stein BD, Jaycox LH. Evidence-based mental health programs in schools: barriers and facilitators of successful implementation. Sch Ment Heal. 2010;2(3):105–13. https://doi.org/10.1007/s12310-010-9038-1.
Locke J, Beidas RS, Marcus S, Stahmer A, Aarons GA, Lyon AR, et al. A mixed methods study of individual and organizational factors that affect implementation of interventions for children with autism in public schools. Implement Sci. 2016;11(1):135. https://doi.org/10.1186/s13012-016-0501-8.
Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45. https://doi.org/10.1186/1748-5908-9-45.
Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Admin Pol Ment Health. 2011;38(1):4–23. https://doi.org/10.1007/s10488-010-0327-7.
Moullin JC, Ehrhart MG, Aarons GA. The role of leadership in organizational implementation and sustainment in service agencies. RSWP. 2018;28(5):558–67. https://doi.org/10.1177/1049731517718361.
Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, et al. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implement Sci. 2012;7(1):1–9 Available from: http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-7-32.
Beidas RS, Aarons G, Barg F, Evans A, Hadley T, Hoagwood K, et al. Policy to implementation: evidence-based practice in community mental health – Study protocol. Implement Sci. 2013;8(1):1–9 Available from: http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-8-38.
Beidas RS, Edmunds J, Ditty M, Watkins J, Walsh L, Marcus S, et al. Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Admin Pol Ment Health. 2014;41(6):788–99. https://doi.org/10.1007/s10488-013-0529-x.
Beidas RS, Marcus S, Aarons GA, Hoagwood KE, Schoenwald S, Evans AC, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatr. 2015;169(4):374–82. https://doi.org/10.1001/jamapediatrics.2014.3736.
Bonham CA, Sommerfeld D, Willging C, Aarons GA. Organizational factors influencing implementation of evidence-based practices for integrated treatment in behavioral health agencies. Psychiatry J. 2014;2014:e802983 Available from: https://www.hindawi.com/journals/psychiatry/2014/802983/.
Brookman-Frazee L, Chlebowski C, Suhrheinrich J, Finn N, Dickson KS, Aarons GA, et al. Characterizing shared and unique implementation influences in two community services systems for autism: applying the EPIS framework to two large-scale autism intervention community effectiveness trials. Admin Pol Ment Health. 2020;47(2):176–87. https://doi.org/10.1007/s10488-019-00931-4.
Suhrheinrich J, Melgarejo M, Root B, Aarons GA, Brookman-Frazee L. Implementation of school-based services for students with autism: barriers and facilitators across urban and rural districts and phases of implementation. Autism. 2021;25(8):2291–304. https://doi.org/10.1177/13623613211016729.
Stadnick NA, Meza RD, Suhrheinrich J, Aarons GA, Brookman-Frazee L, Lyon AR, et al. Leadership profiles associated with the implementation of behavioral health evidence-based practices for autism spectrum disorder in schools. Autism. 2019;23(8):1957–68. https://doi.org/10.1177/1362361319834398.
Bass BM, Avolio BJ. Transformational leadership and organizational culture. PAQ. 1993;17(1):112–21 Available from: http://www.jstor.org/stable/40862298.
Schneider B, Ehrhart MG, Mayer DM, Saltz JL, Niles-Jolly K. Understanding organization-customer links in service settings. Acad Manag J. 2005;48(6):1017–32 Available from: http://www.jstor.org/stable/20159727.
Barling J, Loughlin C, Kelloway EK. Development and test of a model linking safety-specific transformational leadership and occupational safety. J Appl Psychol. 2002;87(3):488–96. https://doi.org/10.1037/0021-9010.87.3.488.
Blase J, Blase J. Effective instructional leadership: teachers’ perspectives on how principals promote teaching and learning in schools. J Educ Adm. 2000;38(2):130–41. https://doi.org/10.1108/09578230010320082.
Goldring E, Huff J, May H, Camburn E. School context and individual characteristics: What influences principal practice? J Educ Adm. 2008;46(3):332–52. https://doi.org/10.1108/09578230810869275.
Hong Y, Liao H, Hu J, Jiang K. Missing link in the service profit chain: a meta-analytic review of the antecedents, consequences, and moderators of service climate. J Appl Psychol. 2013;98(2):237–67. https://doi.org/10.1037/a0031666.
Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255–74.
Schein EH. Organizational culture and leadership. Vol. 2: Wiley; 2010.
Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: A cluster randomized trial study protocol. Implement Sci. 2017;12(1):29. https://doi.org/10.1186/s13012-017-0562-3.
Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prev Med. 2013;45(2):237–43 Available from: https://www.sciencedirect.com/science/article/pii/S0749379713002651.
Finn NK, Torres EM, Ehrhart MG, Roesch SC, Aarons GA. Cross-validation of the implementation leadership scale (ILS) in child welfare service organizations. Child Maltreat. 2016;21(3):250–5. https://doi.org/10.1177/1077559516638768.
Aarons GA, Ehrhart MG, Torres EM, Finn NK, Roesch SC. Validation of the implementation leadership scale (ILS) in substance use disorder treatment organizations. J Subst Abus Treat. 2016;68:31–5 https://www.sciencedirect.com/science/article/pii/S0740547216300654.
Shuman CJ, Ehrhart MG, Torres EM, Veliz P, Kath LM, VanAntwerp K, et al. EBP implementation leadership of frontline nurse managers: validation of the implementation leadership scale in acute care. Worldviews Evid-Based Nurs. 2020;17(1):82–91. https://doi.org/10.1111/wvn.12402.
Hu J, Gifford W, Ruan H, Harrison D, Li Q, Ehrhart MG, et al. Translation and linguistic validation of the implementation leadership scale in Chinese nursing context. J Nurs Manag. 2019;27(5):1030–8 Available from: http://onlinelibrary.wiley.com/doi/abs/10.1111/jonm.12768.
Mandrou E, Tsounis A, Sarafis P. Validity and reliability of the Greek version of Implementation Leadership Scale (ILS). BMC Psychol. 2020;8(1):49. https://doi.org/10.1186/s40359-020-00413-5.
Ehrhart MG, Torres EM, Green AE, Trott EM, Willging CE, Moullin JC, et al. Leading for the long haul: a mixed-method evaluation of the Sustainment Leadership Scale (SLS). Implement Sci. 2018;13(1):1–11 Available from: http://implementationscience.biomedcentral.com/articles/10.1186/s13012-018-0710-4.
Carlson MA, Morris S, Day F, Dadich A, Ryan A, Fradgley EA, et al. Psychometric properties of leadership scales for health professionals: a systematic review. Implement Sci. 2021;16(1):1–22 Available from: http://implementationscience.biomedcentral.com/articles/10.1186/s13012-021-01141-z.
Fazel M, Hoagwood K, Stephan S, Ford T. Mental health interventions in schools in high-income countries. Lancet Psychiatry. 2014;1(5):377–87 Available from: https://www.sciencedirect.com/science/article/pii/S2215036614703128.
Weist MD, Rowling L. International Efforts to Advance Mental Health in Schools. Int J Ment Health Promot. 2002;4(4):3–7. https://doi.org/10.1080/14623730.2002.9721882.
Weist MD, Lever NA, Bradshaw CP, Owens JS. Handbook of school mental health: Research, training, practice, and policy: Springer; 2013.
Weist MD, Bruns EJ, Whitaker K, Wei Y, Kutcher S, Larsen T, et al. School mental health promotion and intervention: experiences from four nations. Sch Psychol Int. 2017;38(4):343–62. https://doi.org/10.1177/0143034317695379.
Farmer EMZ, Burns BJ, Phillips SD, Angold A, Costello EJ. Pathways into and through mental health services for children and adolescents. Psychiatr Serv. 2003;54(1):60–6 Available from: http://ps.psychiatryonline.org/doi/full/10.1176/appi.ps.54.1.60.
Langer DA, Wood JJ, Wood PA, Garland AF, Landsverk J, Hough RL. Mental health service use in schools and non-school-based outpatient settings: comparing predictors of service use. Sch Ment Heal. 2015;7(3):161–73. https://doi.org/10.1007/s12310-015-9146-z.
Lyon AR, Ludwig KA, Stoep AV, Gudmundsen G, McCauley E. Patterns and predictors of mental healthcare utilization in schools and other service sectors among adolescents at risk for depression. Sch Ment Heal. 2013;5(3):155–65. https://doi.org/10.1007/s12310-012-9097-6.
Kessler RC, Demler O, Frank RG, Olfson M, Pincus HA, Walters EE, et al. Prevalence and treatment of mental disorders, 1990 to 2003. NEJM. 2005;352(24):2515–23. https://doi.org/10.1056/NEJMsa043266.
Merikangas KR, Ping HJ, Burstein ME, Swendsen J, Avenevoli S, Case B, et al. Service utilization for lifetime mental disorders in u.s. adolescents: results of the national comorbidity survey adolescent supplement (NCS-A). J Am Acad Child Adolesc Psychiatry. 2011;50(1):32–45 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4408275/.
Bruns EJ, Duong MT, Lyon AR, Pullmann MD, Cook CR, Cheney D, et al. Fostering SMART partnerships to develop an effective continuum of behavioral health services and supports in schools. Am J Orthop. 2016;86(2):156–70 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4788041/.
Cook CR, Williams KR, Guerra NG, Kim TE, Sadek S. Predictors of bullying and victimization in childhood and adolescence: a meta-analytic investigation. Sch Psychol Q. 2010;25(2):65–83. https://doi.org/10.1037/a0020149.
Rones M, Hoagwood K. School-based mental health services: A research review. Clin Child Fam Psychol Rev. 2000;3(4):223–41. https://doi.org/10.1023/A:1026425104386.
Walker HM, Horner RH, Sugai G, Bullis M, Sprague JR, Bricker D, et al. Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. J Emot Behav Disord. 1996;4(4):194–209. https://doi.org/10.1177/106342669600400401.
Durlak JA, DuPre EP. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3):327. https://doi.org/10.1007/s10464-008-9165-0.
Durlak J, Weissberg R. A major meta-analysis of positive youth development programs. Washington D.C: Invited presentation at the Annual Meeting of the American Pschological Association; Washington, DC; 2005.
Gottfredson DC, Gottfredson GD. Quality of school-based prevention programs: results from a national survey. J Res Crime Delinq. 2002;39(1):3–35. https://doi.org/10.1177/002242780203900101.
Ringwalt C, Ennett ST, Vincus AA, Rohrbach LA, Simons-Rudolph A. Who’s calling the shots? Decision-makers and the adoption of effective school-based substance use prevention curricula. J Drug Educ. 2004;34(1):19–31. https://doi.org/10.2190/D2DF-KYDP-P49A-ELC0.
Rowan B. Commitment and control: alternative strategies for the organizational design of schools. Rev Res Educ. 1990;16:353–89 Available from: http://www.jstor.org/stable/1167356.
Urick A, Bowers AJ. What are the different types of principals across the United States? A latent class analysis of principal perception of leadership. Educ Adm Q. 2014;50(1):96–134. https://doi.org/10.1177/0013161X13489019.
Marks HM, Printy SM. Principal leadership and school performance: an integration of transformational and instructional leadership. Educ Adm Q. 2003;39(3):370–97. https://doi.org/10.1177/0013161X03253412.
Griffith J. Relation of principal transformational leadership to school staff job satisfaction, staff turnover, and school performance. J Educ Adm. 2004;42(3):333–56. https://doi.org/10.1108/09578230410534667.
Hallinger P, Heck RH. Reassessing the principal’s role in school effectiveness: a review of empirical research, 1980-1995. Educ Adm Q. 1996;32(1):5–44. https://doi.org/10.1177/0013161X96032001002.
Brackett MA, Reyes MR, Rivers SE, Elbertson NA, Salovey P. Assessing teachers’ beliefs about social and emotional learning. J Psychoeduc Assess. 2012;30(3):219–36. https://doi.org/10.1177/0734282911424879.
McIntosh K, Kelm JL, Canizal DA. In search of how principals change: a qualitative study of events that help and hinder administrator support for school-wide PBIS. J Posit Behav Interv. 2016;18(2):100–10. https://doi.org/10.1177/1098300715599960.
Pinkelman SE, McIntosh K, Rasplica CK, Berg T, Strickland-Cohen MK. Perceived enablers and barriers related to sustainability of school-wide positive behavioral interventions and supports. Behav Disord. 2015;40(3):171–83 Available from: http://www.jstor.org/stable/43740445.
Meza RD, Beidas RS, Ehrhart MG, Mandell DS, Dorsey S, Frederick L, et al. Discrepancies and agreement in perceptions of implementation leadership: Associations with dosage of school-based evidence-based practices for children with autism. Admin Pol Ment Health. 2019;46(4):518–29. https://doi.org/10.1007/s10488-019-00932-3.
Locke J, Lawson GM, Beidas RS, Aarons GA, Xie M, Lyon AR, et al. Individual and organizational factors that affect implementation of evidence-based practices for children with autism in public schools: a cross-sectional observational study. Implement Sci. 2019;14(1):1–9 Available from: http://implementationscience.biomedcentral.com/articles/10.1186/s13012-019-0877-3.
Murphy J, Goldring E, Cravens X, Elliott S, Porter A. The vanderbilt assessment of leadership in education: measuring learning-centered leadership. ECNU. 2007;29(1):1–10.
Hallinger P. Principal instructional management rating scale. Sarasota: Leading Development Associates; 1990.
Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M, et al. Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implement Sci. 2018;13(1):5. https://doi.org/10.1186/s13012-017-0705-6.
Locke J, Shih W, Kang-Yi CD, Caramanico J, Shingledecker T, Gibson J, et al. The impact of implementation support on the use of a social engagement intervention for children with autism in public schools. Autism. 2018;23(4):834–45. https://doi.org/10.1177/1362361318787802.
Lewis TJ, Sugai G. Effective behavior support: systems approach to proactive schoolwide management. FOEC. 1999;31(6):1–24 Available from: https://pdfs.semanticscholar.org/4770/0eae832c0bd2103a36b319dd8477ec7d689c.pdf.
Greenberg MT, Kusche CA, Cook ET, Quamma JP. Promoting emotional competence in school-aged children: The effects of the PATHS curriculum. Dev Psychopathol. 1995;7(1):117–36. https://doi.org/10.1017/S0954579400006374.
Hambleton RK. Guidelines for adapting educational and psychological tests; 1996. Available from: https://eric.ed.gov/?id=ED399291
Hambleton RK, Merenda PF, Spielberger CD. Adapting educational and psychological tests for cross-cultural assessment: Psychology Press; 2004. p. 391.
Antonakis J, Avolio BJ, Sivasubramaniam N. Context and leadership: an examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. Leadersh Q. 2003;14(3):261–95 Available from: https://www.sciencedirect.com/science/article/pii/S1048984303000304.
Muenjohn DN, Armstrong PA. Evaluating the structural validity of the Multifactor Leadership Questionnaire (MLQ), capturing the leadership factors of transformational-transactional leadership. Contemp Manag Res. 2008;4(1):3–14.
National Center for Education Statistics. Teacher questionnaire: Schools and staffing survey: 2007-08 school year: U.S. Department of Education: National Center on Educational Statistics; 2008. Available from: https://nces.ed.gov/surveys/sass/pdf/0708/sass4a.pdf
Rimm-Kaufman SE, Sawyer BE. Primary-grade teachers’ self-efficacy beliefs, attitudes toward teaching, and discipline and teaching practice priorities in relation to the “responsive classroom” approach. Elem Sch J. 2004;104(4):321–41 Available from: http://www.jstor.org/stable/3202945.
Muthén LK, Muthén BO. Mplus: Statistical Analysis with Latent Variables: User’s Guide (Version 8). Los Angeles: Muthén & Muthén; 2017.
Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model. 1999;6(1):1–55. https://doi.org/10.1080/10705519909540118.
Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38(1):1–10. https://doi.org/10.1007/BF02291170.
Bollen KA, Long JS. Introduction. In: Testing structural equation models. Newbury Park: Sage Publications; 1993. p. 1–9. Sage focus editions.
Rigdon EE. CFI versus RMSEA: a comparison of two fit indexes for structural equation modeling. Struct Equ Model. 1996;3(4):369–79. https://doi.org/10.1080/10705519609540052.
Tabachnick BG, Fidell LS. Using multivariate statistics. 7th ed: Pearson; 2018.
De Ayala RJ. The theory and practice of item response theory: Guilford Publications; 2013.
Meade AW, Johnson EC, Braddy PW. Power and sensitivity of alternative fit indices in tests of measurement invariance. J Appl Psychol. 2008;93(3):568–92. https://doi.org/10.1037/0021-9010.93.3.568.
Cochran WG. The comparison of percentages in matched samples. In: Biometrika: Oxford University Press; 1950. p. 256–66. Available from: http://www.jstor.org/stable/2332378.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Lawrence Erlbaum Associates; 1988. p. 20–6.
Tehseen S, Ramayah T, Sulaiman S. Testing and controlling for common method variance: a review of available methods. J Manage Sci. 2017;4(2):146–75 Available from: https://geistscience.com/JMS/Issue2-17/Article4/JMS1704202.pdf.
Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implement Sci. 2014;9(1):1–11 Available from: http://implementationscience.biomedcentral.com/articles/10.1186/s13012-014-0157-1.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38(2):65–76. https://doi.org/10.1007/s10488-010-0319-7.
Bryk A, Sebring P, Allensworth E, Luppescu S, Easton J. Organizing schools for improvement: Lessons from Chicago: University Of Chicago Press; 2010.
Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4(1):67. https://doi.org/10.1186/1748-5908-4-67.
Schwarzer R, Lippke S, Luszczynska A. Mechanisms of health behavior change in persons with chronic illness or disability: The Health Action Process Approach (HAPA). Rehabil Psychol. 2011;56(3):161–70. https://doi.org/10.1037/a0024509.
Kotter JP. Leading change: Harvard Business Press; 1996.
Richter A, von Thiele SU, Lornudd C, Lundmark R, Mosson R, Hasson H. iLead—a transformational leadership intervention to train healthcare managers’ implementation leadership. Implement Sci. 2015;11(1):1–13 Available from: http://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0475-6.
Tian M, Risku M, Collin K. A meta-analysis of distributed leadership from 2002 to 2013: Theory development, empirical evidence and future research focus. EMAL. 2016;44(1):146–64. https://doi.org/10.1177/1741143214558576.
Emmons KM, Weiner B, Fernandez ME, Tu SP. Systems antecedents for dissemination and implementation: a review and analysis of measures. Health Educ Behav. 2012;39(1):87–105. https://doi.org/10.1177/1090198111409748.
We would like to thank Mahima Joshi for assisting with references and final formatting of the manuscript prior to submission.
This publication was funded by the Institute of Education Sciences (R305A160114 and R305A200023). GAA was supported by NIDA grant R01DA04989 and NIMH grant R03MH117493. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Institute of Education Sciences, which had no decision-making authority or input surrounding this manuscript.
Ethics approval and consent to participate
All study procedures were reviewed and approved by the University of Washington institutional review board. The study team conducted online informed consent with prospective participants prior to survey completion.
Consent for publication
GAA is an Associate Editor of Implementation Science. All decisions on this paper were made by another editor. The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lyon, A.R., Corbin, C.M., Brown, E.C. et al. Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS). Implementation Sci 17, 48 (2022). https://doi.org/10.1186/s13012-022-01222-7
- Implementation leadership
- Mental health