Skip to main content
Download PDF
Download ePub

The Evidence-based Practice Attitude Scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples

Implementation Science volume 12, Article number: 44 (2017) Cite this article

Abstract

Background

Short and valid instruments for measuring factors facilitating or hindering implementation efforts are called for. This article describes (1) the adaptation of a shorter version of the Evidence-based Practice Attitude Scale (EBPAS-50 items), and (2) the psychometric properties of the shortened version in both US and Norwegian data.

Methods

The US participants were mental health service providers (N = 418) recruited from clinics providing mental health services in San Diego County, California. The Norwegian participants were psychologists, psychiatric nurses, and psychology students (N = 838) recruited from the Norwegian Psychological Association and the Norwegian Nurses Organization. A confirmatory factor analysis (CFA) approach was used.

Results

The reduction resulted in 36 items named EBPAS-36, and the original 12 factor model was maintained. The EBPAS-36 had acceptable model fit, as indicated by a low degree of misspecification errors in both the US (RMSEA = .045 (CI90% .040–.049); SRMR = .05) and the Norwegian data (RMSEA = .052 (CI90% .047–.056, SRMR = .07). Incremental model fit was fair in the US (CFI = .93, TLI = .91) and in the Norwegian samples (CFI = .91, TLI = .89). The internal consistency (Cronbach’s α) in the US and the Norwegian samples were good for the total EBPAS-36 score (.79 and .86, respectively) and were ranged from adequate to excellent for the subscales (US .60–.91 and Norway .61–.92).

Conclusions

The EBPAS-36 has adequate psychometric properties both in US and Norwegian samples, hence indicating cross-cultural validity. It is a brief, pragmatic, and more user-friendly instrument than the EBPAS-50, yet maintains a broad scope by retaining the original 12 measurement domains.

Peer Review reports

Background

Most evidence-based interventions never become implemented in real-world practice despite a substantial focus on implementation of evidence-based psychological interventions [1, 2]. A remedy is to increase knowledge about what makes implementation successful, and hence development and validation of pragmatic, yet psychometrically strong instruments, becomes crucial [3]. Use of instruments that cover a broad area of factors that facilitate or hinder implementation may provide valuable knowledge to help tailor implementation strategies in order to overcome implementation obstacles. In contrast, the use of poor quality instruments might slow advances of the implementation knowledge base [3, 4], which ultimately may negatively influence the quality of services provided.

Several concerns regarding instrumentation may impede advances in implementation science. This includes a growing number of frameworks, theories or models, an increasing diversity in the operationalization of constructs, improper psychometric testing of instruments, and scant practicality and pragmatism of the available instruments [3–9]. The Society for Implementation Research Collaboration (SIRC) Instrument Review Project [3] recently identified over 420 instruments covering 48 different implementation constructs: “factors inside domains (characteristics of the intervention, characteristics of individuals involved in the implementation, outer settings, inner setting, process, implementation outcomes and client outcomes) that may predict, moderate, or mediate evidence-based intervention dissemination and implementation, as well as implementation outcomes” (p. 3). The preliminary results from the SIRC project suggest that few instruments are psychometrically strong or have been developed through a sufficiently systematic approach [3].

Moreover, the implementation process encompasses different phases involving complex multilevel challenges, such as the exploration, the preparation, the implementation, and the sustainment phase characterized by the Exploration Preparation Implementation and Sustainment (EPIS) implementation framework [10]. In all phases, specific characteristics of the clinical interventions, the patients, the health care professionals, the organizations, and even the policies of health authorities may involve barriers or facilitators for a successful implementation [5, 11–17]. Regarding the health care professionals, one factor that affects the process and outcomes of implementation is service providers’ attitudes to evidence-based practice (EBP) [11, 13, 18, 19]. This is important as it may influence the initial process of deciding whether to launch new practices, the actual implementation process, and how to sustain of interventions efforts within service settings [11, 13].

The Evidence-based Practice Attitude Scale (EBPAS) was developed from theories of dissemination and implementation in mental health, as well as consultations with mental health service providers and researchers [11, 13]. The original EBPAS consisted of 15 items (EBPAS-15) covering four attitude domains: (1) the intuitive appeal of EBP, (2) the likelihood of adopting EBP given requirements to do so, (3) Openness to new practices, and (4) the perceived divergence of one’s usual practice with research-based/academically developed interventions. The EBPAS-15 has good psychometric properties [11, 20–23] and is highlighted as psychometrically strong by the SIRC Instrument Review Project [3]. The scores from the EBPAS-15 are associated with relevant provider demographic characteristics, organizational characteristics, leadership [11, 12, 20, 23], as well as provider adoption and use of EBP [24]. For instance, higher educational status is associated with more favorable attitudes [11]. Also, higher levels of positive leadership styles [20] and more constructive organizational culture [23] is associated with more positive provider attitudes, while poorer organizational climate is associated with greater perceived divergence between ones usual practice and EBPs [23]. More recent work has expanded the purview of attitudes and resulted in the development of eight additional domains dispersed across 35 new items (EBPAS-50) [13]: (5) the limitations of EBPs, (6) the EBPs fit with values and needs of client and clinician, (7) the negative perceptions of monitoring, (8) the balance between perceptions of clinical skills and science as important in service provision, (9) the time and administrative burden with learning EBPs, (10) job security related to expertise in EBP, (11) perceived organizational support, and (12) positive perceptions of receiving feedback.

The expansion from 15 to 50 items covers a wider domain of attitude concepts. However, cross-cultural translations and validation studies are lacking. As part of a Norwegian survey among psychologists and nurses examining challenges with implementation of evidence-based interventions and systems for improving the quality of mental health service settings, a Norwegian translation of the EBPAS-50 was included. Given the need for briefer, yet reliable and valid instruments [3–6], we examined ways of shortening the EBPAS-50.

Aims of the present study

The present study aimed to shorten the original EBPAS-50 but maintain the original 12 subscales. Furthermore, we examined the factor structure, and the reliability of its subscale scores across two cultures contexts (US and Norway). We expected that the shortened version would have higher user acceptability, retain the original factors structure, show satisfactory reliability, and display indices of good convergent and discriminant validity.

Methods

US: procedure

The study was approved by the appropriate institutional review boards prior to recruitment, and informed consent was obtained prior to administering surveys. The research team recruited participants from mental health clinics providing mental health services for children, adolescents, and families in San Diego County, California, United States. Of the initial 99 county run and contracted programs identified, 72 programs were eligible because they provided either outpatient or day treatment mental health services. Twenty-six of the 99 clinics were identified as ineligible because they were residential treatment facilities, lacked the appropriate organizational structure (i.e., no supervisor or program manager for the clinic), or due to inability to make contact with the program. Of the 72 eligible programs, seven programs refused (90.3% response rate). The total number of eligible participants from the 65 participating programs was 440, of which 435 agreed to participate (98.9% response rate). Fifteen individuals were administrative assistants and were not asked to respond to the EBPAS portion of the survey, and two individuals were excluded due to missing data resulting in a total sample size of 418.

Norway: procedure

Participants were invited by emails sent out by the Norwegian Psychological Association to half of their members (Norwegian sample 1, N = 3598) and by the Norwegian Nurses Organization, professional group for nurses in mental health and substance abuse, to all of their members (Norwegian sample 2, N = 1436). The survey was also announced on the Internet sites of these two organizations. The invitation email contained a web link providing access to the survey, as well as information about the study provided by the research group. All respondents provided informed consent according to recommendations of the Norwegian data protection authority for the project. Completion of the survey was accepted as consent to participate in the project. The online SurveyMonkey software was used to collect data during May–October 2014. One and two reminders were sent to samples 2 and 1, respectively. For incentives, all participants had the opportunity to participate in random drawings for one iPad mini, and two psychology and nursing handbooks.

A total of 856 psychologists and psychology students (24.0% response rate for sample (1) and 191 nurses (13.3% response rate for sample (2) completed the survey (N = 1047). Subjects not completing any of the EBPAS-50 items were excluded, as well as those with missing data for whole subscales, >1 item on a 3-item scale or >2 items on a 4-item scale (N = 209). Thus, the final Norwegian sample included data from 838 respondents.

Norwegian translation procedure

The Norwegian translation of the EBPAS-50 was conducted by the first author (MR) in 2013, and back-translated by a professional. The procedure followed recommended guidelines for cross-cultural translation, adaptation, and validation of instruments [25]. Any deviations between the original and the back-translated version were resolved by a consensus discussion between the first (MR) and the last author (IS). The final Norwegian version was reviewed, revised, and approved through an iterative process that resulted in consensus between MR and the original EBPAS-50 author (GAA). The measure was then given to a sample of clinicians, psychology and PhD students, and comments related to readability were used to finalize the translation.

Some translational adaptions with regard to the definition of the concept of EBP were made for the written instructions of the instruments. In the original version, the instructions specified that “evidence-based practice” refers to any intervention that is supported by empirical research. The Norwegian instructions were limited to evidence-based interventions (i.e., therapies, methods). This was consistent with the American Psychological Association and Norwegian Psychological Associations’ definitions of evidence-based psychological practice. It thus makes an important distinction between the more comprehensive concept of evidence-based practice, referring to the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences, and evidence-based treatments and interventions, referring to specific research-supported interventions [26, 27].

Measures/assessment

US: demographic characteristics

Participants’ gender, age, ethnicity/race, level of education, primary discipline, years worked in mental health, and years worked in current agency were collected.

Norway: demographic characteristics

Participants’ gender, age (response categories < 30 years, 31–40 years, 41–50 years, 51–60 years, and > 60 years), level of education, profession, and years worked in substance abuse- and/or mental health service were collected.

The Evidence-based Practice Attitude Scale (EBPAS-50)

EBPAS-50 is a 50-item instrument developed to assess mental health and social service providers’ attitudes toward adopting EBP [13]. The 50 EBPAS-items cover 12 subscales: appeal (four items), requirements (three items), openness (four items), divergence (four items), limitations (seven items), fit (seven items), monitoring (four items), balance (four items), burden (four items), job security (three items), organizational support (three items), and feedback (three items). The 12 subdomains sum up in a higher order total scale score representing respondent’s global attitudes toward evidence-based practice. The items are formulated as statements, and responses are given on a 5-point Likert scale ranging from 0-“not at all” to 4-“to a very great extent”. In order to assess different perspectives and to reduce response biases, 23 items belonging to five subscales (divergence, limitations, monitoring, balance, and burden) are negatively framed. According to the EBPAS-50 scoring instructions, for the total score these items are reversed scored, and the mean subscale scores recomputed, before a mean score for the total EBPAS-50 item score is computed. A higher total score indicate a more positive attitude towards adoption of evidence-based practice.

Open-comment fields

For the Norwegian survey, respondents also had the opportunity to convey their opinions and to supply supplementary information in open-comment fields to provide feedback regarding content and the measure overall.

Statistical analyses

Confirmatory factor analyses (CFA) for item reduction evaluations were conducted in Mplus v7.2. The model specification was based on the 12 subscales of the expanded EBPAS-50. The Norwegian sample was split using the first sample to identify a shorter version (N = 413), and the second sample to validate the factor structure (N = 425). For the US CFA’s, the same sample was used for both the reduction and the validation process. The parameters were estimated with the full information maximum likelihood procedure (FIML), and robust standard errors (MLR) were requested in order to accommodate for non-normal item distributions. The MLR procedure is efficient and works comparably well as weighted least squares procedures for ordinal data with five or more ordinal categories [28]. The following model fit indices were used: χ2, root mean square error of approximation (RMSEA), standardized root mean error (SRMR), and comparative fit indices (CFI) [29]. Following Hu and Bentler’s cutoff recommendations [30], RMSEA values close to .06, SRMR close to .08, and CFI close to 0.95 indicate acceptable model fit. The US sample controlled for the nested data structure of providers within program. Given the aim to reduce the length of the EBPAS-50 while retaining the original 12 factors, a minimum of three items per factor were retained. Subscales containing four or more items were thus shortened based on a combination of the following criteria: (1) retain items with the highest factor loadings; (2) evaluations of modification indices; (3) items being conceptually similar or adding unique information. The reduction process was done separately for the US and the Norwegian samples, and the resulting versions were compared and discussed before reaching a final consensus version. It is important to note that our primary goal was to develop a brief and a pragmatic measure that represented the original constructs identified in the EBPAS-50 subscales. The removal procedure put weight on keeping items that preserved the content and meaning of the subscales, in addition to information about factor loadings and scale reliability.

Calculations of test parameters

The validity of the questionnaire was measured by acceptability, the percentage of items left unanswered, and the interpretability of the components. SPSS version 22 was used for basic statistical analyses and estimation of internal consistency (Cronbach’s α).

Results

Samples

For the US sample, the average age of the 418 participants was 36.3 (SD = 10.6; range = 21–66), and the majority of respondents were female (79.8%). Participants worked in the mental health services field for a range of 0–43 years, in child and/or adolescent mental health services for a range of 0–42.7 years, and in their present agency for a range of 0–29.1 years participants’ areas of primary discipline included: 2.5% child development, 0.2% drug/alcohol counseling, 1.5% human relations, 48% marriage and family therapy, 1% nursing, 0.2% probation, 0.5% psychiatry, 15.3% psychology, 24.6% social work, and 6.2% other discipline. Participant demographic characteristics for the US sample are provided in Table 1.

Table 1 Demographic characteristics of the US sample
Full size table

For the Norwegian sample, the majority of the sample (N = 838) were in the age category 31–50 years, one third were older than 50, and a sixth younger than 30. An estimated mean age for the Norwegian sample was thus 43,4 years, based on the midpoint of each age category and weighted by the numbers falling in each category (for those in the category under 30, a midpoint of 27 years was chosen, since psychologists and nurses graduate at minimum age 24–25. The majority of the participants were female (68.6%). Clinical psychologist reported working in the substance abuse and mental health service field for a range of 0–45 years, while nurses reported a range of 2–42 years of experience. Sixty-two (7.4%) participants were psychology students following a six-year university education and training program in clinical psychology leading to the postgraduate cand. psychol. degree, 655 were authorized clinical psychologists with an accomplished postgraduate cand. psychol. degree (78.1%), and 121 were authorized nurses with an accomplished Bachelor’s degree in nursing (14.5%). Participants came from a diverse setting of mental health services, with the largest group representing outpatient units for adults (18.0%). Participant demographic characteristics for the Norwegian sample are presented in Table 2.

Table 2 Demographic characteristics of the Norwegian sample (N = 838)
Full size table

Acceptability

Among the 418 respondents in the US sample 319 had complete data (76%), but of those with missing data, 49 of the 101 (49%) had missing information on only one item, 24 had 2 to 5 items missing (24%), and the remaining 28 (27%) had more than five items missing. The reporting of unanswered items in the Norwegian sample is based on cases that completed the EBPAS-50 items (N = 884). More than three quarters (76.81%) completed all 50 items of the EBPAS-50. No single item was left open by more than 3.3%. Item 17 was omitted most frequently. Moreover, participants provided comments recommending shortening, such as: “I think there are too many questions of quite similar nature, which makes it boring and difficult to answer”, or “Too many questions with same content made it tempting to quit without completing”.

Item reduction

The reduction process was done separately for the US and Norwegian sample. Following the reduction criteria as described in the Statistics section, one item in each of the following subscales having four items were removed: appeal, openness, divergence, monitoring, balance, and burden. The final 36-item version was agreed on following a consensus discussion and is presented in Table 3 for the US and Norwegian samples, respectively. Item 9 from the appeal subscale (“intuitively appealing”) and item 8 from the openness subscale (“Would try therapy/interventions different than usual”) were removed due to content overlap with other items in these scales. Item 34 in the balance subscale (“Satisfied with my skills”) was removed as we considered the content validity of this item as poorer compared to the other three items. Item 3 on the divergence subscale had a low factor loading and was removed. A discrepancy between the US and the Norwegian sample with regard to item 30 (“I prefer to work on my own without oversight”) and item 33 (“I do not need to be monitored”) on the monitoring subscale was solved following a consensus discussion. Consequently, item 33 was removed as it was framed too generally and not specifically referring to work or services. A similar discrepancy occurred with regard to two items on the burden subscale (item 38 “Don’t have time to learn anything new” and item 41 “EBP will cause too much paperwork”). A consensus was reached for excluding item 41, since, unlike the items retained, the referent was not specific to the individual (i.e., EBP will cause too much paperwork for me). Also, four items in each of the two subscales with seven items were excluded: limitations and fit. In determining which items to exclude from the limitations subscale, three items were removed on the basis of having the lowest factor loadings. However, item 27 “Families with multiple problems” was excluded despite a high factor loading due to content similarity with item 26 “Clients with multiple problems”. The latter is more universal and was thus retained. There was an additional discrepancy between the US and the Norwegian samples for two items from the limitations subscale (item 24 “Evidence-based practice makes it harder to develop a strong working alliance” and item 28 “Evidence-based practice is not individualized treatment”). Item 28 was retained as it matches the underlying construct of the scale better. For the fit subscale, the three lowest loading items were removed. An additional item was removed (item 19 “Had a say in which evidence-based practice”) as many therapists have less influence on which specific evidence-based practice to adopt rather than how an evidence-based practice will be used. In addition, item 22 “Fit with your treatment philosophy” was excluded despite a high factor loading in the Norwegian sample due to content overlap with item 21 “Fit with your clinical approach”. The English and Norwegian instruments, including scoring instructions, can be found in Additional files 1, 2, 3, and 4.

Table 3 EBPAS-50 and EBPAS-36 items standardized factor loadings, means, and standard deviations in US and Norwegian samples
Full size table

Subscale score correlations

The correlation coefficients (Pearson r) between the 12 EBPAS-36 subscales are presented in Table 4 separately for the Norwegian and US samples. Norwegian sample: the highest correlations were between the limitations and the divergence subscales (r = .56), the job Security and the organizational support subscales (r = .53), the openness and the divergence subscales (r = −.49), the appeal and the fit subscales (r = .41), the organizational support and the openness subscales (r = .40), the divergence and the balance subscales (r = .38), and the limitations and openness subscales (r = −.37), all in the expected directions. There were some similarities with the Norwegian sample correlations that emerged in the US sample, specifically between organizational support and openness (r = .40). Though the highest correlation found was between appeal and organizational support (r = .57).

Table 4 EBPAS-36 scale factor intercorrelations for US and Norwegian samples
Full size table

Confirmatory factor analyses

In the US sample, after adjusting for the nested data structure (i.e., providers within programs), the absolute fit was significant (χ2 = 968.85, p < .001); however, model fit was adequate in terms of low misspecification (RMSEA = .045, CI 90% [.040, .049]; SRMR = .05), and fair with regard to incremental fit (CFI = .93, TLI = .91). The standardized factor loadings ranged from .48–.98 (all p’s < .001). In the Norwegian sample, the absolute fit again was significant fit (χ2 = 1125.04, p < .001), but adequate in terms of low misspecification (RMSEA = .052, CI 90% [.047, .056]; SRMR = .07), and fair in terms of incremental fit (CFI = .91, TLI = .89). The standardized factor loadings ranged from .52 to 1.00 (all p’s < .001, except for item 18 with p = .003).

Internal consistency

The internal consistency of the US and Norwegian EBPAS-36, as well as the US EBPAS-50 [11, 13], is presented in Table 5. The US EBPAS-36 total scale Cronbach’s α was .80, with subscale α that ranged between .60 and .91. The divergence subscale had the lowest α (.60), which could not be improved by removing the lowest item-total correlation, and hence could be interpreted as questionable [31]. The remaining subscales had Cronbach’s α above .70, indicating acceptable to excellent levels of internal consistency. The Norwegian EBPAS-36 total scale had Cronbach’s α .86, and subscales α ranging between .61 and .92. The Appeal, Fit, Balance and Divergence subscales had the lowest alphas (.61, .62, .64 and .68, respectively) and did not improve above > .70 following removal of items with low item-total correlations, which was less satisfactory. The remaining subscales α were above > .70. Compared to the Cronbach’s α in the US EBPAS-50, both the US and Norwegian EBPAS-36 had lower α values, as expected for shorter scales, as well as implementation constructs of a broad nature.

Table 5 Internal consistency, Cronbach’s α, for US and Norwegian samples, and original version of EPBPAS-50
Full size table

Discussion

There is a profound need for shorter and pragmatic instruments that at the same time cover a wide spectrum of measurement constructs in implementation research. This article describes the shortening of the EBPAS-50 from 50 to 36 items based on data collected in the US and in Norway. The revised instrument, named EBBAS-36, measures 12 dimensions of provider’s attitudes to adopt new practices in mental health care service settings, similarly as the original EBPAS-50 does. Data from both cultures indicated adequate psychometric properties of the EBPAS-36, hence being cross-culturally valid. The shortened version is not compromised by narrowing of the measurement domain as it retains the original 12 factor structure. The internal consistency of most subscales was good to high, and on par with the EBPAS-50. Some subscales had slightly lower internal consistencies in the US and the Norwegian versions: these were the appeal, the fit, and the balance subscales. Lower internal consistency is in general expected if reducing the number of items. Given that the EBPAS-36 contains only three items per factor, the lowered internal consistency may be considered as adequate, especially if compensated for by increasing the sample size in research employing these short versions. Another factor in the results may be the complexity of the measured concepts, as more heterogeneous constructs also can attenuate alpha where there are lower item-total correlations. Since attitudes towards implementation of EBP may be considered as relatively broad in scope, we consider the slightly reduced reliability to be well compensated by the broader validity of retaining the original 12 dimensions and the practicality of a brief measure that can be used more efficiently for research, organizational development, and provider development. Furthermore, several comments to the 50-item version indicated that responders were annoyed or fatigued by having to answer several seemingly identical items. Shortening the instrument is rather more likely to strengthen the validity of the scale due to fewer response biases related to irritated or fatigued respondents, which in the test literature should decrease the phenomenon of “satisficing” [32].

The literature has suggested multiple criteria for “pragmatic” measures including being important to stakeholders; having low burden; being sensitive to change; being broadly applicable; can be used for benchmarking; has norms; is unlikely to cause harm; is psychometrically strong; and is related to theory or model [8, 9]. This revision of the EBPAS-50 to a more brief and focused measure fits most of these criteria when considered in the context of studies utilizing the EBPAS and EBPAS-50, from which the EBPAS-36 was adapted. First, the EBPAS and EBPAS-50 may be deemed to be important to stakeholders by virtue of the wide use of the measures for research, service improvement, and practice [33]. Second, burden for the measure is low for respondents and the measure can be completed in just a few minutes. Third, the EBPAS and/or EBPAS-50 have been used in a variety of settings including health/medicine [34], mental health [35], substance abuse [36], education [37], social care [38], and across countries and cultures [34, 39, 40]. Fourth, norms have been established for behavioral health settings in the United States [21], and thus can be used for benchmarking in this setting, and as the measures are utilized more broadly, evidence and normative data will become available to aid in interpretation and in understanding of both mean scores and variability in responses across countries, cultures, and various health settings. Fifth, the measure is very unlikely to cause harm. Sixth, the measure is psychometrically strong [21]. And seventh, the measure is clearly related to theory including theories of links between attitudes and behaviors [41] and as identified in multiple implementation frameworks [10, 14, 42]. While further testing of the EBPAS-36 is clearly warranted, the present study conducted across cultures, languages, and using rigorous approaches to factor structures, reliability, and validity—along with consideration of previous work—supports this new measure as both brief and having very high potential for being pragmatic.

Some limitations of this study should be mentioned. One concerns the low response rate for the Norwegian sample. Some of the Norwegian respondents provided written comments to the survey related to the response scale. The Norwegian translation used the same anchor points as the original version: 0-not at all, 1-to a slight extent, 2-to a moderate extent, 3-to a great extent and 4-to a very great extent. During piloting of the translation, a different response scale was examined, which led to response problems related to two of the items containing negations. In order to avoid changing both the phrasing of the items and the scaling, the original response scale was used. Another concern may be the introductory cross-cultural definition differences that were used to describe the basis for evidence. Normally, comparability across cultures is the rule, which we bypassed for good reasons. The English version describes “evidence-based practice” as referring to any interventions that is supported by empirical research. However, the Norwegian version limited the evidence-based definitions to the methods used (i.e., therapies, interventions). This was an attempt to distinguish the more comprehensive concept of evidence-based practice (referring to the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences) from the concept of evidence-based treatment, which refers to special treatments supported by empirical research. Written comments from the Norwegian respondents indicated some had difficulties with discerning these two complex concepts. Future use of the instrument should pay special attention to this challenge, for example, by comparing the outcome of using these two different introductory statements (broad versus narrow definition) by randomly distributing them to two different samples. However, the solutions in Norwegian and US were so similar, that the impact of the difference in definitions, if existent, is probably minimal.

EBPAS-50 was originally developed both for research and for applied purposes. The intention behind developing the measure was in large extent to provide a relatively brief measure, both to be used in studies of organizational and individual readiness to implement new evidence-based interventions, and for understanding factors related to adoption, implementation, and continued use of evidence-based interventions [20]. The presented 36-item version builds upon this intention, providing an even shorter instrument measuring the same dimensions as the EBPAS-50. Our procedure for creating a short and pragmatic version may serve as a model for other researchers within the field. Future research on the EBPAS-36 may examine how organizational and individual factors relate to the various EBPAS-36 attitude dimensions, which may help tailor implementation strategies that promote an organizational climate that adopt new interventions positively.

Conclusions

The EBPAS-36 has adequate psychometric properties both in US and in Norwegian samples, hence, indicating cross-cultural validity. It is a brief, pragmatic, and more user-friendly instrument than the EPBAS-50, yet maintains a broad scope by retaining the original 12 measurement domains.

Abbreviations

CFA:

Confirmatory factor analysis

EBP:

Evidence-based practice

EBPAS:

Evidence-based Practice Attitude Scale

EPIS:

Exploration Preparation Implementation and Sustainment

References

  1. Haines A, Kuruvilla S, Borchert M. Bridging the implementation gap between knowledge and action for health. Bull World Health Organ. 2004;82(10):724–31.

    PubMed  PubMed Central  Google Scholar 

  2. McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. Am Psychol. 2010;65(2):73–84.

    Article  PubMed  Google Scholar 

  3. Lewis CC, et al. The Society for Implementation Research Collaboration Instrument Review Project: a methodology to promote rigorous evaluation. Implement Sci. 2015;10:2.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014;9:118.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci. 2013;8:22.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Chor KH, et al. Measures for predictors of innovation adoption. Adm Policy Ment Health. 2015;42(5):545–73.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Lewis CC, et al. Advancing implementation science through measure development and evaluation: a study protocol. Implement Sci. 2015;10:102.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Glasgow RE, Riley WT. Pragmatic measures what they are and why we need them. Am J Prev Med. 2013;45(2):237–43.

    Article  PubMed  Google Scholar 

  10. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38(1):4–23.

    Article  Google Scholar 

  11. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004;6(2):61–74.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Aarons GA. Measuring provider attitudes toward evidence-based practice: consideration of organizational context and individual differences. Child Adolesc Psychiatr Clin N Am. 2005;14(2):255–71. viii.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Aarons GA, et al. Expanding the domains of attitudes towards evidence-based practice: the evidence based practice attitude scale-50. Adm Policy Ment Health. 2012;39(5):331–40.

    Article  PubMed  Google Scholar 

  14. Damschroder LJ, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50.

    Article  PubMed  Google Scholar 

  16. Glisson C, et al. Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health. 2008;35(1–2):98–113.

    Article  PubMed  Google Scholar 

  17. Greenhalgh T, et al. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Lilienfeld SO, et al. Why many clinical psychologists are resistant to evidence-based practice: root causes and constructive remedies. Clin Psychol Rev. 2013;33(7):883–900.

    Article  PubMed  Google Scholar 

  19. Wisdom JP, et al. Innovation adoption: a review of theories and constructs. Adm Policy Ment Health. 2014;41(4):480–502.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Aarons GA. Transformational and transactional leadership: association with attitudes toward evidence-based practice. Psychiatr Serv. 2006;57(8):1162–9.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Aarons GA, et al. Psychometric properties and U.S. National norms of the Evidence-Based Practice Attitude Scale (EBPAS). Psychol Assess. 2010;22(2):356–65.

    Article  PubMed  Google Scholar 

  22. Aarons GA, et al. Confirmatory factor analysis of the Evidence-Based Practice Attitude Scale in a geographically diverse sample of community mental health providers. Adm Policy Ment Health. 2007;34(5):465–9.

    Article  PubMed  Google Scholar 

  23. Aarons GA, Sawitzky AC. Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychol Serv. 2006;3(1):61–72.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Henggeler SW, et al. Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. J Consult Clin Psychol. 2008;76(4):556–67.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Sousa DS, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. J Eval Clin Pract. 2010;17:268–74.

    Article  PubMed  Google Scholar 

  26. APA Presidential Task Force on Evidence-Based Practice. Evidence-based practice in psychology. Am Psychol. 2006;61(4):271–85.

    Article  Google Scholar 

  27. Psykologforening N. Prinsipperklæring om evidensbasert psykologisk praksis. Tidsskrift Nor Psykologforening. 2007;9:1127–8.

    Google Scholar 

  28. Rhemtulla M, Brosseau-Liard PÉ, Savalei V. When can categorical variables be treated as continuous? A comparison of robust continuous and Categorical SEM estimation methods under suboptimal conditions. Psychol Methods. 2012;17(3):354–73.

    Article  PubMed  Google Scholar 

  29. Jackson DL, Gillaspy JA, Purc-Stephenson R. Reporting practices in confirmatory factor analysis: an overview and some recommendations. Psychol Methods. 2009;14(1):6–23.

    Article  PubMed  Google Scholar 

  30. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Model-a Multidisciplinary J. 1999;6(1):1–55.

    Article  Google Scholar 

  31. Gliem JA, Gliem RR. Calculating, Interpreting, and Reporting Cronbach’s Alpha Reliability Coefficient for Likert-Type Scales. In: Midwest Research to Practice Conference in Adult, Continuing, and Community Education. 2003.

    Google Scholar 

  32. Streiner DL, Norman GR. Health Measurement Scales: A practical guide to their development and use, 5th ed. Oxford: Oxford University Press; 2014.

  33. Aarons GA, Green AE, Miller E. Researching readiness for implementation of evidence-based practice. In: Kelly B, Perkins DF, editors. Handbook of Implementation Science for Psychology in Education. New York: Cambridge University Press; 2012.

  34. Melas CD, et al. Evaluating the properties of the Evidence-Based Practice Attitude Scale (EBPAS) in health care. Psychol Assess. 2012;24(4):867–76.

    Article  PubMed  Google Scholar 

  35. Aarons GA, et al. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implement Sci. 2012;7:56.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Knight D, et al. Designing the optimal JJ-TRIALS study: EPIS as a theoretical framework for selection and timing of implementation interventions. Addict Sci Clin Pract. 2015;10(1):A29.

    Article  PubMed Central  Google Scholar 

  37. Stahmer AC, Aarons GA. Attitudes toward adoption of evidence-based practices: a comparison of autism early intervention providers and children’s mental health providers. Psychol Serv. 2009;6(3):223–34.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Aarons GA, Sommerfeld DH. Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Adolesc Psychiatry. 2012;51(4):423–31.

    Article  PubMed  Google Scholar 

  39. Egeland KM, et al. Psychometric properties of the Norwegian version of the Evidence-Based Practice Attitude Scale (EBPAS): to measure implementation readiness. Health Res Policy Syst. 2016;14:47.

    Article  PubMed  PubMed Central  Google Scholar 

  40. van Sonsbeek MAMS, et al. Psychometric properties of the Dutch version of the Evidence-Based Practice Attitude Scale (EBPAS). Health Res Policy Syst. 2015;13:69.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Ajzen I. The theory of planned behaviour: reactions and reflections. Psychol Health. 2011;26(9):1113–27.

    Article  PubMed  Google Scholar 

  42. Proctor E, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38(2):65–76.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Norwegian Psychological Association and the Norwegian Nurses Organization for kindly giving us access to their members and for the help and assistance in the data gathering. We also would like to thank the members of these organizations for useful comments inspiring us to develop this abbreviated version of the EBPAS. This study was performed within the framework of the Norwegian project for dual competence in psychology: a national effort to educate clinical psychologists (6 year postgraduate degree) with an additional PhD (3 year) and a specialization in clinical psychology (4 year).

Funding

The work in the US was supported by grants from the US National Institute of Mental Health (K01MH001695, R01MH072961, and R01MH092950). The publication fund at the UIT Arctic University of Norway covers author-side payments for Open Access.

Availability of data and materials

Data can be made available by specific request to the authors.

Authors’ contributions

MR had the original idea for the study, translated the EBPAS-50 into Norwegian, performed the data gathering in Norwegian samples, performed the statistical analyses, and drafted the manuscript. OF assisted in the translation and supervised the statistical analyses and drafted the description of the analyses and the results, and edited the final manuscript. IS developed the idea to perform the study in collaboration with MR, assisted in the translation process and data gathering, and edited the final manuscript. GAA raised the idea of comparing results from US and Norwegian samples, reviewed the back-translation, advised on analyses, and edited versions of the manuscript. EMT conducted analyses for the US sample and contributed to editing the manuscript. All authors read and approved the final manuscript.

Authors’ information

Marte Rye is a clinical psychologist (cand.psychol), and a PhD-candidate at the UiT, The Arctic University of Norway, and is also specializing in clinical psychology with adults at the University Hospital of North Norway, to obtain dual competence in clinical psychology.

Oddgeir Friborg is a clinical psychologist and a PhD in psychology and holds a position as professor in health psychology at the UiT The Arctic University of Norway.

Ingunn Skre is a clinical psychologist and a PhD in psychology and holds a position as associate professor at the UiT The Arctic university of Norway, and as consultant psychologist at the University Hospital of North Norway.

Elisa M. Torres is an organizational psychologist and MS in psychology and holds a position as data and project manager in the Department of Psychiatry at the University of California, San Diego, USA.

Gregory Aarons is a clinical psychologist and a PhD in psychology and holds a position as Professor in the Department of Psychiatry at the University of California, San Diego, USA. He is also Director of the Child and Adolescent Services Research Center (CASRC) and Co-Director of the Center for Organizational Research on Implementation and Leadership (CORIL).

Competing interests

GAA is an associate editor of Implementation Science; all decisions on this paper were made by another editor. The authors declare that they have no other competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

The US study was approved by the appropriate institutional review boards prior to recruitment and informed consent was obtained prior to administering surveys. All Norwegian respondents provided informed consent according to recommendations of the Norwegian data protection authority for the project. Approval from Norwegian regional committees for medical and health research ethics was not needed.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

  1. Department of Psychology, Faculty of Health Sciences, UiT The Arctic University of Norway, N-9037, Tromsø, Norway

    Marte Rye, Oddgeir Friborg & Ingunn Skre

  2. Division of Mental Health and Substance Abuse, University Hospital of North Norway, Tromsø, Norway

    Marte Rye & Ingunn Skre

  3. Department of Psychiatry, University of California, San Diego, CA, USA

    Elisa M. Torres & Gregory A. Aarons

Authors
  1. Marte Rye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elisa M. Torres
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oddgeir Friborg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ingunn Skre
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gregory A. Aarons
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marte Rye.

Additional files

Additional file 1:

The Evidence-based Practice Attitude Scale-36 (EBPAS-36), English version PDF. (PDF 96 kb)

Additional file 2:

The Evidence-based Practice Attitude Scale-36 (EBPAS-36), Norwegian version PDF. (PDF 212 kb)

Additional file 3:

The Evidence-based Practice Attitude Scale-36 (EBPAS-36), English version Scoring instructions PDF. (PDF 217 kb)

Additional file 4:

The Evidence-based Practice Attitude Scale-36 (EBPAS-36), Norwegian version Scoring instructions PDF. (PDF 108 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rye, M., Torres, E.M., Friborg, O. et al. The Evidence-based Practice Attitude Scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples. Implementation Sci 12, 44 (2017). https://doi.org/10.1186/s13012-017-0573-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-017-0573-0

Keywords

Implementation Science

ISSN: 1748-5908