Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Measuring determinants of implementation behavior: psychometric properties of a questionnaire based on the theoretical domains framework

  • Johanna M Huijg1Email author,
  • Winifred A Gebhardt1,
  • Elise Dusseldorp2,
  • Marieke W Verheijden2,
  • Nicolette van der Zouwe3,
  • Barend JC Middelkoop4 and
  • Mathilde R Crone4
Implementation Science20149:33

https://doi.org/10.1186/1748-5908-9-33

Received: 24 September 2013

Accepted: 11 March 2014

Published: 19 March 2014

Abstract

Background

To be able to design effective strategies to improve healthcare professionals’ implementation behaviors, a valid and reliable questionnaire is needed to assess potential implementation determinants. The present study describes the development of the Determinants of Implementation Behavior Questionnaire (DIBQ) and investigates the reliability and validity of this Theoretical Domains Framework (TDF)-based questionnaire.

Methods

The DIBQ was developed to measure the potential behavioral determinants of the 12-domain version of the TDF (Michie et al., 2005). We identified existing questionnaires including items assessing constructs within TDF domains and developed new items where needed. Confirmatory factor analysis was used to examine whether the predefined structure of the TDF-based questionnaire was supported by the data. Cronbach’s alpha was calculated to assess internal consistency reliability of the questionnaire, and domains’ discriminant validity was investigated.

Results

We developed an initial questionnaire containing 100 items assessing 12 domains. Results obtained from confirmatory factor analysis and Cronbach’s alpha resulted in the final questionnaire consisting of 93 items assessing 18 domains, explaining 63.3% of the variance, and internal consistency reliability values ranging from .68 to .93. Domains demonstrated good discriminant validity, although the domains ‘Knowledge’ and ‘Skills’ and the domains ‘Skills’ and ‘Social/professional role and identity’ were highly correlated.

Conclusions

We have developed a valid and reliable questionnaire that can be used to assess potential determinants of healthcare professional implementation behavior following the theoretical domains of the TDF. The DIBQ can be used by researchers and practitioners who are interested in identifying determinants of implementation behaviors in order to be able to develop effective strategies to improve healthcare professionals’ implementation behaviors. Furthermore, the findings provide a novel validation of the TDF and indicate that the domain ‘Environmental context and resources’ might be divided into several environment-related domains.

Keywords

Implementation behavior Determinants Theoretical Domains Framework Questionnaire Physical activity interventions

Background

Much research and funding is invested into developing, piloting, and evaluating evidence-based innovations to promote health. However, the transfer of effective innovations, such as pharmacological and behavior change interventions, into routine healthcare practice often does not happen as desired [15]. With the public health impact of these innovations depending on their implementation in practice, it is important to understand healthcare professionals’ (HCP) implementation behaviors and factors associated with suboptimal use of research evidence [6, 7].

Many factors can potentially influence HCPs’ implementation behaviors. These factors may be related to characteristics of the innovation (e.g., compatibility, complexity), social setting (e.g., norms, support), organizational context (e.g., capacity, resources), innovation strategies (e.g., training, reimbursement), patient (e.g., attitudes, compliance), and the individual HCP (e.g., skills, attitudes) [6, 813]. Identifying the key factors associated with HCP implementation behavior can inform the development of strategies to promote evidence-based behavior [6, 1419].

Research has shown that active implementation strategies, such as educational outreach and reminders, can be effective in enhancing implementation behaviors [20, 21]. However, due to the scarce use of theory to inform the choice and design of implementation strategies [22], there is a lack of understanding of why strategies are effective or not [23]. To enhance the effective development of implementation strategies, therefore, many advocate using a theoretical approach to guide the investigation of implementation determinants [14, 17, 2325].

Behavior change theories provide testable hypotheses about when and why specific factors will lead to a certain implementation behavior. However, a limitation in the use of these theories to asses and identify factors underlying HCP implementation behavior is the large number of theories that might be used and their overlapping constructs [12, 2527]. The Theoretical Domains Framework (TDF) [28, 29] is an integrative framework that can be used to overcome this constraint. Within the original TDF [28], constructs from 33 behavior change theories were grouped into 12 domains of behavioral determinants covering the full range of current scientific explanations for human behavior (i.e., ‘Knowledge’, ‘Skills’, ‘Social/professional role and identity’, ‘Beliefs about capabilities’, ‘Beliefs about consequences’, ‘Memory, attention and decision processes’, ‘Environmental context and resources’, ‘Social influences’, ‘Emotion’, ‘Behavioral regulation’, and ‘Nature of the behaviors’). As a consequence, researchers can use this integrative framework instead of having to choose between different theories.

The TDF has instigated a new line of investigation and has been applied in many implementation studies. Specifically, qualitative studies concluded that the TDF was useful for the comprehensive exploration of possible explanations for suboptimal implementation behavior (e.g., [3035]) and for the identification of suitable theories to further investigate these behaviors [27, 36]. Furthermore, the framework was used for the development of questionnaires to assess potential implementation behavior determinants [3739]. So far, however, questionnaires’ internal consistency reliability was insufficient [3739], and only one out of three questionnaires was able to measure the theoretical domains independently [39]. Consequently, there is need for a valid and reliable method to identify theory-based factors influencing HCPs’ implementation behaviors to be able to design effective implementation strategies [12].

Recently, the TDF [28] has been validated, leading to the revised TDF including 14 domains [29]. Main differences between the original and the revised framework include the separation of the domain ‘Optimism’ from the domain ‘Beliefs about capabilities’ and the domain ‘Reinforcement’ from the domain ‘Beliefs about consequences’. Moreover, the domain ‘Motivation and goals’ was divided into two separate domains, i.e., ‘Intentions’ and ‘Goals’, and the domain ‘Nature of the behaviors’ was omitted in the revised framework. As a first step in the development of a TDF-based questionnaire for the valid and reliable assessment of factors influencing HCP implementation behavior, we developed a generic questionnaire assessing the 14 domains of behavioral determinants of the revised TDF [29]. Investigation of questionnaire items’ discriminant content validity based on judgments of a sample of experts on behavior change theory resulted in a questionnaire able to assess all domains discriminately, except for the domains ‘Reinforcement’, ‘Goals’, and ‘Behavioral regulation’. Accordingly, the findings suggested that the 12-domain original version of the TDF [28] might be more applicable in developing a TDF-based questionnaire [40].

The main aim of the current study was to develop a questionnaire based on the 12-domain version of the TDF [28] and to test the psychometric properties of this questionnaire on a sample of HCPs. To validate the Determinants of Implementation Behavior Questionnaire (DIBQ) the following research questions were addressed: 1) does confirmatory factor analysis support the predefined structure of the TDF-based questionnaire (i.e., construct validity); 2) is the questionnaire able to measure TDF domains in a reliable way (i.e., internal consistency reliability); and 3) are the domains of the questionnaire independently measurable (i.e., discriminant validity)? Our specific interest is in HCPs’ implementation of physical activity (PA) interventions, which we used in this study as a field of application for the DIBQ.

Methods

Development of the determinants of implementation behavior questionnaire

We developed a questionnaire that initially included 100 items assessing each of the domains through their related key constructs (see Additional file 1). First, constructs within domains were selected based on:
  1. 1.

    Their conceptual relatedness to the content of the domain (i.e., Knowledge, Skills, Professional role, and Memory);

     
  2. 2.

    Their inclusion in relevant theories frequently used in the field of behavior change (and thus ready access to existing items): the Theory of Planned Behavior [41] (i.e., Perceived behavioral control, Attitude, Subjective norm, and Intention) and Social Cognitive Theory [42] (i.e., Self-efficacy, Outcome expectancies, and Social support);

     
  3. 3.

    The existence of validated scales to measure constructs (i.e., Role clarity, Optimism, Emotions, Action planning, Coping planning, Automaticity); and/or

     
  4. 4.

    Constructs’ relevance to the implementation of PA intervention in routine healthcare by mapping factors resulting from previous research [13, 43] onto the TDF domains (i.e., Reinforcement, Priority, Characteristics of the innovation, Characteristics of the socio-political context, Characteristics of the organization, Characteristics of the participants, Characteristics of the innovation strategy, Descriptive norm).

     

Second, for each domain a minimum of two and a maximum of 24 items were developed, with an average of 4 items for each construct. Items were related to the target behavior ‘delivering PA interventions following the guidelines’. Items measuring the constructs within the domains ‘Knowledge’, ‘Beliefs about capabilities’, ‘Social influences’, ‘Emotion’, ‘Behavioral regulation’, and ‘Nature of the behaviors’ [37, 41, 42, 4449] were adapted from previously published questionnaires. The content of these items was based on previous research on factors influencing the implementation of PA intervention in routine healthcare [13, 43]. For instance, items measuring the constructs Self-efficacy [41] and Coping planning [47] were developed so that they included HCPs’ barriers of lack of time and patient motivation. Items measuring constructs within the domains ‘Skills’, ‘Social/professional role and identity’, ‘Memory, attention, and decision processes’ were based on results of the discriminant content validity study [40]. With regard to the domain ‘Beliefs about consequences’, items measuring the constructs Attitude [41] and Outcome expectancies [42] were adapted from previously published questionnaires, whereas items measuring the construct Reinforcement were newly developed (as none could be located in the literature). Regarding the domain ‘Motivation and goals’, items measuring the construct Intention were adapted from a previously published questionnaire [41], while items were newly developed for the construct Priority. Furthermore, new items were created for the domain ‘Environmental context and resources’. New items were developed based on discussions between WAG, MRC, and JMH. These discussions were informed by the academic literature on the concept and definition of specific domains and constructs, questions to identify behavior change processes as formulated by Michie et al. [28], and themes emerging from interviews on the implementation of PA interventions [43]. Finally, the questionnaire was piloted among five colleague researchers and a sample of eight physical therapists. Piloting indicated that the questionnaire was easily understood and well received by the respondents.

Respondents and procedure

We recruited physical therapists delivering PA interventions to a variety of target groups (i.e., people with chronic obstructive pulmonary disease, diabetes, arthritis or obesity). They were recruited through physical therapist associations and contacted opportunistically via their practice websites. Physical therapists were sent an email including the link to the online questionnaire and were assured that their responses would be confidential and anonymous. They reported on their gender, age, practice experience, sort of practice/workplace, and the socioeconomic status (SES) of the majority of their intervention participants. Full questionnaire completion was rewarded with a 25 euro voucher. Non-respondents were sent an email with a questionnaire on their demographic characteristics.

Data management

Questionnaires were exported from Qualtrics software, version 45433 [50] to IBM SPSS Statistics version 19.0 [51] for analyses. Responses were scored from 1 (strongly disagree) to 7 (strongly agree). Items worded negatively, such as ‘Delivering [PA intervention] following the guidelines is something I often forget’, were reverse-coded. For the six social support items, it was possible to fill in ‘Not applicable’, because not all physical therapists work together with others in delivering PA interventions, and some are part of the management of their organization and therefore do not receive management support. Scores on this category were recoded as missing.

Data analyses

Confirmatory factor analyses

Confirmatory factor analysis was used to examine whether the a priori assignment of items to Michie et al.’s [28] TDF domains was supported by the data (i.e., research question 1). To perform the confirmatory factor analysis, we used the oblique multiple group (OMG) method [52, 53], which has been previously shown to perform better or to be highly comparable to the more well-known confirmatory common factor analysis [5456]. The OMG method involves calculating correlations between items and domains, from which the following conclusions are drawn: if an item correlates highest with the domain the item was assigned to, the item is correctly assigned to the domain (and the predefined structure is confirmed); if an item correlates highest with a domain the item was not assigned to, the item is incorrectly assigned to the domain (and the predefined structure is not confirmed). In the OMG method, correlations between items and domains are corrected for self-correlation and test length [52].

When an item is assigned incorrectly, adjustments should be made. We used the iterative OMG procedure to make adjustments to the structure of our questionnaire. This step-wise procedure involves testing the adjusted assignment obtained from an OMG analysis in a subsequent OMG analysis on the same data set, which will either support the assignment or provide suggestions for new adjustments. When, based on these suggestions, a new adjustment is made, this assignment can be tested again on the same data set. The iterative procedure continues until the adjusted assignment is supported by the data (i.e., items correlate highest with the domain they are assigned to; the adjustment leads to a higher total explained variance) or when none of the adjusted assignments are supported by the data and a newly obtained adjusted assignment is equal to one of the previously assignments. Preferably, changes in item assignment can be justified by a theoretical or conceptual link between the incorrect assigned item and the domain to which it has been assigned [54].

In this study, the iterative procedure of adjustment consisted of two iterations. In the first iteration, adjustments were made based on suggestions from the OMG analyses and theoretical or conceptual links between items and domains. In the second iteration, adjustments were also based on suggestions from the OMG analyses and theoretical or conceptual assumptions. In addition, we compared poor fitting domains from the OMG solution to the solution based on exploratory factor analysis (i.e., principal component analysis; PCA [57]) to guide adjustments of the assignment of items to domains. Following the iterative OMG procedure, adjustments were only retained when they were supported by the new results from the OMG analysis. Finally, the variance-accounted-for by the adjusted predefined components was compared to the variance-accounted-for by the components resulting from the PCA. Preferably this difference is small, which indicates that the adjusted predefined structure fits the data well.

Internal consistency reliability and discriminant validity

Cronbach’s alpha [58] was computed to assess the internal consistency reliability of the items assessing each domain (i.e., research question 2). Two tests of discriminant validity [59] were undertaken to assess if the DIBQ was able to measure the TDF domains discriminately (i.e., research question 3). First, discriminant validity was assessed by determining whether the bootstrapped 95% confidence interval around Pearson’s correlations between domains included 1.00 [60]. Second, we calculated attenuation-corrected correlations to discover the ‘true correlation’ between the domains [61].

Computational note

The analyses were performed using IBM SPSS statistics version 19.0 [51]. For the OMG analyses, we used a SPSS-macro file obtained from Timmerman and Stuive [62]. Attenuation-corrected correlations were calculated using the R software environment [63] using the R-package Psy [64].

Ethics

The Medical Ethics Committee of the Leiden University Medical Centre granted ethical approval of this study (reference number NV/CME 09/081).

Results

Characteristics of the respondents

Of the 496 physical therapists who were invited for the study, 274 (55.2%) delivering 15 different PA interventions completed the questionnaire. The number of questionnaires analyzed was 270, following removal of physical therapists reporting no experience with PA intervention delivery. Table 1 shows characteristics of respondents and non-respondents. Of the respondents, 58.1% (n = 157) were female, they were on average 39.7 (SD = 12.3) years old, and had on average 14.9 (SD = 11.3) years of practice experience. Most of them worked in a group practice (68.5%, n = 185), and most delivered PA interventions to an equal percentage of participants with a low and high SES (53%, n = 143) or to people with a low SES (44.8%, n = 121). A total of 68 out of 222 non-respondents (30.6%) filled in the non-respondents questionnaire. Comparisons between respondents and non-respondents indicated that the latter were significantly older and had more practice experience.
Table 1

Demographic characteristics of respondents and non-respondents

Demographic variable

Respondents (N = 270)

Non-respondents (N = 68)

Mean

(SD)

n

(%)

Mean

(SD)

n

(%)

Gender

        

Male

  

113

(12.3)

  

27

(38.6)

Female

  

157

(58.1)

  

39

(55.7)

Age

39.7

(12.3)*

  

45.6

(11.7)*†

  

Practice experience (years)

14.9

(11.3)*

  

19.8

(11.8)*‡

  

Sort of practice/workplace

        

Solo practice

  

7

(2.6)

  

3

(4.3)†

Duo practice

  

9

(3.3)

  

1

(1.4)†

Group practice

  

185

(68.5)

  

36

(51.4)†

Multidisciplinary HC center

  

61

(22.6)

  

11

(15.7)†

Other

  

8

(3.0)

  

4

(5.7)†

SES intervention participants

        

Mostly high SES

  

6

(2.2)

  

4

(5.7)†

50-50

  

143

(53.0)

  

30

(42.9)†

Mostly low SES

  

121

(44.8)

  

21

(30.0)†

Note. Results of chi-square tests and independent t-tests are reported; *,p <0.05; †, based on N = 55; ‡, based on N = 54; HC: Healthcare; SES: Socioeconomic status.

Psychometric properties of the questionnaire

Confirmatory factor analysis

OMG analyses showed that the total variance explained by the initial questionnaire was 48.0%. In other words, the initial assignment of the items to the 12 domains of the TDF explained about half of the total variance in item scores. In the first iteration of adjustments, results of the OMG analysis indicated that model fit could be improved by adjusting the domains ‘Environmental context and resources’ and ‘Beliefs about capabilities’. Based on Fleuren et al.’s [8] categorization of innovation determinants into factors related to the innovation, socio-political context, organization, innovation strategy, and Chaudoir et al.’s [12] additional category of factors related to the patient, the first adjustment of the questionnaire included dividing the domain ‘Environmental context and resources’ into the domains ‘Innovation’, ‘Socio-political context’, ‘Organization’, ‘Patient’, and ‘Innovation strategy’. This process was done in five subsequent steps (in each step, one new domain was entered), with every step leading to a higher total explained variance, validating the adjustment. With regard to the domain ‘Beliefs about capabilities’, the constructs Self efficacy and Perceived behavioral control did not fit well with the conceptually different ‘Optimism’ items, and therefore ‘Optimism’ items were assigned to a standalone domain. Subsequently, this adjustment was supported by the results of the re-run of the OMG analysis.

In the second iteration, further improvement of model fit was informed by comparing the poor fitting domains from the OMG solution with the solution from the PCA. This led to the assignment of items measuring social support from the management to the domain ‘Organization’, and ‘Priority’ items to a separate domain. Furthermore, the domain ‘Emotion’ was divided into two domains (i.e., ‘Negative emotions’ and ‘Positive emotions’) and items measuring the domain ‘Memory, attention, and decision processes’ and the construct Automaticity were combined in the ‘Nature of the behaviors’ domain. Again, these adjustments were validated by re-running the OMG analyses.

For each of the resulting 18 domains, a Cronbach’s alpha was computed. Investigation of ‘alpha, if item deleted’ values revealed that seven items could be deleted. These were one item measuring the domain ‘Priority’, one item measuring the domain ‘Innovation’, three items measuring the domain ‘Organization’, one item measuring the domain ‘Socio-political context’, and one item measuring the domain ‘Patient’. After these adjustments, the final questionnaire included 93 items assessing 18 domains (see Table 2). Definitions of these domains are shown in Table 3. In addition, OMG results showed that the total variance explained by the domains was increased with more than 15% to 63.3%. The variance-accounted-for by the structure of the questionnaire as we built it differed 4.7% with the variance-accounted-for by the components resulting from the PCA. This can be considered a small difference [65], indicating that the predefined (and adjusted) structure fits the data well. A comparison between the initial and the final questionnaire is shown in Table 4.
Table 2

Final questionnaire

Domains

Constructs

Items

Source

D1

Knowledge

Knowledge (1)

I know how to deliver [PA intervention] following the guidelines.

Adapted from Amemori et al. [37]

  

Role clarity (3)

Objectives of [PA intervention] and my role in this are clearly defined for me.

Adapted from Wännström [66]

   

With regard to [PA intervention], I know what my responsibilities are.

 
   

In my work with [PA intervention], I know exactly what is expected from me.

 

D2

Skills

Skills (3)

I have been trained in delivering [PA intervention] following the guidelines.

New items

   

I have the skills to deliver [PA intervention] following the guidelines.

 
   

I am practiced to deliver [PA intervention] following the guidelines.

 

D3

Social/professional role and identity

Professional role (3)

Delivering [PA intervention] following the guidelines is part of my work as a PT.

New items

   

As a PT, it is my job to deliver [PA intervention] following the guidelines.

 
   

It is my responsibility as a PT to deliver [PA intervention] following the guidelines.

 

D4

Beliefs about capabilities

Self-efficacy (4)

I am confident that I can deliver [PA intervention] following the guidelines.

Adapted from Content based on Huijg, van der Zouwe et al. [43] and Huijg, Gebhardt et al. [13]

   

I am confident that I can deliver [PA intervention] following the guidelines even when other professionals with whom I deliver [PA intervention] do not do this.

 
   

I am confident that I can deliver [PA intervention] following the guidelines even when there is little time.

 
   

I am confident that I can deliver [PA intervention] following the guidelines even when participants are not motivated.

 
  

Perceived behavioral control (7)

I have control over delivering [PA intervention] following the guidelines.

Adapted from Ajzen [41]

   

For me, delivering [PA intervention] following the guidelines is (very difficult – very easy).

 
   

For me, performing the intake is (very difficult – very easy).

 
   

For me, delivering the training program is (very difficult – very easy).

 
   

For me, performing the evaluation is (very difficult – very easy).

 
   

For me, giving attention to participant’s maintenance of PA behavior outside [PA intervention] is (very difficult – very easy).

 
   

For me, reporting about the [PA intervention] to the referring professional is (very difficult – very easy).

 

D5

Optimism

Optimism (3)

In my work as a PT, in uncertain times, I usually expect the best.

Adapted from Scheier et al. [48]

   

In my work as a PT, I’m always optimistic about the future.

 
   

In my work as a PT, overall, I expect more good things to happen than bad.

 

D6

Beliefs about consequences

Attitude (4)

For me, delivering [PA intervention] following the guidelines is (not useful at all – very useful).

Adapted from Ajzen [41]

   

For me, delivering [PA intervention] following the guidelines is (not worthwhile at all – very worthwhile).

 
   

For me, delivering [PA intervention] following the guidelines is (not pleasurable at all – very pleasurable).

 
   

For me, delivering [PA intervention] following the guidelines is (not interesting at all – very interesting).

 
  

Outcome expectancies (5)

If I deliver [PA intervention] following the guidelines, [PA intervention] will be most effective.

Adapted from Bandura [42]

   

If I deliver [PA intervention] following the guidelines, participants will appreciate this.

 
   

If I deliver [PA intervention] following the guidelines, this will strengthen the collaboration with professionals with whom I deliver [PA intervention].

Content based on Huijg, van der Zouwe et al. [43] and Huijg, Gebhardt et al. [13]

   

If I deliver [PA intervention] following the Guidelines, I will feel satisfied.

 
   

If I deliver [PA intervention] following the Guidelines, it will help participants to be more physically active.

 
  

Reinforcement (3)

When I deliver [PA intervention] following the guidelines, I get financial reimbursement.

New items

   

When I deliver [PA intervention] following the guidelines, I get recognition from the work context.

Content based on Huijg, van der Zouwe et al. [43] and Huijg, Gebhardt et al. [13]

   

When I deliver [PA intervention] following the guidelines, I get recognition from participants.

 

D7

Intentions

Intention (3)

I intend to deliver [PA intervention] following the guidelines in the next three months.

Adapted from Ajzen [41]

   

I will definitely deliver [PA intervention] following the guidelines in the next three months.

 
   

How strong is your intention to deliver [PA intervention] following the guidelines in the next three months?

 

D8

Goals

Priority (2)

How often is working on something else on your agenda a higher priority than delivering [PA intervention] following the guidelines?

New items

   

How often is working on something else on your agenda more urgent than delivering [PA intervention] following the guidelines?

 

D9

Innovation

Innovation characteristics (5)

It is possible to tailor [PA intervention] to participants’ needs?

New items

   

It is possible to tailor [PA intervention] to professionals’ needs?

Content based on Rogers [11]

   

[PA intervention] costs little time to deliver.

 
   

[PA intervention] is compatible with daily practice.

 
   

[PA intervention] is simple to deliver.

 

D10

Socio-political context

Socio-political context (3)

Government and local authorities provide sufficient support to interventions such as [PA intervention].

New items

   

Insurance companies provide sufficient support to interventions such as [PA intervention].

Content based on Huijg, van der Zouwe et al. [43] and Huijg, Gebhardt et al. [13]

   

PHC is sufficiently oriented towards prevention.

 

D11

Organization

Organizational resources and support (4)

In the organization I work, all necessary resources are available to deliver [PA intervention].

New items

   

I can count on support from the management of the organization I work in, when things get tough guidelines.

Content based on Huijg, van der Zouwe et al. [43] and Huijg, Gebhardt et al. [13]

   

The management of the organization I work in is willing to listen to my problems with delivering [PA intervention] following the guidelines.

 
   

The management of the organization I work in is helpful with delivering [PA intervention] following the guidelines.

 

D12

Patient

Patient characteristics (2)

Participants of [PA intervention] are motivated.

New items

   

Participants of [PA intervention] are positive about [PA intervention].

Content based on Huijg, van der Zouwe et al. [43] and Huijg, Gebhardt et al. [13]

D13

Innovation strategy

Innovation strategies (7)

[Implementing organization] provides professionals with a training to deliver [PA intervention].

New items

   

[Implementing organization] provides the possibility to experience delivering [PA intervention] before professionals need to commit to it.

Content based on Huijg, van der Zouwe et al. [43] and Huijg, Gebhardt et al. [13]

   

[Implementing organization] provides sufficient intervention materials.

 
   

[Implementing organization] provides assistance to professionals with delivering [PA intervention].

 
   

[Implementing organization] organizes intervision meetings for professionals.

 
   

[Implementing organization] provides sufficient financial reimbursement to professionals for [PA intervention] delivery.

 
   

[Implementing organization] provides insights into results of [PA intervention].

 

D14

Social influences

Subjective norm (2)

Most people who are important to me think that I should deliver [PA intervention] following the guidelines.

Adapted from Ajzen [41]

   

Professionals with whom I deliver [PA intervention] think I should deliver [PA] intervention] following the guidelines.

 
  

Descriptive norm (2)

Professionals with whom I deliver [PA intervention] deliver [PA] intervention following the guidelines.

Adapted from Cialdini et al. [49]

   

Other professionals who work with [PA intervention] deliver [PA intervention] following the guidelines.

 
  

Social support (3)

I can count on support from professionals with whom I deliver [PA intervention] when things get tough around delivering [PA intervention] following the guidelines.

Adapted from Frese [45]

   

Professionals with whom I deliver [PA intervention] are willing to listen to my problems with delivering [PA intervention] following the guidelines.

 
   

Professionals with whom I deliver [PA intervention] are helpful with delivering [PA intervention] following the guidelines.

 

D15

Positive emotions

Positive emotions (6)

When I work with [PA intervention] I feel optimistic.

Adapted from van Veldhoven et al. [44]

   

When I work with [PA intervention] I feel comfortable.

 
   

When I work with [PA intervention] I feel calm.

 
   

When I work with [PA intervention] I feel relaxed.

 
   

When I work with [PA intervention] I feel cheerful.

 
   

When I work with [PA intervention] I feel elated.

 

D16

Negative emotions

Negative emotions (6)

When I work with [PA intervention] I feel nervous.

Adapted from van Veldhoven et al. [44]

   

When I work with [PA intervention] I feel pessimistic.

 
   

When I work with [PA intervention] I feel depressed.

 
   

When I work with [PA intervention] I feel agitated.

 
   

When I work with [PA intervention] I feel sad.

 
   

When I work with [PA intervention] I feel uncomfortable.

 

D17

Behavioral regulation

Action planning (3)

I have a clear plan of how I will deliver [PA intervention] following the guidelines.

Adapted from Sniehotta et al. [47]

   

I have a clear plan under what circumstances I will deliver [PA intervention] following the guidelines.

 
   

I have a clear plan when I will deliver [PA intervention] following the guidelines.

 
  

Coping planning (3)

I have a clear plan with regard to delivering [PA intervention] following the guidelines when participants are not motivated.

Adapted from Sniehotta et al. [47]

   

I have a clear plan with regard to delivering [PA intervention] following the guidelines when there is little time.

Content based on Huijg, van der Zouwe et al. [43] and Huijg, Gebhardt et al. [13]

   

I have a clear plan with regard to delivering [PA intervention] following the guidelines when other professionals with whom I deliver [PA intervention] do not do this.

 

D18

Nature of the behaviors

Automaticity (4)

Delivering [PA intervention] following the guidelines is something I do automatically.

Gardner et al. [46]

   

Delivering [PA intervention] following the guidelines is something I do without having to consciously remember.

 
   

Delivering [PA intervention] following the guidelines is something I do without thinking.

 
   

Delivering [PA intervention] following the guidelines is something I start doing before I realize I am doing it.

 
  

Memory (2)

Delivering [PA intervention] following the guidelines is something I seldom forget.

New items

   

Delivering [PA intervention] following the guidelines is something I often forget.

 

Note. PA: Physical activity; PT: Physical therapist; PHC: Primary healthcare.

Table 3

Domain definitions

Domain

Definition

D1

Knowledge

An awareness of the existence of something.

D2

Skills

An ability or proficiency acquired through practice.

D3

Social/professional and role and identity

A coherent set of behaviors and displayed personal qualities of an individual in a social or work setting.

D4

Beliefs about capabilities

Acceptance of the truth, reality, or validity about an ability, talent, or facility that a person can put to constructive use.

D5

Optimism

The confidence that things will happen for the best or that desired goals will be attained.

D6

Beliefs about consequences

Acceptance of the truth, reality, or validity about outcomes of a behavior in a given situation.

D7

Intentions

A conscious decision to perform a behavior or a resolve to act in a certain way.

D8

Goals

Mental representations of outcomes or end states that an individual wants to achieve.

D9

Innovation

Any characteristics of the innovation that discourages or encourages the development of skills and abilities, independence, social competence, and adaptive behavior.

D10

Socio-political context

Any characteristics of the socio-political context that discourages or encourages the development of skills and abilities, independence, social competence, and adaptive behavior.

D11

Organization

Any characteristics of the organization that discourages or encourages the development of skills and abilities, independence, social competence, and adaptive behavior.

D12

Patient

Any characteristics of the patient that discourages or encourages the development of skills and abilities, independence, social competence, and adaptive behavior.

D13

Innovation strategy

Any characteristics of the innovation strategy that discourages or encourages the development of skills and abilities, independence, social competence, and adaptive behavior.

D14

Social influences

Those interpersonal processes that can cause individuals to change their thoughts, feelings, or behaviors.

D15

Positive emotions

A complex positive reaction pattern, involving experiential, behavioral, and physiological elements, by which the individual attempts to deal with a personally significant matter or event.

D16

Negative emotions

A complex negative reaction pattern, involving experiential, behavioral, and physiological elements, by which the individual attempts to deal with a personally significant matter or event.

D17

Behavioral regulation

Anything aimed at managing or changing objectively observed or measured actions.

D18

Nature of the behaviors

The nature of the aggregate of all responses made by an individual in any situation.

Note. All domain definitions, except for the definition of the domain ‘Nature of the behaviors,’ were based on definitions from Cane et al. [29], who derived their definitions from the American Psychological Associations’ Dictionary of Psychology [67].

Table 4

Comparison between initial and final questionnaire

Domains

Item numbers

Amount of items

Cronbach’s alpha

Explained variance OMG method

Explained variance PCA

(from initial questionnaire)

D1

Knowledge

1 - 4

4

.93

48.0%

56.5%

D2

Skills

5 - 7

3

.86

  

D3

Social/ professional role and identity

8 - 10

3

.90

  

D4

Beliefs about capabilities

11 - 24

14

.84

  

D5

Beliefs about consequences

25 - 36

12

.83

  

D6

Motivation and goals

37 - 42

6

.77

  

D7

Memory, attention, and decision p.

43 - 44

2

.71

  

D8

Environmental context and res.

45 - 68

24

.82

  

D9

Social influences

69 - 78

10

.84

  

D10

Emotion

79 - 90

12

.88

  

D11

Behavioral regulation

91 - 96

6

.79

  

D12

Nature of the behaviors

97 - 100

4

.85

  

D1

Knowledge

1 - 4

4

.93

63.3%

68.0%

D2

Skills

5 - 7

3

.86

  

D3

Social/ professional role and identity

8 - 10

3

.90

  

D4

Beliefs about capabilities

11 - 21

11

.84

  

D5

Optimism

22 - 24

3

.79

  

D6

Beliefs about consequences

25 - 36

12

.83

  

D7

Intentions

37 - 39

3

.91

  

D8

Goals

40, 41

2

.88

  

D9

Innovation

45 - 49

5

.68

  

D10

Socio-political context

51 - 53

3

.73

  

D11

Organization

57, 76 - 78

4

.85

  

D12

Patient

60, 61

2

.74

  

D13

Innovation strategy

62 - 68

7

.82

  

D14

Social influences

69 - 75

7

.86

  

D15

Positive emotions

80, 82, 84, 87, 89, 90

6

.85

  

D16

Negative emotions

79, 81, 83, 85, 86, 88

6

.85

  

D17

Behavioral regulation

91 - 96

6

.79

  

D18

Nature of the behaviors

43, 44, 97 - 100

6

.85

  

Note. OMG method, oblique multiple group method; PCA, principal component analysis.

Internal consistency reliability and discriminant validity

Internal consistency reliability values for the 18 domains of the final questionnaire ranged from .68 for the domain ‘Innovation’ (i.e., the only domain with an alpha < .70) to .93 for the domain ‘Knowledge’. None of the bootstrapped 95% confidence intervals around Pearson’s correlations included 1.00, indicating sufficient discriminant validity (for an overview of all correlations between domains, see Additional file 2). In addition, we found high attenuation-corrected correlations between the domains ‘Knowledge’ and ‘Skills’ (r = .80) and the domains ‘Skills’ and ‘Social/professional role and identity’ (r = .86), which suggests overlap between these domains (see Additional file 3).

Discussion

We developed and tested a questionnaire assessing factors influencing HCPs’ implementation behaviors that was based on a theoretical framework of behavioral determinants [28]. The DIBQ was one of the first TDF-based questionnaires that was developed in a rigorous manner, and showing very good psychometric properties. That is, it had good construct validity, and the majority of domains showed high internal consistency reliability and discriminant validity. While our focus was on the measurement of factors influencing the implementation of PA interventions in PHC, we suggest that the DIBQ can be applied more broadly, as the questionnaire can easily be adapted to other contexts in which implementation research takes place. Consequently, the DIBQ can solve previously reported problems with the measurement of theory-based factors underlying HCP behavior [12, 2527]. This can contribute to the development of effective implementation strategies and subsequently the impact of evidence-based interventions.

With regard to the questionnaire’s construct validity, our findings supported the majority of the predefined structure of the questionnaire that was based on the 12 domains of the TDF [28]. They correspond with Taylor et al. [39, 68], who found good discriminant validity of TDF domains in a questionnaire measuring influences on patient safety behaviors [39] and in the Determinants of Physical Activity Questionnaire [68]. These results provide an additional level of validation for the content of the TDF, and they confirm the viability of using the framework for construction of a theory-based questionnaire. Nevertheless, the questionnaire’s construct validity could be enhanced by some adjustments in content of the domains and the structure of the questionnaire to 18 domains.

The main adjustment we made to the structure of the questionnaire was dividing the domain ‘Environmental context and resources’ into five different environment-related domains: ‘Innovation’, ‘Socio-political context’, ‘Organization’, ‘Patient’, and ‘Innovation strategy’. This adjustment is consistent with leading theoretical models on the introduction of innovations in healthcare [6, 812]. Replication of this domain-structure in future research may suggest including five different environment-related domains in the TDF. Next, ‘Optimism’ items were separated from the domain ‘Beliefs about capabilities’. This separation makes sense because ‘Optimism’ items were measured as a general disposition (e.g., ‘In my work as a physical therapist, in uncertain times, I usually expect the best’), whereas ‘Beliefs about capabilities’ items concerned capabilities that are required to achieve a specific outcome (e.g., ‘I am confident that I can deliver [PA intervention] following the guidelines’). Furthermore, the adjustment corresponds with the results of the recent validation of the TDF [29]. Items measuring social support from the management were assigned to the domain ‘Organization’ and ‘Priority’ items were separated from ‘Intention’ items. The first adjustment could also be justified by conceptual links between items and domains, and the latter adjustment corresponded with results of the validated TDF [29]. In addition, dividing the domain ‘Emotion’ into the domains ‘Positive emotions’ and ‘Negative emotions’ could be explained by previous research that indicated that positive and negative affect are two relatively independent constructs that can be measured discriminately [69, 70]. Based on similarities in their content, items measuring the domain ‘Memory, attention, and decision processes’ and Automaticity items were merged into the domain ‘Nature of the Behaviors’. Moreover, the link between automatic behaviors and memory was highlighted by Wood and Neal [71]. When developing a TDF-based questionnaire, it is possible that adding questions on attention and decision making to the memory items might decrease the overlap between the domains ‘Memory, attention, and decision processes’ and ‘Nature of the Behaviors’. Finally, some items measuring the domains ‘Priority’, ‘Innovation’, ‘Organization’, ‘Socio-political context’, and ‘Patient’ were deleted based on the domains’ Cronbach’s alpha values. An explanation based on the content of these items could not be found; however, lack of internal consistency reliability of the domains ‘Priority’, ‘Innovation’, ‘Organization’, ‘Socio-political context’, and ‘Patient’ might be related to the fact that the items measuring these domains were all newly developed. This suggests that items measuring the domain ‘Environmental context and resources’ can be improved (see Chaudoir et al. [12] for an overview of measures assessing these domains related to the environment).

No adjustments were needed for five out of the 12 domains of the initial questionnaire: ‘Knowledge’, ‘Skills’, ‘Social/professional role and identity’, ‘Beliefs about consequences’, and ‘Behavioral regulation’. This might be explained by the use of previously published questionnaires for the development of ‘Knowledge’ and ‘Behavioral regulation’ items, and most of the ‘Beliefs about consequences’ items. Furthermore, items measuring the domains ‘Skills’ and ‘Social/professional role and identity’ were validated by the discriminant content validity study [40]. Noticeably, the ‘Knowledge’ item ‘I know how to…,’ ‘Reinforcement’ items, and items measuring the construct Action Planning performed well, while they could not be validated by the discriminant content validity study [40]. This might be explained by the divergence in the main aims of the two studies; the increased focus on differences between individual items when investigating items’ discriminant content validity, and the emphasis on similarities between groups of items when examining a questionnaire’s construct validity. Indeed, in the present study, items that were not validated in the discriminant content validity study were surrounded by other previously validated items.

Compared to three other studies using a TDF-based questionnaire to identify implementation behavior determinants [3739], our questionnaire demonstrated high internal consistency reliability for the majority of domains. Explanations for this might be the lower number of items that the previous studies used to measure each domain [3739] and the development of items for domains instead of constructs within domains [38, 39]. Furthermore, it is not clear to what extent Beenstock et al. [38] and Taylor et al. [39] used items from previously published questionnaires.

Although OMG analyses revealed sufficient discriminant validity on item level, attenuation-corrected correlations revealed overlap between the domains ‘Knowledge’ and ‘Skills’ and ‘Skills’ and ‘Social/professional role and identity’. On the other hand, bootstrapped 95% confidence intervals around correlations suggested that the questionnaire was able to measure TDF domains discriminately. Based on these results and the different content of the domains, we did not merge them into one single domain. However, high correlations between domains might be problematic when analyzing associations between domains and outcome variables taking a multivariate approach.

While our focus was on the measurement of factors influencing HCPs’ implementation of PA interventions, the questionnaire was designed to be easily adaptable so it can be used in studies investigating implementation behaviors performed by other HCPs in other settings. However, depending on the behavior, the implementing HCP, and the context, it may be necessary to include items for specific barriers and facilitators. For example, time, patient motivation, and financial support may play a role in the delivery of PA interventions by physical therapists, while these factors might not relate to other behaviors, HCPs and settings. Moreover, validity and reliability of use of the questionnaire for other behaviors, HCPs and settings needs further investigation.

Some limitations of this study need to be taken into consideration when interpreting the results. First, respondents were physical therapists delivering PA interventions to a variety of target groups. In this study, we did not distinguish between the different PA interventions. Our results suggest sufficient internal validity of the DIBQ. However, a question remains as to whether the structure of the DIBQ holds for every specific PA intervention. In this study, small sample sizes within each PA intervention (sample sizes varied from 4 to 101) hindered the performance of confirmatory factor analysis for each PA intervention separately. A recommendation for future applications of the DIBQ is to replicate the reliability analysis for the target group at hand. Second, the questionnaire assessed TDF domains through their related constructs. However, to develop a questionnaire that is of an acceptable length to fill in, only a selection of constructs could be measured. Although the selection of key-constructs was based on previous research on factors influencing the implementation of PA interventions in primary healthcare [13, 43], it could be that some of the domains’ key-constructs are not part of the questionnaire leading to decreased validity of the measurement of domains. For example, the construct Intrinsic motivation [72] was not included to measure the domain ‘Motivation and goals’ and the construct Burnout [73] was not included to measure the domain ‘Emotion’, although we know from previous research that these are important determinants for HCPs’ evidence-based practice [74, 75]. Nevertheless, a questionnaire including 93 items might still be too long to fill in. This could also be an explanation for the 55.2% response rate, which was comparable to previous reported response rates of 54% [76] and 57% [77] in surveys among physical therapists, but can be considered low in comparison to Barrett et al. [78], who reached a response rate of 88%. A next step in the development process could be to develop a shorter version of the DIBQ and assess its psychometric properties. One strategy to decrease the amount of items would be to select items measuring the domains directly, instead of through their related key construct. Taking into account the criterion for a reliable component (i.e., at least three items with a loading above .80 [79]), this could decrease the average of 4 items for each construct to 4 items for each domain. The results of the discriminant content validity study [40] may guide the selection of items in order to obtain a shortened version of the questionnaire. Comparisons between respondents and non-respondents indicated that the latter were significantly older and had more practice experience, which limits the generalizability of our results. Finally, the methods used to validate our questionnaire were limited to factor analyses and the examination of discriminant validity of the domains, and only internal consistency reliability was assessed. Future research should also investigate items’ predictive validity and test-retest reliability of the questionnaire.

Conclusions

This study describes the development and initial validation of the DIBQ. The questionnaire showed good construct validity (i.e., research question 1) and the majority of domains showed high internal consistency reliability (i.e., research question 2) and discriminant validity (i.e., research question 3). Therefore, the questionnaire is viable to measure potential determinants of implementation behavior in a theory-based and comprehensive way. The identification of factors influencing implementation behaviors provides important information on what factors should be targeted when designing strategies to promote the effective implementation of interventions [6, 1419]. This is highly likely to increase the impact of health behavior change interventions. Future studies on the psychometric properties of the questionnaire are warranted and should go beyond construct validity, internal consistency reliability, and discriminant validity. In addition, more research is needed to understand the strengths and limitations of the questionnaire when it is used for other behaviors among other HCPs and in other settings.

Consent

In our study, completion of the questionnaire indicated participants’ consent for their participation in the study.

Abbreviations

HCP: 

Healthcare professional

TDF: 

Theoretical domains framework

PA: 

Physical activity

SES: 

Socioeconomic status.

Declarations

Acknowledgements

This research was funded by ZonMw, The Netherlands Organisation of Health Research and Development.

Authors’ Affiliations

(1)
Clinical, Health and Neuropsychology, Leiden University
(2)
Netherlands Organization for Applied Scientific Research (TNO), Leiden
(3)
Regional Public Health Service Hollands Midden
(4)
Department of Public Health and Primary Care, Leiden University Medical Center

References

  1. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE: Knowledge translation of research findings. Implement Sci. 2012, 7: 50-10.1186/1748-5908-7-50.View ArticlePubMedPubMed CentralGoogle Scholar
  2. Glasgow RE, Lichtenstein E, Marcus AC: Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003, 93: 1261-1267. 10.2105/AJPH.93.8.1261.View ArticlePubMedPubMed CentralGoogle Scholar
  3. Haines A, Kuruvilla S, Borchert M: Policy and practice bridging the implementation gap between knowledge and action for health. Bull World Health Organ. 2004, 82: 724-732.PubMedPubMed CentralGoogle Scholar
  4. Glasgow R, Klesges L, Dzewaltowski DA, Bull SS, Estabrooks PA: The future of physical activity behavior change research: what is needed to improve translation of research into health promotion practice?. Ann Behav Med. 2004, 27: 3-12. 10.1207/s15324796abm2701_2.View ArticlePubMedGoogle Scholar
  5. Eccles MP, Armstrong D, Baker R, Cleary K, Davies H, Davies S, Glasziou P, Ilott I, Kinmonth A-L, Leng G, Logan S, Marteau T, Michie S, Rogers H, Rycroft-Malone J, Sibbald B: An implementation research agenda. Implement Sci. 2009, 4: 18-10.1186/1748-5908-4-18.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003, 362: 1225-1230. 10.1016/S0140-6736(03)14546-1.View ArticlePubMedGoogle Scholar
  7. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S: A conceptual framework for implementation fidelity. Implement Sci. 2007, 2: 40-10.1186/1748-5908-2-40.View ArticlePubMedPubMed CentralGoogle Scholar
  8. Fleuren M, Wiefferink K, Paulussen T: Determinants of innovation within health care organizations: literature review and Delphi study. Int J Qual Health Care. 2004, 16: 107-123. 10.1093/intqhc/mzh030.View ArticlePubMedGoogle Scholar
  9. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004, 82: 581-629. 10.1111/j.0887-378X.2004.00325.x.View ArticlePubMedPubMed CentralGoogle Scholar
  10. Grol R, Wensing M, Eccles M: Improving Patient Care: The Implementation of Change in Clinical Practice. 2005, Oxford: ElsevierGoogle Scholar
  11. Rogers EM: Diffusion of Innovations. 1983, New York: The Free PressGoogle Scholar
  12. Chaudoir SR, Dugan AG, Barr CHI: Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level. Implement Sci. 2013, 8: 22-10.1186/1748-5908-8-22.View ArticlePubMedPubMed CentralGoogle Scholar
  13. Huijg JM, Crone MR, Verheijden MW, van der Zouwe N, Middelkoop BJC, Gebhardt WA: Factors influencing the adoption, implementation, and continuation of physical activity interventions in primary health care: a Delphi study. BMC Fam Pract. 2013, 14: 142-10.1186/1471-2296-14-142.View ArticlePubMedPubMed CentralGoogle Scholar
  14. Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005, 58: 107-112. 10.1016/j.jclinepi.2004.09.002.View ArticlePubMedGoogle Scholar
  15. Foy R, Eccles M, Grimshaw J: Why does primary care need more implementation 5esearch?. Fam Pract. 2001, 18: 353-355. 10.1093/fampra/18.4.353.View ArticlePubMedGoogle Scholar
  16. French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM: Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the theoretical domains framework. Implement Sci. 2012, 7: 38-10.1186/1748-5908-7-38.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Michie S, Johnston M, Francis J, Hardeman W, Eccles M: From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. 2008, 57: 660-680. 10.1111/j.1464-0597.2008.00341.x.View ArticleGoogle Scholar
  18. Presseau J, Johnston M, Francis J, Hrisos S, Stamp E, Steen N, Hawthorne G, Grimshaw J, Elovainio M, Hunter M, Eccles M: Theory-based predictors of multiple clinician behaviors in the management of diabetes. J Behav Med. 2013, epub ahead of printGoogle Scholar
  19. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP: A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013, 8: 35-10.1186/1748-5908-8-35.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O’Brien MA: Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001, 39: II2-II45.View ArticlePubMedGoogle Scholar
  21. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8: 6-View ArticleGoogle Scholar
  22. Davies P, Walker AE, Grimshaw JM: A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010, 5: 14-10.1186/1748-5908-5-14.View ArticlePubMedPubMed CentralGoogle Scholar
  23. The Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG): Designing theoretically-informed implementation interventions. Implement Sci. 2006, 1: 4-View ArticleGoogle Scholar
  24. Francis JJ, Tinmouth A, Stanworth SJ, Grimshaw JM, Johnston M, Hyde C, Stockton C, Brehaut JC, Fergusson D, Eccles MP: Using theories of behaviour to understand transfusion prescribing in three clinical contexts in two countries: development work for an implementation trial. Implement Sci. 2009, 4: 70-10.1186/1748-5908-4-70.View ArticlePubMedPubMed CentralGoogle Scholar
  25. Grol RPTM, Bosch MC, Hulscher MEJL, Eccles MP, Wensing M: Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007, 85: 93-138. 10.1111/j.1468-0009.2007.00478.x.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Rothman AJ: “Is there nothing more practical than a good theory?”: why innovations and advances in health behavior change will arise if interventions are used to test and refine theory. Int J Behav Nutr Phys. 2004, 1: 11-10.1186/1479-5868-1-11.View ArticleGoogle Scholar
  27. Francis JJ, Stockton C, Eccles MP, Johnston M, Cuthbertson BH, Grimshaw JM, Hyde C, Tinmouth A, Stanworth SJ: Evidence-based selection of theories for designing behaviour change interventions: using methods based on theoretical construct domains to understand clinicians’ blood transfusion behaviour. Br J Health Psychol. 2009, 14: 625-646. 10.1348/135910708X397025.View ArticlePubMedGoogle Scholar
  28. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14: 26-33. 10.1136/qshc.2004.011155.View ArticlePubMedPubMed CentralGoogle Scholar
  29. Cane J, O’Connor D, Michie S: Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012, 7: 37-10.1186/1748-5908-7-37.View ArticlePubMedPubMed CentralGoogle Scholar
  30. McSherry LA, Dombrowski SU, Francis JJ, Murphy J, Martin CM, O’Leary JJ, Sharp L, ATHENS Group: ‘It’s a can of worms’: understanding primary care practitioners' behaviours in relation to HPV using the theoretical domains framework. Implement Sci. 2012, 7: 73-10.1186/1748-5908-7-73.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Patey AM, Islam R, Francis JJ, Bryson GL, Grimshaw JM: Anesthesiologists’ and surgeons’ perceptions about routine pre-operative testing in low-risk patients: application of the Theoretical Domains Framework (TDF) to identify factors that influence physicians’ decisions to order pre-operative tests. Implement Sci. 2012, 7: 52-10.1186/1748-5908-7-52.View ArticlePubMedPubMed CentralGoogle Scholar
  32. Islam R, Tinmouth AT, Francis JJ, Brehaut JC, Born J, Stockton C, Stanworth SJ, Eccles MP, Cuthbertson BH, Hyde C, Grimshaw JM: A cross-country comparison of intensive care physicians’ beliefs about their transfusion behaviour: a qualitative study using the theoretical domains framework. Implement Sci. 2012, 7: 93-10.1186/1748-5908-7-93.View ArticlePubMedPubMed CentralGoogle Scholar
  33. Curran JA, Brehaut J, Patey AM, Osmond M, Stiell I, Grimshaw JM: Understanding the Canadian adult CT head rule trial: use of the theoretical domains framework for process evaluation. Implement Sci. 2013, 8: 25-10.1186/1748-5908-8-25.View ArticlePubMedPubMed CentralGoogle Scholar
  34. Dyson J, Lawton R, Jackson C, Cheater F: Does the use of a theoretical approach tell us more about hand hygiene behaviour? The barriers and levers to hand hygiene. J Infect Prev. 2011, 12: 17-24. 10.1177/1757177410384300.View ArticleGoogle Scholar
  35. Mazza D, Chapman A, Michie S: Barriers to the implementation of preconception care guidelines as perceived by general practitioners: a qualitative study. BMC Health Serv Res. 2013, 13: 36-10.1186/1472-6963-13-36.View ArticlePubMedPubMed CentralGoogle Scholar
  36. Michie S, Pilling S, Garety P, Whitty P, Eccles M, Johnston M, Simmons J: Difficulties implementing a mental health guideline: an exploratory investigation using psychological theory. Implement Sci. 2007, 2: 8-10.1186/1748-5908-2-8.View ArticlePubMedPubMed CentralGoogle Scholar
  37. Amemori M, Michie S, Korhonen T, Murtomaa H, Kinnunen TH: Assessing implementation difficulties in tobacco use prevention and cessation counselling among dental providers. Implement Sci. 2011, 6: 50-10.1186/1748-5908-6-50.View ArticlePubMedPubMed CentralGoogle Scholar
  38. Beenstock J, Sniehotta FF, White M, Bell R, Milne EM, Araujo-Soares V: What helps and hinders midwives in engaging with pregnant women about stopping smoking? A cross-sectional survey of perceived implementation difficulties among midwives in the northeast of England. Implement Sci. 2012, 7: 36-10.1186/1748-5908-7-36.View ArticlePubMedPubMed CentralGoogle Scholar
  39. Taylor N, Parveen S, Robins V, Slater B, Lawton R: Development and initial validation of the influences on patient safety behaviours questionnaire. Implement Sci. 2013, 8: 81-10.1186/1748-5908-8-81.View ArticlePubMedPubMed CentralGoogle Scholar
  40. Huijg JM, Gebhardt WA, Crone MR, Dusseldorp E, Presseau J: Discriminant content validity of a Theoretical Domains Framework questionnaire for use in implementation research. Implement Sci. 2014, 9: 11-10.1186/1748-5908-9-11.View ArticlePubMedPubMed CentralGoogle Scholar
  41. Ajzen I: The theory of planned behavior. Organ Behav Hum Decis. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.View ArticleGoogle Scholar
  42. Bandura A: Health promotion from the perspective of social cognitive theory. Psychol Health. 1998, 13: 623-649. 10.1080/08870449808407422.View ArticleGoogle Scholar
  43. Huijg JM, van der Zouwe N, Gebhardt WA, Crone MR, Verheijden MW, Middelkoop BJC: Introducing physical activity interventions in primary health care: a qualitative study of perceived facilitators and barriers [abstract]. Int J Behav Med. 2012, 19: S1-S341.Google Scholar
  44. van Veldhoven M, Meijman T, Broersen J, Fortuin R: Handleiding VBBA. 2002, Amsterdam: SKB Vragenlijst ServicesGoogle Scholar
  45. Frese M: Social support as a moderator of the relationship between work stressors and psychological dysfunctioning: a longitudinal study with objective measures. J Occup Health Psychol. 1999, 4: 179-192.View ArticlePubMedGoogle Scholar
  46. Gardner B, Abraham C, Lally P, de Bruijn G-J: Towards parsimony in habit measurement: testing the convergent and predictive validity of an automaticity subscale of the self-report habit index. Int J Behav Nutr Phys. 2012, 9: 102-10.1186/1479-5868-9-102.View ArticleGoogle Scholar
  47. Sniehotta FF, Schwarzer R, Scholz U, Schüz B: Action planning and coping planning for long-term lifestyle change: theory and assessment. Eur J Soc Psychol. 2005, 35: 565-576. 10.1002/ejsp.258.View ArticleGoogle Scholar
  48. Scheier MF, Carver CS, Bridges MW: Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): a reevaluation of the life orientation test. J Pers Soc Psychol. 1994, 67: 1063-1078.View ArticlePubMedGoogle Scholar
  49. Cialdini R, Kallgren C, Reno R: A focus theory of normative conduct: a theoretical refinement and reevaluation of the role of norms in human behavior. Adv Exp Soc Psychol. 1991, 24: 202-232.Google Scholar
  50. Qualtrics: Qualtrics software, version 45433. 2013, Provo, Utah, USA: QualtricsGoogle Scholar
  51. Corp IBM: IBM SPSS Statistics for Windows, Version 19.0. 2010, IBM Corp.: Armonk, NYGoogle Scholar
  52. Stuive I, Kiers HAL, Timmerman ME, ten Berge JMF: The empirical verification of an assignment of items to subtests: the oblique multiple group method versus the confirmatory common factor method. Educ Psychol Meas. 2008, 68: 923-939. 10.1177/0013164408315264.View ArticleGoogle Scholar
  53. Holzinger K: A simple method of factor analysis. Psychometrika. 1944, 9: 257-262. 10.1007/BF02288737.View ArticleGoogle Scholar
  54. Stuive I, Kiers HAL, Timmerman ME: Comparison of methods for adjusting incorrect assignments of items to subtests: oblique multiple group method versus confirmatory common factor method. Educ Psychol Meas. 2009, 69: 948-965. 10.1177/0013164409332226.View ArticleGoogle Scholar
  55. Jöreskog K: A general approach to confirmatory maximum likelihood factor analysis. Psychometrika. 1969, 34: 183-202. 10.1007/BF02289343.View ArticleGoogle Scholar
  56. Jöreskog K: Testing a simple structure hypothesis in factor analysis. Psychometrika. 1966, 31: 165-178. 10.1007/BF02289505.View ArticlePubMedGoogle Scholar
  57. Hotelling H: Analysis of a complex of statitical variables into principal components. J Educ Psychol. 1933, 24: 417-441.View ArticleGoogle Scholar
  58. Cronbach L: Coefficient alpha and the internal structure of tests. Psychometrika. 1951, 16: 297-333. 10.1007/BF02310555.View ArticleGoogle Scholar
  59. Campbell D, Fiske D: Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1962, 59: 257-272.View ArticleGoogle Scholar
  60. Anderson JC, Gerbing DW: Structural equation modeling in practice: a review and recommended two-step approach. Psychological Bulletin. 1988, 103: 411-423.View ArticleGoogle Scholar
  61. Spearman C: The proof and measurement of association between two things. Am J Psychol. 1904, 15: 72-101. 10.2307/1412159.View ArticleGoogle Scholar
  62. Timmerman M, Stuive I: Multiple Group Method with Corrections for Selfcorrelation and Test Length. 2007Google Scholar
  63. R Development Core Team R: R: a Language and Environment for Statistical Computing. 2012, Vienna, Austria: R Foundation for Statistical ComputingGoogle Scholar
  64. Falissard B: Psy: Various Procedures Used in Psychometry. 2012, Vienna, Austria: R Foundation for Statistical Computing, R Package Version 1.1Google Scholar
  65. Cohen J: Statistical Powe Analysis for the Behavioral Sciences. 1988, Hillsdale, New Jersey: Lawrence ErlbaumGoogle Scholar
  66. Wännström I, Peterson U, Asberg M, Nygren A, Gustavsson JP: Psychometric properties of scales in the General Nordic Questionnaire for Psychological and Social Factors at Work (QPS): confirmatory factor analysis and prediction of certified long-term sickness absence. Scand J Psychol. 2009, 50: 231-244. 10.1111/j.1467-9450.2008.00697.x.View ArticlePubMedGoogle Scholar
  67. American Psychological Association (APA): APA Dictionary of Psychology. 2007, Washington, DC: American Psychological AssociationGoogle Scholar
  68. Taylor N, Lawton R, Conner M: Development and initial validation of the determinants of physical activity questionnaire. Int J Behav Nutr Phys Act. 2013, 10: 74-10.1186/1479-5868-10-74.View ArticlePubMedPubMed CentralGoogle Scholar
  69. Watson D, Clark LA, Tellegen A: Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988, 54: 1063-1070.View ArticlePubMedGoogle Scholar
  70. Crawford JR, Henry JD: The positive and negative affect schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. Br J Clin Psychol. 2004, 43: 245-265. 10.1348/0144665031752934.View ArticlePubMedGoogle Scholar
  71. Wood W, Neal DT: The habitual consumer. J Consum Psychol. 2009, 19: 579-592. 10.1016/j.jcps.2009.08.003.View ArticleGoogle Scholar
  72. Deci E, Ryan R: Intrinsic Motivation and Self-Determination in Human Behavior. 1985, New York: Plenum Publishing Co.View ArticleGoogle Scholar
  73. Maslach C, Jackson SE: The measurement of experienced burnout. J Organ Behav. 1981, 2: 99-113. 10.1002/job.4030020205.View ArticleGoogle Scholar
  74. Dannapfel P, Peolsson A, Nilsen P: What supports physiotherapists’ use of research in clinical practice? A qualitative study in Sweden. Implement Sci. 2013, 8: 31-10.1186/1748-5908-8-31.View ArticlePubMedPubMed CentralGoogle Scholar
  75. Forsman H, Rudman A, Gustavsson P, Ehrenberg A, Wallin L: Nurses’ research utilization two years after graduation: a national survey of associated individual, organizational, and educational factors. Implement Sci. 2012, 7: 46-10.1186/1748-5908-7-46.View ArticlePubMedPubMed CentralGoogle Scholar
  76. Shirley D, Van der Ploeg HP, Bauman AE: Physical activity promotion in the physical therapy setting: perspectives from practitioners and students. Phys Ther. 2010, 90: 1311-1322. 10.2522/ptj.20090383.View ArticlePubMedGoogle Scholar
  77. van der Wees PJ, Zagers CA, de Die SE, Hendriks EJ, Nijhuis-van der Sanden MW, de Bie RA: Developing a questionnaire to identify perceived barriers for implementing the Dutch physical therapy COPD clinical practice guideline. BMC Health Serv Res. 2013, 13: 159-10.1186/1472-6963-13-159.View ArticlePubMedPubMed CentralGoogle Scholar
  78. Barrett EM, Darker CD, Hussey J: Promotion of physical activity in primary care: knowledge and practice of general practitioners and physiotherapists. J Public Health. 2013, 21: 63-69. 10.1007/s10389-012-0512-0.View ArticleGoogle Scholar
  79. Stevens J: Applied Multivariate Statistics for the Social Sciences. 1992, Hillsdale, New Jersey: Lawrence ErlbaumGoogle Scholar

Copyright

© Huijg et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.