Investigating the complementary value of discrete choice experiments for the evaluation of barriers and facilitators in implementation research: a questionnaire survey

Background The potential barriers and facilitators to change should guide the choice of implementation strategy. Implementation researchers believe that existing methods for the evaluation of potential barriers and facilitators are not satisfactory. Discrete choice experiments (DCE) are relatively new in the health care sector to investigate preferences, and may be of value in the field of implementation research. The objective of our study was to investigate the complementary value of DCE for the evaluation of barriers and facilitators in implementation research. Methods Clinical subject was the implementation of the guideline for breast cancer surgery in day care. We identified 17 potential barriers and facilitators to the implementation of this guideline. We used a traditional questionnaire that was made up of statements about the potential barriers and facilitators. Respondents answered 17 statements on a five-point scale ranging from one (fully disagree) to five (fully agree). The potential barriers and facilitators were included in the DCE as decision attributes. Data were gathered among anaesthesiologists, surgical oncologists, and breast care nurses by means of a paper-and-pencil questionnaire. Results The overall response was 10%. The most striking finding was that the responses to the traditional questionnaire hardly differentiated between barriers. Forty-seven percent of the respondents thought that DCE is an inappropriate method. These respondents considered DCE too difficult and too time-consuming. Unlike the traditional questionnaire, the results of a DCE provide implementation researchers and clinicians with a relative attribute importance ranking that can be used to prioritize potential barriers and facilitators to change, and hence to better fine-tune the implementation strategies to the specific problems and challenges of a particular implementation process. Conclusion The results of our DCE and traditional questionnaire would probably lead to different implementation strategies. Although there is no 'gold standard' for prioritising potential barriers and facilitators to the implementation of change, theoretically, DCE would be the method of choice. However, the feasibility of using DCE was less favourable. Further empirical applications should investigate whether DCE can really make a valuable contribution to the implementation science.

the implementation strategies to the specific problems and challenges of a particular implementation process.

Conclusion:
The results of our DCE and traditional questionnaire would probably lead to different implementation strategies. Although there is no 'gold standard' for prioritising potential barriers and facilitators to the implementation of change, theoretically, DCE would be the method of choice. However, the feasibility of using DCE was less favourable. Further empirical applications should investigate whether DCE can really make a valuable contribution to the implementation science.

Background
There are numerous implementation strategies available that have proven to be at least moderately effective in bringing about change [1][2][3]. Current insight in implementation research is that the choice of implementation strategy should be guided by a diagnostic analysis that starts with describing the gap between current care and optimal care [4]. An important part of the diagnostic analysis is the identification of barriers and facilitators to change. Until now, studies identifying barriers and facilitators to change have been carried out using a combination of qualitative and quantitative methods, such as casespecific questionnaires, semi-structured in-depth interviews, focus group interviews, and non-participating observation [5]. These methods have some limitations. First, they generally yield many barriers, but do not provide information with respect to the relative importance, or prioritizing, of the barriers. Second, existing instruments hardly differentiate between barriers, and consequently may overestimate the importance of less important barriers and underestimate the importance of more important barriers. Third, these traditional methods have a non-compensatory character, which may be problematic because in decision processes concerning the implementation of change it is often the case that facilitators can partly compensate for barriers. These limitations associated with the methods that are currently applied in implementation research have revealed the need for an alternative research methodology for the evaluation of barriers and facilitators. Discrete choice experiments (DCE) to investigate preferences are relatively new in the health care sector, and may be of value in the field of implementation research. DCE is a stated preference method that presents individuals with a number of choices. Each choice consists of two or more hypothetical profiles, and for each choice, people are asked which profile they would choose. Forcing people to make choices and trade-offs is a big advantage of DCE over the methods that are currently applied in implementation research. For us, this is a strong motivation to introduce DCE in implementation research. If DCE proves to have a complementary value for the evaluation of barriers and facilitators, the choice of implementation strategy will be based on factors that more accurately reflect individuals' prefer-ences and trade-offs, and the strategies will be tailored to the preferences of those concerned with the actual implementation.
The objective of our study was to investigate the complementary value of DCE for the evaluation of barriers and facilitators in implementation research. To meet this objective, we compared the results of a traditional questionnaire with the results of a DCE. Note that we use the term 'traditional' to refer to common, well-known research methodologies that are usually used in implementation research, in contrast to the DCE method that is new in the field of implementation research. We would expect a priori that DCE would overcome the previously mentioned limitations associated with using a traditional questionnaire. Especially, DCE makes it possible to gain insight into the relative importance of barriers to the implementation of change, and the trade-offs people make between barriers on the one hand and facilitators on the other hand. DCE therefore more closely reflects actual implementation decisions.
Respondents were expected to be unfamiliar with DCE, but also are actually involved in implementation processes and hence users of the results and responsible for getting the research findings into practice. Therefore, we asked respondents for their opinions about the feasibility of DCE. In particular, we asked for the completion time, the difficulty of the questions, and the appropriateness of DCE. The traditional questions are well-known, easy to administer and take little time to complete. Furthermore, the traditional method is often applied in implementation research, which is a strong indicator of its feasibility. DCE, on the other hand is new in implementation research, so we decided to evaluate only the feasibility and acceptance of this novel development.

Clinical subject
The clinical subject of this study was the implementation of the guideline for breast cancer surgery in day care. Breast cancer care causes a significant burden to the health care budget, which can mainly be attributed to surgical treatment [6], including hospitalization. In the University Hospital Maastricht, the Netherlands, an ambulatory surgery programme for breast cancer was developed and evaluated under the title of 'Breast pathology treated in day care: blessing or a curse?' [7]. Major objective of this programme was to reduce the length of hospital stay. The programme consists of a structured care system, including education and counselling, dedicated anaesthesia, active participation of the patient in her own treatment plan and in the decision to go home, and home nursing care. Traditionally, the surgical oncologist has been the specialist who analyses the type of breast pathology presented. Preventive procedures, early diagnosis of non-palpable lesions, breast conserving therapy, and lymph node sparing therapy have raised possibilities to reduce the burden of breast cancer surgery. The role of the breast care nurse was introduced to improve patient counseling. While the clinician's activities are limited to solving medical problems, the breast care nurse performs all coordinating tasks to create a programme that runs smoothly. Through these developments, diagnosis and treatment of breast pathology have gained a more multidisciplinary character. A decrease in the burden of surgery may limit the need for hospital-based supportive care. Similarly, this leads to a demand for strict coordination of the different steps and disciplines involved. Moreover, responsibilities for aspects of care need to be reallocated to other persons: from hospital-based supportive caregiver towards informal caregiver, from clinician to nurse specialist, from ward nursing staff to outpatient nursing staff, and from hospital-based nursing staff to home care nursing staff. Evaluation of the programme revealed that the breast cancer care programme was safe, reduced hospital stay, resulted in a high level of patient satisfaction, and resulted in lower cost of hospital stay. The guideline for breast cancer surgery in day care is currently being implemented in four other centres in the Netherlands as part of the implementation study entitled 'Introduction of a breast cancer care program in ultra short stay in four early adopter hospitals: implementation and evaluation'. Up till now, the Dutch Institute for Healthcare Improvement has not published the guideline for breast cancer surgery in day care officially.

Discrete choice experiment
The DCE presented in this paper is described in detail elsewhere [8]. Therefore, we will now present the DCE method to the extent that the information is relevant for understanding of the present paper. For more theoretical background information about DCE, the interested reader is referred to Louviere et al. [9]. DCE is a stated preference method to measure preferences for products and services. It is based on the assumption that in general decisions are not based on a single criterion, but on several factors considered jointly [10]. In DCE jargon, these factors are called 'attributes'. A DCE consists of five stages: identifying the attributes of interest, assigning levels to the attributes, presenting profiles to individuals which involve different levels of the attributes, obtaining preferences for the profiles, and analyzing the responses.
A typical DCE starts with identifying the potential influential attributes. In our experiment, the attributes, being the potential barriers and facilitators to the implementation of the guideline for breast cancer surgery in day care, were selected on the basis of interviews with doctors and nurses who are all specialized in breast cancer surgery. In addition, we used the experience gained from the previously mentioned study 'Breast pathology treated in day care: blessing or a curse?' [7]. From the guideline for breast cancer surgery in day care that is composed of multiple recommendations, we selected the 12 key recommendations closely related to the surgical procedure (Appendix 1). Recommendations for the other parts of the programme were omitted. The 17 most frequently mentioned potential barriers and facilitators to the implementation of the 12 key recommendations were retained and included in the DCE as decision attributes.
Each of these attributes had two levels (Table 1). To handle these large numbers of potentially influential attributes, we used hierarchical information integration (HII). This is an alternative to standard discrete choice experiments when too many attributes are involved. HII relies on the assumption that, when confronted with complex decisions or evaluations involving numerous elements, people are able to divide a set of decision attributes that influence their choice behaviour into subsets that can be labelled in terms of high-order decision constructs, then evaluate each decision construct separately and aggregate their evaluations of each decision construct to choose between competing opportunities [11][12][13]. For details about our HII application, we refer to our Health Economics paper [8].
A full factorial design would require 512 profiles, too many for meaningful research. We therefore used fractional factorial designs based on an orthogonal main effects design. Profiles were paired into choice sets using a foldover design. This means that each of the profiles was combined with its 'foldover' profile. A foldover profile includes the exact opposite attribute levels of the original profile and, therefore, ensures a completely orthogonal design.
Next, respondents were presented with a series of pairwise profiles (choice sets) involving different levels of the attributes. Within each choice set, the profiles were labelled 'circumstances A' and 'circumstances B', and respondents were asked to choose between the two profiles for the implementation of breast cancer surgery in day care. Furthermore, respondents could select the 'neither' option if they did not find any of the two profiles acceptable for implementation. For an example of a discrete choice task see Additional file 1.
The choices were used to estimate the overall choice -or utility -model. Random parameters logit modelling was used to estimate this discrete choice model. We used the software package NLOGIT 3.0 (Econometric Software Inc.). The dependent variable was the choice alternative selected by the respondent. The independent variables were the 17 potential barriers and facilitators to the implementation of the guideline for breast cancer surgery in day care. We calculated the relative importance of the attributes as described in the literature [e.g., [12]]. The 17 attributes were regarded as sources of respondents' utility, because the attributes are more or less important for successful implementation of this guideline. The overall utility may therefore be described as an evaluation of how attractive it is to implement the guideline for breast cancer surgery in day care, given the circumstances described by the attributes.

Traditional questionnaire
To enable comparison of DCE with a traditional questionnaire, the same potential barriers and facilitators to the implementation of the guideline for breast cancer surgery in day care that were selected for the DCE were translated into statements (Appendix 2). Nine statements (1, 2, 4, 5, 11, 12, 13, 14, and 16) were phrased as preconditions that measure to what extent people believe that the specified requirements should be fulfilled to successfully implement change. The remaining eight statements (3, 6, 7, 8, 9, 10, 15, and 17) were phrased as expectations that measure to what extent respondents expect that the specified barriers will actually occur in their own hospitals.
Respondents were asked to respond to the 17 statements on a five-point scale from one (fully disagree) to five (fully agree).
The responses to the traditional questionnaire were described in terms of means and medians. We also tested whether the responses were equally distributed across the response categories using Chi-Square tests. We used SPSS for Windows version 11.5 for the computations.

Sample and data collection
Three groups of health care professionals were considered important for successful implementation of breast cancer surgery in day care, namely anaesthesiologists, surgical oncologists, and breast care nurses. The Dutch Society for Anaesthesiology and the Dutch Society for Surgical Oncology supplied the address files of their members. Questionnaires were sent by postal mail to anaesthesiologists and surgical oncologists, together with an informative letter to explain the background and aim of the study, and signed by the chair of the Dutch Society for Surgical Oncology. The distribution of questionnaires, including the informative letters, among breast care nurses took place through the chair of the special interest group Mammacare, which is part of the Society for Oncology Nurses. This group consists of nine members who represent the nine comprehensive cancer centre regions in the Netherlands. The questionnaires were distributed in the regions via these key contacts. Furthermore, the breast care nurses were encouraged to fill in the questionnaire by a message on the society's website. A paper-and-pencil questionnaire was developed that consisted of five sections. Because the research presented in the present paper was part of a larger research project entitled 'Investigating the added value of conjoint analysis for the evaluation of barriers and facilitators in implementation studies: the case of breast cancer surgery in ultrashort stay', we will mention only those parts of the questionnaire that relate to the comparison between DCE and a traditional questionnaire. Respondents first completed some background questions, i.e., age, sex, work experience, and mode of employment (salaried versus partnership). The next section included some questions about hospital characteristics. Then, respondents were asked to indicate their perceptions about the 17 potential barriers and facilitators in the 'traditional' way. In the final section, we first introduced the DCE thoroughly. We explained DCE by means of an example from everyday life. Then, respondents were asked to study the attributes and levels, and finally they completed a warm-up task with fill-in instructions. The attribute descriptions and levels, the warm-up task, and the fill-in instructions were presented on an insert. Respondents could take out this insert and re-read the information while evaluating the actual profiles. Following this introduction, respondents completed 14 actual discrete choice tasks. The section ended with three questions about the feasibility of DCE. First, respondents were asked how long it took to complete the discrete choice tasks. Secondly, they were asked to indicate on a scale from one (extremely difficult) to nine (extremely easy) the difficulty of the choice tasks. Finally, they were asked to briefly describe the appropriateness of DCE in implementation research. We calculated the mean number of minutes to complete the discrete choice tasks, and the mean score on the difficulty scale. The answers to the open question about the appropriateness of DCE were classified.
The questionnaire was pilot-tested beforehand. The aim of the pilot was to examine the respondents' understanding of the questionnaire. The pilot was designed as a think-aloud study; this involves respondents thinking aloud as they are completing the questionnaire. Eight respondents participated in the pilot test (three anaesthesiologists; two surgical oncologists; three breast care nurses). Based on the pilot test several changes were made to the layout of the questionnaire, and the wording of the instructions and questions.

Response
We anticipated beforehand that health care professionals (especially physicians) would be difficult respondents to recruit for surveys. Therefore, we decided to approach all anaesthesiologists, all surgical oncologists and all breast care nurses. A total of 1,713 questionnaires were sent to One hundred and seventy-four respondents returned the questionnaire (Table 2), resulting in an overall response of 10%. The response rates were 8%, 14%, and 13% for anaesthesiologists, surgical oncologists, and breast care nurses, respectively. The response rates differed statistically significant across these three professional disciplines (Chi-Square test; p < 0.001). We could not investigate whether selection bias has occurred because we lack information on the non-responders.

Discrete choice experiment
Of 174 respondents who returned the questionnaire, 18 did not answer the discrete choice tasks. As a result, the responses of 156 respondents could be analyzed. Nearly all attributes were significant at the 1% level, implying that these attributes are relevant to the implementation of the guideline for breast cancer surgery in day care. Only the coefficient for time investment (p = 0.174) was not significant.
The relative importances of the attributes (Table 3) show that respondents' choices were influenced most strongly by the attribute 'cooperation of colleagues'. Therefore, whether or not colleagues would assist in implementing the guideline is the most important factor for health care professionals when they think of breast cancer surgery in day care. Also, the cooperation of the ward nursing staff and management was considered highly important. Cooperation of patients and patient organizations was considerably less important. Time investment, status of the guideline, and patient satisfaction were least influential. Table 4 presents the results of the traditional questionnaire. Statements related to preconditions and statements related to expectations are presented separately. The rankings of the responses are based on decreasing means. The most striking finding was that the responses to the statements hardly differentiated between barriers. This is particularly true when the median values are considered. Respondents considered the availability of patient information brochures after the diagnosis and at discharge, as well as clear-cut criteria for discharge as the most important requirements for successful implementation of the guideline for breast cancer surgery in day care. The presence of a Day Surgery Unit was considered the least important precondition.

(page number not for citation purposes)
The ranking of the expectations reflects to what extent each of the potential barriers to the implementation of the guideline for breast cancer surgery in day care is expected to actually become a barrier in respondents' own hospitals. Table 4 shows that respondents were, on average, most hopeful about the cooperation of those concerned with the implementation of the guideline, i.e., the ward nursing staff, patients and patient organizations, colleagues, and the management. Working according to the guideline is not expected to be more time-consuming because respondents, on average, neither agreed nor disa-greed with the concerning statement (Appendix 2, statement 17). This suggests that respondents do not perceive time as a major barrier to the implementation of the guideline. For all 17 statements, the responses were not equally distributed across the response categories (Chi-Square tests; p < 0.001).

Feasibility of DCE
The number of minutes to complete the choice tasks was on average 25.5 ± 14.7 minutes. The completion time was 23.7 ± 15.4 minutes for anaesthesiologists, 24.9 ± 12.8  We classified the answers to the open question about the appropriateness of DCE into three main categories. Twenty-six respondents found DCE an appropriate method, and 63 respondents thought that the method is inappropriate. Forty-five respondents did not explicitly state whether or not they think it is an appropriate method. These respondents did not respond to the methodology but instead they focussed on the clinical subject; other respondents answered: 'I do not know (or I doubt) whether this is an appropriate method' or 'I have no opinion'. Twenty-two respondents did not complete the question. The 63 respondents who judged DCE inappropriate gave in total 85 reasons for their opinion. We subdivided these reasons into seven subcategories. Table 5 shows that 'too difficult' was by far the most frequently mentioned reason (51%).

Discussion
To the best of our knowledge this is the first application of DCE in implementation research. We used DCE to identify barriers and facilitators to the implementation of the guideline for breast cancer surgery in day care. The objective of our study was to investigate the complementary value of DCE for the evaluation of barriers and facilitators in implementation research. To meet this objective we compared the results of a traditional questionnaire with the results of a DCE. In addition, we asked respondents for their opinions about the feasibility of DCE.
Neither DCE nor a traditional questionnaire was considered the 'gold standard' for identifying potential barriers and facilitators to the implementation of breast cancer surgery in day care. The reason why we conducted this study was that we would expect that DCE would provide implementers with more specific information to better fine-tune the implementation strategies. The results of a DCE provide implementation researchers and clinicians with a relative attribute importance ranking that can be used to prioritize potential barriers and facilitators to change. Prioritizing is useful to tailor the implementation strategies to the specific problems and challenges of a particular implementation process. Furthermore, the DCE method makes it possible to gain insight into the tradeoffs people make between barriers to change on the one hand and facilitators on the other hand. The traditional questionnaire was made up of statements that respondents answered on a five-point scale ranging from one (fully disagree) to five (fully agree). The questionnaire included both statements related to preconditions and statements related to expectations. Both kinds of statements cannot be compared to each other because the interpretation of the responses is considerably different. Statements phrased as preconditions measure whether the specified requirements should be fulfilled to successfully implement change. Statements phrased as expectations measure whether the specified barriers are expected to actually occur in respondents' own hospitals. Although the extent to which respondents agreed with the statements about the preconditions might be interpreted as a measure of the importance respondents attach to each of the requirements, it does not provide information about the relative importance of the barriers and facilitators and the trade-offs people make. Hence, a traditional questionnaire cannot easily be used to identify the most crucial barriers and facilitators to successful implementation and to leave aside the relatively unimportant ones. Furthermore, we must interpret the results of the traditional questionnaire with caution because the responses to the statements hardly differed from each other. One could argue that essentially respondents agreed with all statements to the same extent, which makes ranking less useful.
The results of our DCE and traditional questionnaire would possibly lead to different implementation strategies. This confronts those who are involved in implementation processes with serious difficulties, because we do not know which method is the best one. There are three major differences between DCE and the traditional ques-tions that might challenge the comparability of the results. First, DCE is based on random utility theory that assumes that an individual acts rationally and always chooses the alternative with the highest level of utility. In our experiment, the 17 attributes (the independent variables in the utility function) are regarded as sources of respondents' utility because the attributes are more or less important for successful implementation of the guideline for breast cancer surgery in day care. The overall utility may therefore be described as an evaluation of how attractive it is to implement the guideline for breast cancer surgery in day care given the circumstances as described by the attributes. DCE is therefore a preference-based method in contrast with the traditional questionnaire that has no theoretical underpinnings. Second, the regular questions are more likely to measure the perceived barriers and facilitators that may not reflect the actual barriers and facilitators. The results of a DCE are likely to give a more objective view of the factors that are important for successful implementation. When responding to the regular questions, respondents are generally inclined to refer to the circumstances in their own hospitals ('which barriers would we encounter in our hospital?').
Whether respondents really answer the regular questions with their own hospitals' situations in mind depends to a great extent on the wording of the questions, which is not standardized. Discrete choice tasks present subjects with hypothetical scenarios; hence there is no link between the choices and subjects' current clinical practice. Third, in a DCE people are forced to make choices and trade off attributes, which is closer to the decision-making context in reality than the regular questions. We encountered a low overall response rate of 10%. In accordance with Sitzia and Wood [14], sending reminders did little to increase the response rate. Response rates for mail surveys are approximately 62% [15]. Large variation is reported for response rates for DCE in the health care literature, ranging from 18% [16] to 98% [17]. We do not know if, to what extent, and how the low response has influenced the results. In the current study, there are four possible explanations for the low response. First, we investigated a complex multi-faceted health care decision -the implementation of the guideline for breast cancer surgery in day care -that moreover involves multiple professional disciplines. In implementation research, it is important to take into consideration this multidisciplinary nature of many health care decisions because different groups of health care professionals may express different opinions, interests, and preferences. Hence, the potential barriers and facilitators to the implementation of the guideline need not necessarily be the same for anaesthesiologists, surgical oncologists, and breast care nurses. To facilitate comparison between these groups, we developed an identical questionnaire for all three disciplines. Although the advantages of this approach are obvious, a disadvantage is that we could not completely finetune the questions to the specific circumstances of each professional group. Therefore, respondents may have considered the questions too broad and not fully applicable to their own situations. In particular, anaesthesiologists were expected to respond suboptimally because this group of health care professionals is less closely associated with the subject of the survey compared to surgical oncologists and breast care nurses. Respondents may thus have had less affinity with the questions. A second explanation for the low response may be the use of self-complete postal questionnaires. Other more personal data collection methods are available, and telephone or face-to-face interviews might well have increased the response rate, though at much higher costs. The phenomenal growth of internet technology in recent years has prompted many health care researchers to find ways of using this technology to communicate more effectively. Braithwaite et al. [18] examined whether internet-based surveys among health professionals can offer a valid alternative to traditional survey methods. They performed a systematic review of published internet-based surveys among health professionals and found response rates ranging from 9% to 94%. The authors also conducted an internet-based survey among general practitioners that explored attitudes about using an internet-based decision support system for the management of familial cancer. They achieved a response rate of 52.4% after five e-mail reminders. Brown and Kittleson [19] compared the response rates from an email survey (43%) and a web-based survey (48%), and found no statistically significant differences between the two groups. An online survey among chiropractors yielded a response rate of 35.8% [20]. As online surveys become more feasible for more populations, it is worth considering the use of e-mail or the web as data collection tools in future research.
Third, the clinical subject may have been too complex to introduce DCE in implementation research. The implementation of the guideline for breast cancer surgery in day care is a complex process that involves changes on the organizational and management level, as well as the level of health care professionals and patients. The complexity of the clinical subject furthermore required that we had to include large numbers of potentially influential attributes. Therefore, we used HII, which is a more complex alternative to standard DCE.
Fourth, the questionnaire included both the traditional questions and the discrete choice tasks. All respondents first answered the traditional questions and then the discrete choice tasks. We do not know whether the response would have been better if respondents had been offered only the traditional questions or only the discrete choices. Several studies have shown that response rate is not correlated to questionnaire length [14,15,21]. Intuitively, we would nevertheless expect a higher response rate if the questionnaire included only the traditional questions for three reasons. First, our questionnaire was lengthy, and obviously completing only the traditional questions takes less time. Second, because respondents were likely to be unfamiliar with DCE, the discrete choice tasks were introduced thoroughly. Yet, this extensive introduction and warm-up task with fill-in instructions required a lot of reading, and thus time and motivation. Mean completion time of the discrete choice tasks was more than 25 minutes. Third, discrete choice tasks are without doubt more cognitively demanding than the traditional questions. The mean difficulty score was 4.8 on a scale from 1 to 9, which means 'somewhat difficult' to 'not difficult/not easy'. Eighteen respondents did not answer the discrete choice tasks. Although we did not systematically investigate why these respondents did not complete the choice tasks, we suppose that the two main reasons are the complexity of the DCE and the length of the questionnaire. We do not expect that these 18 respondents would have completed the DCE if the choices were presented before the traditional questions. The answers to the open question about the appropriateness of DCE are in support of this, and suggest doubt about the feasibility of DCE.
A considerable variety exists in the design and analysis of studies investigating barriers and facilitators to the implementation of change. In most cases, the analysis is constrained to descriptive statistics. Because of limitations associated with the methods typically applied in implementation research (see introduction), uncertainty exists with respect to the most useful research methodology. DCE should theoretically overcome the limitations of traditional methods, and provide implementers with more specific information, i.e., insight into the trade-offs people make and relative attribute importance. Because DCE probably reflects actual implementations decisions more closely, it is expected that the implementation strategies will become more tailored to the actual preferences, needs, and wishes of those who are involved in the actual implementation process. It is furthermore expected that this will increase the cost-effectiveness of implementation strategies. So, on theoretical and conceptual grounds it can be suggested that DCE should be considered the reference standard in our study [8]. From a practical point of view, however, our study revealed that DCE could not entirely fulfil the role of 'gold standard'. In brief, respondents considered the method too difficult and too timeconsuming, which may partly explain the low response. The feasibility of any method is -at least partly -dependent on the study context such as the target respondents, study design, clinical subject, and data collection method. Further empirical applications should investigate whether DCE can really make a valuable contribution to the implementation science.

Conclusion
DCE was proposed as a tool to identify potential barriers and facilitators to the implementation of change. The results of a DCE and a traditional questionnaire would probably lead to different implementation strategies. Although there is no 'gold standard' for prioritising potential barriers and facilitators to the implementation of change, theoretically, DCE would be the method of choice. However, the feasibility of using DCE was less favourable; respondents considered the method too difficult and too time-consuming, which may -at least partly -explain the low response.