- Open Access
- Open Peer Review
Evidence summaries (decision boxes) to prepare clinicians for shared decision-making with patients: a mixed methods implementation study
© Giguere et al.; licensee BioMed Central Ltd. 2014
- Received: 14 May 2014
- Accepted: 19 September 2014
- Published: 5 October 2014
Decision boxes (Dboxes) provide clinicians with research evidence about management options for medical questions that have no single best answer. Dboxes fulfil a need for rapid clinical training tools to prepare clinicians for clinician-patient communication and shared decision-making. We studied the barriers and facilitators to using the Dbox information in clinical practice.
We used a mixed methods study with sequential explanatory design. We recruited family physicians, residents, and nurses from six primary health-care clinics. Participants received eight Dboxes covering various questions by email (one per week). For each Dbox, they completed a web questionnaire to rate clinical relevance and cognitive impact and to assess the determinants of their intention to use what they learned from the Dbox to explain to their patients the advantages and disadvantages of the options, based on the theory of planned behaviour (TPB). Following the 8-week delivery period, we conducted focus groups with clinicians and interviews with clinic administrators to explore contextual factors influencing the use of the Dbox information.
One hundred clinicians completed the web surveys. In 54% of the 496 questionnaires completed, they reported that their practice would be improved after having read the Dboxes, and in 40%, they stated that they would use this information for their patients. Of those who would use the information for their patients, 89% expected it would benefit their patients, especially in that it would allow the patient to make a decision more in keeping with his/her personal circumstances, values, and preferences. They intended to use the Dboxes in practice (mean 5.6 ± 1.2, scale 1–7, with 7 being “high”), and their intention was significantly related to social norm, perceived behavioural control, and attitude according to the TPB (P < 0.0001). In focus groups, clinicians mentioned that co-interventions such as patient decision aids and training in shared decision-making would facilitate the use of the Dbox information. Some participants would have liked a clear “bottom line” statement for each Dbox and access to printed Dboxes in consultation rooms.
Dboxes are valued by clinicians. Tailoring of Dboxes to their needs would facilitate their implementation in practice.
- Clinical practice guidelines
- Knowledge translation
- Decision support
- Evidence-based practice
- Patient-centred care
- User experience
- Continuing professional development
- Communication competency
Patient-centred care requires enhancing the patient-clinician relationship through compassion, empathy, trust, spirituality, and sharing of power ,. Shared decision-making (SDM) structures the sharing of power between clinicians and patients by proposing joint decisions based on an understanding of the benefits and harms of all health-care options and patients’ preferences in regard to those options .
Patient decision aids (PtDAs) are among the strategies most often used to effectively facilitate SDM ,. These tools provide patients with information on the options and research-based outcomes relevant to their health status and help clarify values regarding the benefits and harms of each option . They work as “patient-directed interventions” as they actively engage patients to enhance their knowledge and health behaviour but are also “patient-mediated interventions” as they spur patients to change the behaviours of health professionals through patient-provider interaction . PtDAs have been shown to improve patients’ knowledge, reduce their decisional conflict, and lower the proportion of people passive in the decision-making process or undecided . While clinicians generally agree with SDM principles , they may be reluctant to implement SDM because they perceive it as not applicable to some patients or clinical situations or because of time constraints . In addition, the distribution of PtDAs to patients in clinical settings is arduous, and many studies that relied on clinicians to refer patients to these tools report limited utilization, often because clinicians do not view the task of referring patients to PtDAs as their role . However, in most of these studies, clinicians were either unexposed to any intervention -, received a financial incentive for time spent prescribing PtDAs  or viewing the PtDAs , or were exposed to a limited intervention such as brief training on how to use PtDAs .
Training clinicians can improve the transmission of PtDAs to patients , and increase SDM implementation in practice ,,. Participation in educational meetings can, however, be challenging for busy health-care professionals, even more so if they work in remote areas . Additionally, in primary care settings, evidence on the benefits and harms of diagnostic or treatment options and clinician training in SDM are needed for multiple health problems that are too numerous to be covered in a single educational meeting.
We have thus developed a series of two-page clinical summaries known as “decision boxes” (Dboxes) that integrate the best evidence from studies and knowledge syntheses to provide information on management options for medical questions that have no single best answer ,. Dboxes are most appropriate in the face of situations where trustworthy guidelines should issue weak recommendations . They present information in numerical, textual, and graphical formats that follow risk communication principles  to allow people to make their own informed decisions. They also use colours, layout, and typographic features to enhance first impressions, clarity of content, and user experience. They facilitate critical appraisal of the evidence as they describe the included studies’ design and population, and synthesize, using the grading of recommendations, assessment, development and evaluation (GRADE) approach , study limitations, inconsistency of results, indirectness of the evidence, imprecision, and publication bias. Dboxes are meant to help the clinician recognize equipoise and the need to share a decision with the patient and to provide the information about the risks and benefits of all the options, so clinicians are prepared for shared decision-making . Dboxes are more flexible than educational meetings, as clinicians can receive them by email or access them online, then read them at their convenience, in the setting of their choice, and use them at their own pace according to their patients’ needs. Delivery of these documents can be spaced in time, which has been shown to improve medical knowledge acquisition and retention -.
In an earlier study, we interviewed patients and clinicians after they had read two Dboxes and learned how they valued this source of information for clinicians . From the results, we adapted the Dboxes to improve users’ first impressions as well as their understanding of and trust in the information they provide. However, this first study did not allow testing the usefulness of the Dbox information in clinical settings, so we sought insight into the barriers and facilitators influencing clinicians’ transmission of the Dbox information to their patients during the primary care consultation. In line with Moore et al. , the present study aimed to document one level of outcomes of continuing education, specifically the use of educational information from Dboxes in practice. Our objectives were to measure the value of and intention to use Dboxes in practice and to describe barriers and facilitators of their use. The Dboxes will subsequently be tailored to the identified barriers in order to optimize their implementation.
As described elsewhere , this project was based on the theory of mechanisms of planned change as described in the Ottawa Model of Research Use (OMRU) ,. Quantitative and qualitative sequential phases of data collection and analysis framed this mixed methods study (sequential explanatory design) .
Participants and recruitment strategy
Using the professional networks of the research team members, we recruited six primary care clinics: two French-speaking and four English-speaking. Among these six clinics, two declined to participate in the second phase consisting of focus groups, so they participated only in the quantitative phase. We invited all the family physicians, nurses, and residents from these six clinics to participate. For phase two, we selected a purposeful sample of eight clinicians per clinic among extreme cases on the web questionnaires, i.e. clinicians who consistently attributed higher or lower scores to the Dboxes they rated. Thus, only the participants who completed at least one of the eight web questionnaires during phase one were eligible to participate in phase two. We also recruited the medical director of each clinic.
Health topic covered by each Dbox
Order of delivery
Health topic covered by Dboxes (abbreviation)
Cholinesterase inhibitors to reduce the symptoms of Alzheimer’s disease (ChEIs)
Acetylsalicylic acid for primary prevention of cardiovascular disease (ASA)
Faecal occult blood test to screen for colorectal cancer (FOBT)
Serum integrated test to screen women for fetal trisomy 21 (Prenatal)
Statins for primary prevention of cardiovascular disease (Statins)
BRCA1/2 gene mutation test to evaluate the risks of breast and ovarian cancer (BRCA)
Bisphosphonates to prevent osteoporotic fractures in postmenopausal women (Osteo)
Prostate-specific antigen test to screen men for prostate cancer (PSA)
Data collection and procedures
Phase one: quantitative study
At study entry, all participating clinicians consented and completed a questionnaire assessing their demographic and professional characteristics (age, gender, number of years of clinical practice). They also rated their interest for each of the eight Dbox topics using a visual analog scale ranging from “no interest” to “high interest”.
When receiving a Dbox, the participants completed a three-part web-based questionnaire (available in ). In part I, a 5-point rating scale assessed overall satisfaction. In part II, the validated Information Assessment Method (IAM) questionnaire spurred reflection on the cognitive impact of this information, its relevance for at least one patient in the practice, its use for this patient, and if used, any expected health benefits . The questionnaire also contained free text fields that provided an optional opportunity for readers to comment. Part III consisted of a 12-item questionnaire based on the theory of planned behaviour (TPB) , with each item scored on a −3 to +3 rating scale, to study the factors influencing their intention to use what they learned from the Dbox to explain the advantages and disadvantages of the options to their next patient to whom this intervention might apply. Based on the TPB, the three determinants of a clinician’s intention to perform a given behaviour are attitude towards performing this behaviour, subjective norm, and perceived behavioural control. Each determinant is one construct of the TPB questionnaire. Internal consistency (Cronbach’s alpha) for each construct was very good (0.89–0.94).
Phase 2: qualitative study
After the 8-week intervention, we conducted a 60-min semi-structured focus group in each clinic with a mixed group comprising family physicians, nurses, and a resident to describe how they used what they learned in the Dbox in their practice and the influence of contextual factors and team dynamics on this use (Additional file 2). A 30-min individual interview with the medical director of each clinic also provided an administrator’s perspective of the contextual factors influencing use of the Dboxes and the potential resources needed to facilitate using them. The same person (AMCG) moderated all the interviews. One observer took notes on the conduct and content of the discussions. Each discussion was audiotaped and transcribed.
Phase one: quantitative
We performed descriptive statistical analyses of the answers to the web questionnaires.
A generalized linear mixed model was used to evaluate whether clinicians’ perceptions of the Dboxes (5-point scale and IAM) were influenced by the Dbox topic, the clinicians’ profession (family physician, resident, or nurse), the clinical site, and participants’ gender. Dbox topics, clinicians’ profession, and gender were introduced as fixed effects, and the clinical site as a random effect.
For the TPB questions, we replaced missing data with the means of the values from the questions from the same construct. Repeated measure regression models were used to evaluate relationships between clinicians’ intention to use what they learned from the Dboxes and the potential determinants of their intention (attitude, subjective norm, perceived behavioural control), their profession (nurse, resident, physician), their sociodemographic characteristics (age, gender, number of years in practice), the study site, and the clinical topic of the Dbox. We also used a mixed model to test whether the determinants of the intention varied depending on the Dbox topic. All analyses were performed using SAS (Version 9.3, copyright 2002–2010 SAS Institute Inc.).
Phase two: qualitative
One researcher (AG) and three research assistants who had not been involved in the development of the Dboxes and the data collection performed a thematic qualitative data analysis of the focus group discussions and of the open-ended questions from the web questionnaire, following a hybrid deductive/inductive approach . The deductive analysis searched for OMRU attributes , and for attributes of the steps involved in the users’ experience of an evidence-based shared decision-making support tool over time , or of the IP-SDM model . The inductive analysis incorporated new themes mentioned by participants.
First, to assess whether the chosen models and frameworks applied and to explore possible sub-themes, the researcher and two research assistants went separately through the same portions of two of the focus group transcripts. The three coders then compared their results and came to a consensus on a number of themes. They noted these themes in a manual of codes, and labelled and defined them. The transcripts were entered as project documents into specialized software (NVivo 10, QSR International, Cambridge, MA, USA), and the codes developed for the manual were entered as nodes.
One of the research assistants then applied these preliminary codes to half the transcripts to identify meaningful units of text. To validate coding, this research assistant submitted 67 excerpts to a second assistant, who then linked each excerpt to one of the previously defined codes. For 44 of these excerpts (66%), the two assistants chose the same code. Among the 23 excerpts linked to different codes, 12 were discussed briefly and the codes were merged. For the 11 remaining codes, longer discussions led to the creation of new codes, improvement of code definitions, or merging of codes. The first assistant completed the coding of the remaining transcripts using this revised manual.
In the final step, a third research assistant reviewed the excerpts extracted for each code to ensure completeness and appropriateness of the code manual and consistency of approach. Coding was modified, as necessary, and modifications were verified by the first author (AMCG). This third assistant produced the qualitative analyses report that was incorporated to the present article without any modification.
Mixing quantitative and qualitative results
We interpreted quantitative and qualitative study findings together to suggest improvements to the Dbox approach in order to facilitate their implementation in clinical practice.
This project was approved by the research ethics committees of the Research Centre at Centre Hospitalier Universitaire de Quebec, Jewish General Hospital in Montreal, and McMaster University.
Characteristics of the participating clinics and clinicians
Metropolitan area population
Number of clinicians
Clinic (only C1 to C4 participated in the focus groups)
Clinic 1 (C1)
Clinic 2 (C2)
Clinic 3 (C3)
Clinic 4 (C4)
Clinic 5 (C5)
Clinic 6 (C6)
N (total = 101)
Mean age in years (SD)
N women (% women)
N English-speaking (%)
Mean years of practice (SD)
Interviews and focus groups
N (total = 27)
18 (3 clinic administrators)
Mean age in years (SD)
N women (% women)
N English-speaking (%)
Mean years of practice (SD)
Questionnaire completion rate
In total, 808 questionnaires were sent to clinicians (8 per clinician, 101 clinicians), and of these, 61% were completed (496 responses out of 808 questionnaires). Overall, 67% of the clinicians completed at least 4 of the 8 questionnaires.
Satisfaction with the decision box
Clinicians reported a level of satisfaction with the Dboxes of 4 or 5 on a 5-point smiley-face rating scale ranging from 1 (sad face) to 5 (smiling face) in 81% of questionnaires completed (373/463).
The value of decision boxes for practice
Clinicians’ report of the value of decision boxes for practice: ratings based on the Information Assessment Method (IAM)
Cognitive impact of the information
Their practice will be changed and improved
Disease prevention or health education
They learned something new
They are motivated to learn more
They were reminded of something they already knew
They are reassured
This information confirmed current practice
There is a problem with the presentation of this information
Not enough information
Too much information
They are dissatisfied
They disagree with the content of this information
This information is potentially harmful
The information is totally or partially relevant for at least one of their patients
96% (472/489; 7 missing)
Information use (for participants who reported the information to be totally or partially relevant)
They will use this information for a specific patient
To discuss with patient or with other health professionals
To change the way they manage a patient
To justify a choice
To be more certain about the management of a patient
To better understand a particular issue related to a patient
To persuade a patient or other health professionals to make a change
To decide how to manage a patient
Expected benefits of the information (for participants who reported that they will use this information for a specific patient)
They expect patient health benefits as a result of applying this information
89% (166/186; 4 missing)
Allows the patient to make a decision that is more in line with his/her personal circumstances, values, and preferences
Helps to avoid unnecessary or inappropriate treatment, diagnostic procedures, preventive interventions, or a referral for this patient
Helps reduce the patient’s uncertainty about the best decision to make
Of the questionnaires where clinicians reported Dbox information to be totally or partially relevant, 40% reported that they would actually use this information for a patient, and the most frequently reported planned use was in a discussion with a patient or with other health professionals about a patient.
Results of the generalized linear mixed modelling to evaluate the influence of independent factors (clinicians’ profession, Dbox clinical topic, clinical site, gender) on the explanatory variables of clinicians’ perception of the value of the Dbox for practice (satisfaction with regard to Dbox, IAM value of the information)
Dbox clinical topic effect
Clinical site effect
Smiley-face rating scale
Satisfaction with regard to the Dbox
n.s. (P = 0.054)
n.s. (P = 0.15)
n.s. (P = 0.15)
n.s. (P = 0.4)
Their practice will be changed and improved
ns. (P = 0.24)
P < 0.01
n.s. (P = 1.0)
n.s. (P = 0.86)
n.s. (P = 0.052)
n.s. (P = 0.95)
n.s. (P = 1.0)
n.s. (P = 0.41)
Problem with the presentation of the information
P < 0.05
P = 0.01
n.s. (P = 1.0)
n.s. (P = 0.85)
n.s. (P = 0.22)
P < 0.0001
n.s. (P = 1.0)
n.s. (P = 0.98)
n.s. (P = 0.29)
n.s. (P = 0.32)
n.s. (P = 1.0)
n.s. (P = 0.20)
n.s. (P = 0.26)
n.s. (P = 0.75)
n.s. (P = 1.0)
n.s. (P = 0.05)
Intention to use the decision box information in practice
The clinicians’ intention to use what they learned from the Dbox to explain the advantages and disadvantages of the options to their next patient to whom this intervention might apply averaged 5.6 on a scale of 1 (strongly disagree) to 7 (strongly agree), indicating that they had the intention to use what they learned. The clinicians’ intention was significantly related to social norm (P < 0.0001), perceived behavioural control (P < 0.0001), and attitude (P < 0.0001). For an increase of 1 point in each of “social norm,” “perceived behavioural control,” or “attitude,” we obtained an increase in intention of 0.2, 0.5, and 0.4 points, respectively, meaning that among the three determinants, perceived behavioural control has the most influence on this intention.
P values of the significance of the differences in the intention to use the decision box information in practice among Dbox topics (n.s. = P > 0.05)
Learning with the Dbox
Counselling patients with the Dbox
Analyses of the discussions reveal that Dboxes facilitate the patient-provider discussion by empowering clinicians (i) to provide the relevant scientific information to their patients (c12), (ii) to answer patients’ questions (c13), (iii) to consider the patients’ point of view (c14), and (iv) to come up with a plan with the patient (c15).
Clinicians mentioned that the quantitative information helped patients improve their understanding of the stakes and balance the pros and cons, thereby helping them make a better decision (c16).
Clinicians appreciated the fact that Dboxes provide patients with more precise information on multiple reasonable choices (c17). They also mentioned how presenting scientific data helps lower people’s expectations regarding the effectiveness of an intervention, thus facilitating follow-up when a treatment fails, because it clearly lays out the lack of available evidence (c18).
- 3.Critical barriers to implementation: optimizing the intervention
Adding a patient decision aid
We designed the Dboxes to be read before the consultation to facilitate the integration of the required information before the actual patient-clinician encounter. Some clinicians reported this type of use (c23). However, several clinicians considered giving the Dboxes to patients and not merely using them as training material for themselves (c24). However, the Dboxes were designed not for patients but rather as professional training tools, and accordingly most clinicians felt that the information was generally too complex to share with patients in that form (c25).
Some clinicians mentioned that it was paradoxical that Dboxes aim to support SDM when patients could not actually use them (c26), and in all the discussions, considerable emphasis was placed on developing a simplified tool for patients (c27). To adapt the Dbox to patient needs, they suggested removing the section about confidence in the results (c28) and adding visual representations of the risks to facilitate patient understanding (c29). They also proposed developing two versions tailored to lower and higher levels of patient literacy and numeracy (c30).
Clinicians generally felt that patients should have access to the patient decision aid before going through SDM with them to reduce consultation time and give some patients time to think about the options (c31). To implement this, there were suggestions that the information be delivered to patients in the waiting room, either in print or video format. There were also suggestions to go through the information in a pre-visit (c32).
Improving clarity of the information for some Dboxes
While many clinicians mentioned that they appreciated the Dboxes as learning and counselling tools, some specific Dboxes were criticized because they presented too much information or were difficult to understand (c33). Dboxes on bisphosphonates, ChEIs, and prenatal testing were generally perceived as easier to understand, whereas clinicians raised understanding issues for the Dboxes on BRCA, faecal occult blood test to screen for colorectal cancer (FOBT), and ASA. Additionally, a few participants mentioned that the Dboxes generally lacked interpretations or recommendations for them to consider and that they would have appreciated a more practical document that could be read and understood in less time (c34).
- 4.External factors influencing Dbox use
Clinicians mentioned two types of patients for whom they would be reluctant to use the Dbox information: those who prefer that clinicians make the decision (c35) and those who have already made up their mind (c36). Clinicians also felt that patient preferences with regard to their involvement in decision making are highly variable and can depend on patients’ understanding of the information, on their age (with older people less inclined to be involved), on their literacy level and socio-demographic characteristics, on how informed they are, and on social pressure to have a certain procedure performed (c37).
Accessing the Dboxes
To facilitate clinicians’ access to the Dboxes and to address their diverse preferences, participants suggested offering various formats, including applications for smart phones and paper-based formats (c38).
Clinicians mentioned that they would need some time to familiarize themselves with the Dboxes before using them in practice (c39).
One of the clinic directors mentioned that having an opinion leader endorse the Dbox would be an important determinant of its successful implementation (c40).
Having a group meeting to discuss Dboxes (e.g. a journal club) and using them for practice audit activities were perceived as helpful in facilitating the use of Dbox information in practice (c41).
Clinicians perceived the Dboxes to be more relevant in contexts where there is uncertainty with regard to clinical decisions (c42, c43). Since the Dboxes present mostly numbers and statistics, they were perceived as less useful in clinical contexts where there are important ethical issues at stake (c44). The sharing of the Dbox information was also perceived as more difficult with patients presenting multiple health problems, such as is often the case with older patients or with those who have not consulted for some time (c45). Participants perceived the Dboxes as less relevant when they dealt with a topic with which they were already comfortable (c47), when they were already using another information tool (c48), or where the presented options were not currently recommended by clinic staff (e.g. FOBT when everyone is recommending colonoscopy). Prior training in SDM was mentioned as an important facilitator to the use of Dboxes.
Organizational context (setting)
Some participants perceived private practices as a setting that could facilitate sharing the Dbox information with patients, as one might have more latitude, for example, to organize pre-visits. Some mentioned that working in a hospital environment was less favorable, because of a lack of support from other professionals (c49). On the other hand, some participants (themselves working in the public sector) mentioned how physicians in the private sector, paid through fee-for-service, would never have time to review a document during consultations but would rather use the figures. Clinicians from teaching clinics mentioned many facilitators for the use of Dboxes in their particular setting, such as being able to give longer consultations because they are salaried, being more aware of new trends in professional practice as they are frequently involved in research projects (c50), and needing to stay up to date because of their teaching responsibilities.
The clinicians discussed how following-up on all the decisions made from one visit to the next would be costly to incorporate into their clinic’s processes. They also mentioned that organizing pre-clinic visits to provide counselling would require far more human resources than those available (c51). An upgrade of the computers in each consultation room would also facilitate the use of Dboxes, but the clinics would need to find additional funding for this.
Clinicians generally perceived that delivering Dboxes to clinic nurses was useful. In one specific clinic, nurses had more responsibilities and were in charge of a significant portion of patient follow-up and phone calls, especially during more busy periods when physicians are less available (c52). In all settings, nurses mentioned often having to counsel patients on prenatal screening, and their intention to use the Dbox was in fact highest for this topic.
In one of the participating clinics, clinicians were receiving financial incentives from the government to screen patients for colorectal cancer. This was mentioned as a barrier to using the Dbox information with patients, mostly because such an incentive conveys the message that the authorities support colorectal cancer screening.
In this mixed methods implementation study of clinicians’ perceptions of eight Dboxes, which they received by email, clinicians had high intentions of using what they learned from the Dboxes to explain the pros and cons of the options to their patients. They felt that reading Dboxes improved their practice and that the information was relevant for at least one of their patients. Most were satisfied with the Dbox information, especially the figures and statistics that they valued as continuing professional development and for patient counselling. There was a consensus that to facilitate the presentation of options to their patients, the patients should get their own simplified decision aid, ideally before the consultation. The Dboxes that had been through prior testing were perceived as more valuable, suggesting that an iterative development could be required.
Learning with the Dbox
Clinicians were generally satisfied with the Dboxes that offer specific questions to identify the decisional needs of patients and a bullet format to clearly present the information related to the available options, similarly to the symptom-based protocols they are used to, that provide specific questions to assess the severity of the problem and algorithms to identify the recommended level of health-care service .
Many clinicians did not make the leap from recognizing the equipoise to disclosing the need for a decision to their patients. The Dboxes may have brought to light certain undesirable aspects of their practice. The theory of cognitive consistency predicts that information which is compatible with existing beliefs is the most likely to be accepted, and that which emphasizes the undesirable qualities of existing beliefs may be selectively avoided . This could also be because they thought their patients would have aversive reactions to dealing with uncertainty, a factor that is known to affect physicians’ willingness to engage in open communication about scientific uncertainty with their patients .
The strong emotions some participants expressed upon reading the Dboxes should not be interpreted as a negative effect of the intervention, as they could simply represent the first stage of an unfreeze-change-refreeze process. Unfreezing involves a disconfirmation of expectations and an induction of learning anxiety if the disconfirming data are accepted, an anxiety that can be converted into motivation to change .
Sharing the Dbox information with patients
Complex interventions combining patient and professional components appear to be a promising way to translate SDM into routine clinical practice. In a systematic review of 21 interventions to improve health professionals’ adoption of shared decision-making , only the three complex interventions comprising health professional training and a patient decision aid reported positive impacts ,,. Similarly, in a large study aiming to identify the factors differentiating effective and ineffective computerized clinical decision support systems, those systems providing advice to both practitioners and patients were more likely to be effective . A few simple and brief tools already exist to support the provider-patient discussion at the point of care and could be used as co-interventions to the Dboxes -.
Several of the barriers mentioned by participants were related to time constraints and patient preferences that were the most frequent external barriers reported in an earlier review of the barriers to implementing SDM in clinical practice .
Dbox development process
The Prenatal and PSA Dboxes that had already been evaluated and optimized in a first study  were consistently better perceived by clinicians. This suggests that integrating users’ evaluation of each Dbox in a feedback loop is essential and that using the Dbox template alone, although helpful to optimize the overall design of the Dbox, is not sufficient to ensure that the content matches all of the user’s decision-making and reasoning skills and information-processing needs. Iterative development, one principle of usability testing, is used increasingly in health care - and allows users to partake in designing communication materials that are not only readable and understandable but also engaging and actionable .
Implications of the results
In an earlier PtDA implementation study that used academic detailing to promote the distribution of PtDAs to patients, clinicians failed to distribute PtDAs because they did not believe that the decision was preference-sensitive . There is thus a need for an intervention prior to asking clinicians to distribute PtDAs, to provoke a shift in their recognition of equipoise and of the need for a decision. To this end, Dboxes could be distributed to clinicians and staff playing an important role in the delivery of PtDAs . This might limit the extent of the organizational commitment that is perceived as necessary in most studies of SDM implementation .
The Dbox template is valued by clinicians, so it is ready for use by producers of information resources, such as clinical practice guidelines, and systematic review developers to translate research information on the benefits and harms of health options to clinicians.
This project demonstrates the importance of end users’ involvement in the development of any communication interventions before implementation. Our work supports use of the IAM questionnaire to optimize evidence summaries as shown in previous research ,. IAM was sensitive enough to detect differences in clinicians’ perceived value of the information, and its free-text comment box allowed us to better understand how to modify and improve Dboxes.
As we do not have any information on the 40% of the invited clinicians who declined our invitation to participate in this study, we cannot ascertain whether participating clinicians were representative of the target audience. We also cannot extend our conclusions to specialized health care. To improve clinician participation in the next phase of this program, we plan to offer continuing professional development credits for each questionnaire completed, as credits represent a significant incentive for some physicians ,.
The participants felt that all possible topics should have been covered. With only eight Dboxes, they could not offer SDM consistent with all the decisions to be made by a single patient. It would be helpful if Dboxes covered all the available options as this would ensure they reach out to clinicians with different practice styles. For instance, clinicians who always use colonoscopy would not be as interested in a Dbox that compares screening with FOBT to no screening at all.
The measured questionnaire completion rate is relatively high compared to other similar studies . However, we have not explored the reasons why participants did not look at the Dboxes in the first place, so we cannot rule out a selection bias if non-responders were less satisfied with the Dboxes than those who completed the questionnaires.
Clinicians’ perceptions of the Dbox evidence summaries are promising, supporting the idea that we should start using this type of knowledge product to present research data for clinical questions where there is uncertainty that could lead to decisional conflict. However, to go one step further in the SDM process, we should evaluate whether the Dboxes really do support clinicians in involving their patients in SDM. We also need to test diverse development processes, such as using the IAM questionnaire for clinician feedback to optimize each tool or conducting individual interviews that are more resource intensive. We also need to confirm whether two iterations of user feedback are enough to optimize the Dboxes.
This project was funded by the KT Canada Research Network and by the Foundation for Informed Medical Decision Making. AMCG also received a post-doctoral fellowship from the KT Canada Research Network. We wish to acknowledge the help of many research associates and students: Debi Banerjee, Iqra Syed, Amélie Trépanier, Mike Shulha, Allison Brown, Annie Frappier, Carole Puma, Mathieu Bujold, Alain Noël, and Johanna Geraci. We also wish to thank the staff of each clinic; Josée Boulet, who helped design the Dboxes; Katherine Hastings for her writing assistance; and the Information Technology Office of the Faculty of Medicine at Laval University for developing the Dbox website.
- Stewart M: Reflections on the doctor-patient relationship: from evidence and experience. Br J Gen Pract. 2005, 55: 793-801.PubMedPubMed CentralGoogle Scholar
- Morrow CE, Reed VA, Eliassen MS, Imset I: Shared decision making: skill acquisition for year III medical students. Fam Med. 2011, 43: 721-725.PubMedGoogle Scholar
- Charles C, Gafni A, Whelan T: Decision-making in the physician-patient encounter: revisiting the shared treatment decision-making model. Soc Sci Med. 1999, 49: 651-661. 10.1016/S0277-9536(99)00145-8.View ArticlePubMedGoogle Scholar
- Legare F, Ratte S, Stacey D, Kryworuchko J, Gravel K, Graham ID, Turcotte S: Interventions for improving the adoption of shared decision making by healthcare professionals. Cochrane Database Syst Rev. 2010, 5: CD006732.PubMedGoogle Scholar
- Legare F, Turcotte S, Stacey D, Ratte S, Kryworuchko J, Graham ID: Patients’ perceptions of sharing in decisions: a systematic review of interventions to enhance shared decision making in routine clinical practice. Patient. 2012, 5: 1-19. 10.2165/11592180-000000000-00000.View ArticlePubMedGoogle Scholar
- Stacey D, Legare F, Col NF, Bennett CL, Barry MJ, Eden KB, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Thomson R: Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2014, 1: CD001431.PubMedGoogle Scholar
- Stacey D, Hill S: Patient-direct and patient-mediated KT interventions. Knowledge Translation in Health Care: Moving from Evidence to Practice. Edited by: Straus SE, Tetroe J, Graham ID. 2013, Wiley, ChichesterGoogle Scholar
- Lin GA, Halley M, Rendle KAS, Tietbohl C, May SG, Trujillo L, Frosch DL: An effort to spread decision aids in five California primary care practices yielded low distribution, highlighting hurdles. Health Aff. 2013, 32: 311-320. 10.1377/hlthaff.2012.1070.View ArticleGoogle Scholar
- Legare F, Ratte S, Gravel K, Graham ID: Barriers and facilitators to implementing shared decision-making in clinical practice: update of a systematic review of health professionals’ perceptions. Patient Educ Couns. 2008, 73: 526-535. 10.1016/j.pec.2008.07.018.View ArticlePubMedGoogle Scholar
- Elwyn G, Scholl I, Tietbohl C, Mann M, Edwards A, Clay C, Legare F, Weijden Tvd, Lewis C, Wexler R, Frosch D: "Many miles to go ...": a systematic review of the implementation of patient decision support interventions into routine clinical practice. BMC Medical Informatics and Decision Making. 2013, 13: S14-10.1186/1472-6947-13-S2-S14.View ArticlePubMedPubMed CentralGoogle Scholar
- Brackett C, Kearing S, Cochran N, Tosteson AN, Blair Brooks W: Strategies for distributing cancer screening decision aids in primary care. Patient Educ Couns. 2010, 78: 166-168. 10.1016/j.pec.2009.06.013.View ArticlePubMedGoogle Scholar
- Silvia KA, Ozanne EM, Sepucha KR: Implementing breast cancer decision aids in community sites: barriers and resources. Health Expect. 2008, 11: 46-53. 10.1111/j.1369-7625.2007.00477.x.View ArticlePubMedGoogle Scholar
- Silvia KA, Sepucha KR: Decision aids in routine practice: lessons from the breast cancer initiative. Health Expect. 2006, 9: 255-264. 10.1111/j.1369-7625.2006.00393.x.View ArticlePubMedGoogle Scholar
- Feibelmann S, Yang TS, Uzogara EE, Sepucha K: What does it take to have sustained use of decision aids? A programme evaluation for the Breast Cancer Initiative. Health Expect. 2011, 14 (Suppl 1): 85-95. 10.1111/j.1369-7625.2010.00640.x.View ArticlePubMedGoogle Scholar
- Frosch DL, Singer KJ, Timmermans S: Conducting implementation research in community-based primary care: a qualitative study on integrating patient decision support interventions for cancer screening into routine practice. Health Expect. 2011, 14 (Suppl 1): 73-84. 10.1111/j.1369-7625.2009.00579.x.View ArticlePubMedGoogle Scholar
- Elwyn G, Rix A, Holt T, Jones D: Why do clinicians not refer patients to online decision support tools? Interviews with front line clinics in the NHS. BMJ Open. 2012, 2: e001530-10.1136/bmjopen-2012-001530.View ArticlePubMedPubMed CentralGoogle Scholar
- Uy V, May SG, Tietbohl C, Frosch DL: Barriers and facilitators to routine distribution of patient decision support interventions: a preliminary study in community-based primary care settings. Health Expect. 2012, 17: 353-364. 10.1111/j.1369-7625.2011.00760.x.View ArticlePubMedGoogle Scholar
- Holmes-Rovner M, Valade D, Orlowski C, Draus C, Nabozny-Valerio B, Keiser S: Implementing shared decision-making in routine practice: barriers and opportunities. Health Expect. 2000, 3: 182-191. 10.1046/j.1369-6513.2000.00093.x.View ArticlePubMedGoogle Scholar
- O’Cathain A, Walters SJ, Nicholl JP, Thomas KJ, Kirkham M: Use of evidence based leaflets to promote informed choice in maternity care: randomised controlled trial in everyday practice. BMJ. 2002, 324: 643-10.1136/bmj.324.7338.643.View ArticlePubMedPubMed CentralGoogle Scholar
- Krones T, Keller H, Sonnichsen A, Sadowski EM, Baum E, Wegscheider K, Rochon J, Donner-Banzhoff N: Absolute cardiovascular disease risk and shared decision making in primary care: a randomized controlled trial. Ann Fam Med. 2008, 6: 218-227. 10.1370/afm.854.View ArticlePubMedPubMed CentralGoogle Scholar
- Stacey D, Graham ID, O’Connor AM, Pomey MP: Barriers and facilitators influencing call center nurses’ decision support for callers facing values-sensitive decisions: a mixed methods study. Worldviews Evid Based Nurs. 2005, 2: 184-195. 10.1111/j.1741-6787.2005.00035.x.View ArticlePubMedGoogle Scholar
- Legare F, Labrecque M, Cauchon M, Castel J, Turcotte S, Grimshaw J: Training family physicians in shared decision-making to reduce the overuse of antibiotics in acute respiratory infections: a cluster randomized trial. CMAJ. 2012, 184: E726-E734. 10.1503/cmaj.120568.View ArticlePubMedPubMed CentralGoogle Scholar
- Bieber C, Muller KG, Blumenstiel K, Schneider A, Richter A, Wilke S, Hartmann M, Eich W: Long-term effects of a shared decision-making intervention on physician-patient interaction and outcome in fibromyalgia. A qualitative and quantitative 1 year follow-up of a randomized controlled trial. Patient Educ Couns. 2006, 63: 357-366. 10.1016/j.pec.2006.05.003.View ArticlePubMedGoogle Scholar
- Allaire AS, Labrecque M, Giguere A, Gagnon MP, Legare F: What motivates family physicians to participate in training programs in shared decision making?. J Contin Educ Health Prof. 2012, 32: 98-107. 10.1002/chp.21132.View ArticlePubMedGoogle Scholar
- Giguere A, Légaré F, Grad R, Pluye P, Haynes R, Cauchon M, Rousseau F, Argote J, Labrecque M: Decision boxes for clinicians to support evidence-based practice and shared decision making: the user experience. Implem Sci. 2012, 7: 72-10.1186/1748-5908-7-72.View ArticleGoogle Scholar
- Giguere A, Legare F, Grad R, Pluye P, Rousseau F, Haynes RB, Cauchon M, Labrecque M: Developing and user-testing Decision boxes to facilitate shared decision making in primary care - a study protocol. BMC Med Inform Decis Mak. 2011, 11: 17-26. 10.1186/1472-6947-11-17.View ArticlePubMedPubMed CentralGoogle Scholar
- van der Weijden T, Boivin A, Burgers J, Schunemann HJ, Elwyn G: Clinical practice guidelines and patient decision aids. An inevitable relationship. J Clin Epidemiol. 2012, 65: 584-589. 10.1016/j.jclinepi.2011.10.007.View ArticlePubMedGoogle Scholar
- Fischhoff B, Brewer NT, Downs JS: Communicating Risks and Benefits: An Evidence-Based User’s Guide. 2011, Food and Drug Administration (FDA), US Department of Health and Human Services, Silver SpringGoogle Scholar
- Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ: GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011, 64: 383-394. 10.1016/j.jclinepi.2010.04.026.View ArticlePubMedGoogle Scholar
- Légaré F, Witteman HO: Shared decision making: examining key elements and barriers to adoption into routine clinical practice. Health Aff. 2013, 32: 276-284. 10.1377/hlthaff.2012.1078.View ArticleGoogle Scholar
- Kerfoot BP, Brotschi E: Online spaced education to teach urology to medical students: a multi-institutional randomized trial. Am J Surg. 2009, 197: 89-95. 10.1016/j.amjsurg.2007.10.026.View ArticlePubMedGoogle Scholar
- Kerfoot BP, DeWolf WC, Masser BA, Church PA, Federman DD: Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial. Med Educ. 2007, 41: 23-31. 10.1111/j.1365-2929.2006.02644.x.View ArticlePubMedGoogle Scholar
- Kerfoot BP, Baker HE, Koch MO, Connelly D, Joseph DB, Ritchey ML: Randomized, controlled trial of spaced education to urology residents in the United States and Canada. J Urology. 2007, 177: 1481-1487. 10.1016/j.juro.2006.11.074.View ArticleGoogle Scholar
- Moore DE, Green JS, Gallis HA: Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009, 29: 1-15. 10.1002/chp.20001.View ArticlePubMedGoogle Scholar
- Giguere A, Labrecque M, Grad R, Cauchon M, Greenway M, Legare F, Pluye P, Turcotte S, Dolovich L, Haynes RB: Barriers and facilitators to implementing Decision Boxes in primary healthcare teams to facilitate shared decision making: a study protocol. BMC Med Inform Decis Mak. 2012, 12: 85-10.1186/1472-6947-12-85.View ArticlePubMedPubMed CentralGoogle Scholar
- Graham ID, Logan J: Innovations in knowledge transfer and continuity of care. Can J Nurs Res. 2004, 36: 89.PubMedGoogle Scholar
- Graham K, Logan J: Using the Ottawa model of research use to implement a skin care program. J Nurs Care Qual. 2004, 19: 18-26. 10.1097/00001786-200401000-00006.View ArticlePubMedGoogle Scholar
- Pluye P, Nha Hong Q: Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014, 35: 29-45. 10.1146/annurev-publhealth-032013-182440.View ArticlePubMedGoogle Scholar
- Pluye P, Grad RM, Johnson-Lafleur J, Bambrick T, Burnand B, Mercer J, Marlow B, Campbell C: Evaluation of email alerts in practice: part 2 - validation of the information assessment method. J Eval Clin Pract. 2010, 16: 1236-1243. 10.1111/j.1365-2753.2009.01313.x.View ArticlePubMedGoogle Scholar
- Ajzen I: The theory of planned behavior. Organ Behav Hum Decis Process. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.View ArticleGoogle Scholar
- Fereday J, Muir-Cochrane E: Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Meth. 2006, 5: 80-92.Google Scholar
- Legare F, Stacey D, Gagnon S, Dunn S, Pluye P, Frosch D, Kryworuchko J, Elwyn G, Gagnon MP, Graham ID: Validating a conceptual model for an inter-professional approach to shared decision making: a mixed methods study. J Eval Clin Pract. 2011, 17: 554-564. 10.1111/j.1365-2753.2010.01515.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Marriott S, Palmer C, Lelliott P: Disseminating healthcare information: getting the message across. Qual Health Care. 2000, 9: 58-62. 10.1136/qhc.9.1.58.View ArticlePubMedPubMed CentralGoogle Scholar
- Portnoy DB, Han PK, Ferrer RA, Klein WM, Clauser SB: Physicians’ attitudes about communicating and managing scientific uncertainty differ by perceived ambiguity aversion of their patients. Health Expect. 2011, 16: 362-372. 10.1111/j.1369-7625.2011.00717.x.View ArticlePubMedGoogle Scholar
- Weick KE, Quinn RE: Organizational change and development. Annu Rev Psychol. 1999, 50: 361-386. 10.1146/annurev.psych.50.1.361.View ArticlePubMedGoogle Scholar
- Lewiecki EM: Risk communication and shared decision making in the care of patients with osteoporosis. J Clin Densitom. 2010, 13: 335-345. 10.1016/j.jocd.2010.06.005.View ArticlePubMedGoogle Scholar
- Legare F, St-Jacques S, Gagnon S, Njoya M, Brisson M, Fremont P, Rousseau F: Prenatal screening for Down syndrome: a survey of willingness in women and family physicians to engage in shared decision-making. Prenat Diagn. 2011, 31: 319-326. 10.1002/pd.2624.View ArticlePubMedGoogle Scholar
- Bieber C, Muller KG, Blumenstiel K, Hochlehnert A, Wilke S, Hartmann M, Eich W: A shared decision-making communication training program for physicians treating fibromyalgia patients: effects of a randomized controlled trial. J Psychosom Res. 2008, 64: 13-20. 10.1016/j.jpsychores.2007.05.009.View ArticlePubMedGoogle Scholar
- Loh A, Simon D, Wills CE, Kriston L, Niebling W, Harter M: The effects of a shared decision-making intervention in primary care of depression: a cluster-randomized controlled trial. Patient Educ Couns. 2007, 67: 324-332. 10.1016/j.pec.2007.03.023.View ArticlePubMedGoogle Scholar
- Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Spall HG, Garg AX, Haynes RB: Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013, 346: f657-10.1136/bmj.f657.View ArticlePubMedGoogle Scholar
- Elwyn G, Lloyd A, Joseph-Williams N, Cording E, Thomson R, Durand MA, Edwards A: Option grids: shared decision making made easier. Patient Educ Couns. 2013, 90: 207-212. 10.1016/j.pec.2012.06.036.View ArticlePubMedGoogle Scholar
- Mullan RJ, Montori VM, Shah ND, Christianson TJ, Bryant SC, Guyatt GH, Perestelo-Perez LI, Stroebel RJ, Yawn BP, Yapuncich V, Breslin MA, Pencille L, Smith SA: The diabetes mellitus medication choice decision aid: a randomized trial. Arch Intern Med. 2009, 169: 1560-1568. 10.1001/archinternmed.2009.293.View ArticlePubMedGoogle Scholar
- Canadian Medical Association: DECISION 2+: diagnostic decision support tools. 2010,[http://www.cmaj.ca/content/suppl/2012/08/03/cmaj.120568v1.DC1/train-legare-1-at.pdf]
- Kastner M, Lottridge D, Marquez C, Newton D, Straus SE: Usability evaluation of a clinical decision support tool for osteoporosis disease management. Implement Sci. 2010, 5: 96-10.1186/1748-5908-5-96.View ArticlePubMedPubMed CentralGoogle Scholar
- Kushniruk A: Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Comput Biol Med. 2002, 32: 141-149. 10.1016/S0010-4825(02)00011-2.View ArticlePubMedGoogle Scholar
- Weyand SA, Frize M, Bariciak E, Dunn S: Development and usability testing of a parent decision support tool for the neonatal intensive care unit. Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Conference. 2011, 2011: 6430-6433.Google Scholar
- Neuhauser L, Paul K: Readablity, comprehension, and usability. Communicating Risks and Benefits: An Evidence-Based User’s Guide. Edited by: Fischhoff B, Brewer NT, Downs JS. 2011, Food and Drug Administration (FDA), US Department of Health and Human Services, Silver Spring, 129-148.Google Scholar
- Pluye P, Grad RM, Repchinsky C, Jovaisas B, Lewis D, Tang DL, Granikov V, Bonar JG, Marlow B: Better than best’ evidence? The Information Assessment Method can help information providers to use family physicians’ feedback for 2-way knowledge translation. Can Fam Physician. 2014, 60: 415-417.PubMedPubMed CentralGoogle Scholar
- Tang DL: Towards Optimal Management of Health Information Users’ Feedback: The Case of the Canadian Pharmacists Association. 2012, McGill, MontrealGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.