Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Involving patients in setting priorities for healthcare improvement: a cluster randomized trial

  • Antoine Boivin1, 2, 3Email author,
  • Pascale Lehoux4,
  • Réal Lacombe3,
  • Jako Burgers5 and
  • Richard Grol2
Implementation Science20149:24

DOI: 10.1186/1748-5908-9-24

Received: 25 July 2013

Accepted: 18 February 2014

Published: 20 February 2014

Abstract

Background

Patients are increasingly seen as active partners in healthcare. While patient involvement in individual clinical decisions has been extensively studied, no trial has assessed how patients can effectively be involved in collective healthcare decisions affecting the population. The goal of this study was to test the impact of involving patients in setting healthcare improvement priorities for chronic care at the community level.

Methods

Design: Cluster randomized controlled trial. Local communities were randomized in intervention (priority setting with patient involvement) and control sites (no patient involvement). Setting: Communities in a canadian region were required to set priorities for improving chronic disease management in primary care, from a list of 37 validated quality indicators. Intervention: Patients were consulted in writing, before participating in face-to-face deliberation with professionals. Control: Professionals established priorities among themselves, without patient involvement. Participants: A total of 172 individuals from six communities participated in the study, including 83 chronic disease patients, and 89 health professionals. Outcomes: The primary outcome was the level of agreement between patients’ and professionals’ priorities. Secondary outcomes included professionals’ intention to use the selected quality indicators, and the costs of patient involvement.

Results

Priorities established with patients were more aligned with core generic components of the Medical Home and Chronic Care Model, including: access to primary care, self-care support, patient participation in clinical decisions, and partnership with community organizations (p < 0.01). Priorities established by professionals alone placed more emphasis on the technical quality of single disease management. The involvement intervention fostered mutual influence between patients and professionals, which resulted in a 41% increase in agreement on common priorities (95%CI: +12% to +58%, p < 0.01). Professionals’ intention to use the selected quality indicators was similar in intervention and control sites. Patient involvement increased the costs of the prioritization process by 17%, and required 10% more time to reach consensus on common priorities.

Conclusions

Patient involvement can change priorities driving healthcare improvement at the population level. Future research should test the generalizability of these findings to other contexts, and assess its impact on patient care.

Trial registration

The Netherlands National Trial Register #NTR2496.

Keywords

Patient participation Quality improvement Health policy Primary care Chronic diseases Randomized trial

Background

Patients are increasingly seen as active partners in healthcare. The rise of chronic diseases highlights the importance of productive interactions between patients and health professionals [1, 2]. Internationally, a number of healthcare organizations are involving patients in health services delivery and policy decisions, including: guideline and quality indicator development, program development and evaluation, quality improvement, and funding priorities [38]. This growth in patient involvement includes large national organizations (e.g., Patient Centered Outcomes Research Institute, and the National Institute for Clinical Excellence), as well as local healthcare organizations (e.g., Primary Care Trusts, and Patient-Centered Medical Homes) [911].

While patient involvement is widely advocated, its actual impact on healthcare delivery and policy decisions is largely unknown. In the past decades, over 200 trial have assessed the impact of involving patients in individual clinical decisions (e.g., patient decision aids, self-management support, and health education) [1218]. In contrast, no trial assessed the impact of patient involvement on collective healthcare decisions affecting the population [1926]. As a result, the latest Cochrane review on this topic concluded that there is ‘a huge gap in the evidence from comparative studies about desirable and adverse effects of [patient] involvement in healthcare decisions at the population level’ [19].

Deliberation theory posits that effective patient involvement could foster mutual influence and increased agreement between patients and professionals, resulting in collective decisions about healthcare services and policies that are more acceptable by those affected [24, 27, 28]. A number of case reports and observational studies have tested this hypothesis with conflicting results [19, 2124, 29, 30]. Several challenges have been identified to support patient involvement at the population level. First, lack of understanding of scientific literature or resource implications could lead to unrealistic decisions [21]. Unbalanced recruitment may underrepresent the views of vulnerable patients with complex conditions or from disadvantaged socio-economic groups [31]. Finally, a number of critics doubt that patients can actually influence professionals’ decisions and have an impact on collective healthcare choices.

This article reports on results of the first trial of patient involvement in collective healthcare decisions affecting the population. Our primary objective was to assess the impact of patient involvement on healthcare improvement priorities at the community level. We hypothesized that patient involvement would result in mutual influence between patients and professionals, and increase agreement on common improvement priorities.

Methods

Study setting

This study was implemented in a real-world priority-setting exercise organized by the Regional Health Authority of Abitibi-Témiscamingue (Québec), Canada. The region has a population of 145,000 people, and is divided into six local communitites. Local Health and Social Services Centers are responsible for primary healthcare delivery in each of these communitites through direct service provision and contract with primary care providers [32]. In 2010–2011, the Regional Health Authority required its six local Health and Social Services Centers to set priorities for improving chronic disease prevention and management in primary care. Each Health and Social Services Center could choose its own improvement priorities from a list of 37 validated quality indicators. Selected priorities were incorporated in their financial accountability contracts with the Regional Health Authority.

Design

We conducted a cluster-randomized controlled trial comparing healthcare improvement priority-setting with and without patient involvement (Figure 1). At baseline, we collected patients’ priorities from all participating sites. Health and Social Services Centers were then randomized in intervention sites (priority-setting with patient involvement) and control sites (priority-setting by professionals alone, without patient involvement).
https://static-content.springer.com/image/art%3A10.1186%2F1748-5908-9-24/MediaObjects/13012_2013_Article_737_Fig1_HTML.jpg
Figure 1

Trial overview. Patients and professionals were recruited from 6 eligible Health and Social Services Centers. At baseline, patients from all sites were consulted by vote on their baseline priorities. Health and Social Services Centers were then randomized in intervention (priority-setting with patient involvement) and control sites (priority-setting without patient involvement).

Our detailed study protocol has been published elsewhere [33]. We pilot-tested our intervention with 27 participants (15 patients and 12 professionals) in a pilot site outside of the study region, and tested our research instruments with an additional 21 participants (11 patients and 10 professionals) [33]. The study was conducted between January 2010 and May 2011 and was approved by the Université du Québec en Abitibi-Témiscamingue ethics committee.

Quality indicators used as healthcare improvement priorities

In preparation for the trial, we conducted a systematic review of validated quality indicators for chronic disease prevention and management in primary care (including diabetes, chronic obstructive pulmonary disease, and cardiovascular disease) [33]. A total of 1,489 individual quality indicators were identified, 801 of which met our inclusion criteria. A panel of five experts (two physicians, two managers, and one information specialist) independently rated each indicator based on its measurability with existing information systems, and applicability to the canadian primary care context. The comprehensiveness of this preliminary indicator set was tested with a group of nine patients and 11 professionals to ensure that it included important dimensions of quality. Two indicators were added in response to patients and professionals’ comments. The final list of indicators included 37 quality indicators grouped into five domains: access, continuity and integration of care, technical quality of disease prevention and management, interpersonal relationships, and outcomes. Details of the quality indicators used in the trial are available at: http://www.implementationscience.com/content/6/1/45/additional[33].

Recruitment

We sought to recruit the whole population of available study sites in the region (target: six communities). Within each study site, we created a recruitment team composed of the medical director, the local chief executive officer, a manager in charge of chronic disease services, and a patient sitting on the Health and Social Services Center’s user committee. Recruitment teams were responsible for identifying a diversified pool of potential participants through open advertizing, healthcare and patient organizations, and snowballing technique [34].

Selection of participants was conducted by a blinded research assistant, before randomization. The research assistant contacted potential participating patients, collected socio-demographic characteristics, confirmed eligibility criteria, and selected patients using stratified random selection to ensure a balanced representation of age, gender, socio-economic condition, and health status.We sought to recruit patients targeted by chronic disease prevention and management services, including: healthy adults without chronic disease; patients with uncomplicated chronic disease (diabetes, chronic obstructive pulmonary disease, or cardiovascular disease); and patients with complex chronic conditions hospitalized in the previous year. Patients who trained or worked as a health professional were excluded. Professionals were selected to include a balanced group of clinicians and managers involved in chronic disease prevention and management, including: primary care physicians, allied health professionals directly involved in patient care, managers responsible for chronic disease services and existing information systems, and the chief executive officer of each Health and Social Services Center.

Patients consultation meetings

A patient consultation meeting was conducted in all sites before randomization (Figure 1). This one-day patient consultation meeting (target = 15 patients/site) was facilitated by an expert moderator. Patients recounted their personal experiences of care in relation with chronic disease prevention and management, received information about primary care services from their community, and were consulted individually by vote to select five quality indicators that they believed should be prioritized for healthcare improvement in their community (‘baseline patient consultation’).

This patient consultation meeting had two functions. First, it was used for outcome assessment in intervention and control sites, by providing baseline data on patients’ priorities for healthcare improvement. Second, the patient consultation meeting was a component of the involvement intervention aimed at preparing patients who would later participate in deliberation meetings with professionals.

Randomization

Blocked randomization [35] of study sites was performed after participants’ recruitment, and after completion of the patients’ baseline consultation meetings. The random allocation sequence was generated by an independent expert through a randomization software [36]. Participants were thereafter unblinded to their assignment. Participants were required not to disclose any information from meetings until completion of the trial, which we verified at the end of the study. Study sites were more than 100 km apart from one another, and we found no evidence of contamination between intervention and control sites.

Intervention

In intervention sites, Health and Social Services Centers established their improvement priorities with patient involvement. Our patient involvement intervention combined public consultation and participation methods [37], as follows: 1) professionals received feedback on priorities collected from all patients in the baseline consultation meeting; and 2) professionals and patients participated in a two-day deliberation meeting to agree on healthcare improvement priorities for their community, using nominal group technique [38]. Patients who participated in the baseline consultation meeting were invited to participate in the deliberation meeting (target = five patients/site) and were selected by a research assistant through stratified random sampling to include a balanced representation of age, gender, health status, and socio-economic conditions. Each Health and Social Services Center was allowed to select its own priorities. An expert moderator facilitated all meetings with two co-moderators.

Control

Health and Social Services Centers in control sites established their improvement priorities with professionals only, without patient involvement. Professionals in control sites did not receive feedback on patients’ baseline priorities. Professionals deliberated among themselves to agree on healthcare improvement priorities in a two-day deliberation meeting, using similar nominal group techniques and moderators as in other sites.

Data collection and analysis

The primary study outcome was the level of agreement between patients’ and professionals’ priorities. Secondary outcomes included changes in priorities, professionals’ intentions to use quality indicators for healthcare improvement, and the costs of patient involvement.

At baseline and at the end of the trial, each participant was asked to identify five healthcare improvement priorities from the list of 37 quality indicators. Priorities were ranked based on the proportion of participants who selected them. Agreement between patients’ and professionals’ priorities was calculated using the Spearman correlation coefficient (0 = no agreement; 1 = perfect agreement). Statistical differences in correlation coefficients between intervention and control sites were tested using Fisher r-to-z transformation [39], and we used multi-level analysis to account for the clustering of priority scores at the community level [40]. Differences in improvement priorities were calculated from the proportion of participants who selected each indicator, and tested using Generalized Estimating Equation [41]. All differences were assumed to be significant at p <0.05 (two-tailed test).

Professionals’ intention to use the selected priorities for healthcare improvement were collected at the end of the trial from an 11-item questionnaire measuring their perception of the credibility, feasibility, and importance of the selected quality indicators, as well as their intention to use them for healthcare improvement. Each item was scored on a seven-point Likert scale. Mean score differences between intervention and control sites were tested using multi-level analysis. We calculated the marginal cost of patient involvement (patient recruitment, training, and financial compensation of $100/day) in relation with total project cost (health professionals’ salary, quality indicator menu development, moderation of meetings, and project coordination). All costs were calculated from the sponsoring organization’s perspective and assumed an average of 15 patients per training meeting and five patients per deliberation meeting. All statistical analyses were conducted with SAS 9.

Results

Participants

A total of 172 individuals from the six eligible sites participated in the study. Characteristics of study sites and individual participants were similar in intervention and control groups (Table 1). Professionals (n = 89) included managers (35%), physicians (13%), nurses (24%), and allied health professionals (28%). Patients (n = 83) included adults with different health status: 81% of patients had at least one chronic condition, 37% had three chronic conditions or more, and 24% had been hospitalized at least once in the past year. Patients also included individuals from diverse socio-economic status: 15% had a family income of less than $20,000 and 58% had a primary or high-school level education.
Table 1

Characteristics of study sites and individual participants

Sites' characteristics

Intervention

Control

(n = 3)

(n = 3)

 

Average population size

25.002

23.610

 

Family physician/population ratio

1/771

1/789

 

% population 65 y.o and above

15.6%

13.4%

 

% population with low income

12.3%

11.3%

 

% population with diabetes

6.5%

6.5%

Professionals' characteristics

 

Intervention

Control

 

(n = 44)

(n = 45)

  

n

(%)

n

(%)

Age

    
 

20 to 44 years

21

(47.7)

19

(48.7)

 

45 to 64 years

20

(45.5)

20

(51.3)

 

65 years or more

3

(6.8)

0

(0.0)

Gender

    
 

Male

9

(20.5)

10

(23.3)

 

Female

35

(79.5)

33

(76.7)

Type of work

    
 

Physician

6

(13.6)

6

(13.3)

 

Nurse

11

(25.0)

10

(22.2)

 

Allied health professional

11

(25.0)

14

(31.1)

 

Manager

16

(36.4)

15

(33.3)

Patients’ characteristics

 

Baseline consultation

Deliberation

(n = 83)

(n = 17)

  

n

(%)

n

(%)

Age

    
 

20 to 44 years

12

(14.6)

3

(18.8)

 

45 to 64 years

44

(53.7)

9

(56.3)

 

65 years or more

26

(31.7)

4

(25.0)

Gender

    
 

Male

36

(43.4)

10

(58.8)

 

Female

47

(56.6)

7

(41.2)

Family income

    
 

Less than 20 000 $

16

(20.0)

3

(17.6)

 

From 20 000 $ up to 39 999 $

29

(36.3)

8

(47.1)

 

From 40 000 $ up to 59 999 $

13

(16.3)

4

(23.5)

 

60 000 $ or more

22

(27.5)

2

(11.8)

Education level

    
 

Primary school

12

(14.8)

0

(0.0)

 

High school

35

(43.2)

7

(41.2)

 

College or University

34

(42.0)

10

(58.8)

Specific chronic conditions

    
 

Arthritis

13

(15.7)

3

(17.6)

 

Cardiovascular disease

19

(22.9)

3

(17.6)

 

Chronic pain

11

(13.3)

3

(17.6)

 

Diabetes

30

(36.1)

2

(11.8)

 

Dyslipidemia

22

(26.5)

5

(29.4)

 

Hypertension

30

(36.1)

6

(35.3)

 

Mood disorder

9

(10.8)

1

(5.9)

 

Pulmonary disease

17

(20.5)

1

(5.9)

 

Other

24

(28.9)

3

(17.6)

Number of chronic conditions

    
 

0

16

(19.3)

5

(29.4)

 

1

21

(25.3)

5

(29.4)

 

2 or more

46

(55.4)

7

(41.2)

Hospitalisations in the past 12 months

    
 

0

62

(76.5)

13

(76.5)

 

1

10

(12.3)

2

(11.8)

 

2 or more

9

(11.1)

2

(11.8)

All patients (n = 83) participated in the baseline consultation, and five to six patients per site (n = 17) also participated in the deliberation meetings with intervention site professionals. All patients completed the study and one professional did not.

Healthcare improvement priorities

Patients’ baseline priorities were different from those of professionals (Figure 2). Patients placed more importance than professionals on access to primary care, respect and empathy, time available in the consultation, and treatment costs (p < 0.01).
https://static-content.springer.com/image/art%3A10.1186%2F1748-5908-9-24/MediaObjects/13012_2013_Article_737_Fig2_HTML.jpg
Figure 2

Baseline improvement priorities for patients and professionals. Baseline improvement priorities of patients (red) and professionals (blue). **p < 0.01.

At the end of the trial, intervention sites’ priorities identified by patients with professionals were significantly different from control sites’ priorities identified by professionals alone (Figure 3). Priorities established with patients placed more importance on generic aspects of quality, including access to primary care, self-care support, patient participation in clinical decision-making, and partnership with community organizations (p < 0.01). Priorities established by professionals alone placed more emphasis on reducing emergency room visits, collaboration among healthcare organizations, and the technical quality of single disease management (p < 0.01).
https://static-content.springer.com/image/art%3A10.1186%2F1748-5908-9-24/MediaObjects/13012_2013_Article_737_Fig3_HTML.jpg
Figure 3

Final improvement priorities in intervention and control sites. Final healthcare improvement priorities in intervention sites (purple) and control sites (green). *p < 0,05 **p < 0,01.

Differences in healthcare improvement priorities were explained by a process of mutual influence between patients and professionals. Intervention sites selected priorities that both patients and professionals could agreed on. Accordingly, quality indicators that were initially supported by both professionals and patients (e.g., family physicians accepting new patients, patient participation in clinical decision-making, and partnership with community organizations) were selected more often in intervention sites. Initial disagreement between professionals and patients resulted in changes of opinions on both sides. For example, intervention sites’ professionals moved toward priorities that were initially favored by patients (e.g., timely appointment with a primary care provider) and away from indicators that received little patient support (e.g., collaboration among healthcare organizations). Patients’ priorities also moved toward indicators favored by professionals (e.g., self-care support) and away from priorities that received little professional support (e.g., respect and empathy). As a result of this process of mutual influence, agreement between patients and professionals increased by 41% favouring intervention sites (95%CI: +12% to +58%; p < 0 .01) (Table 2).
Table 2

Agreement between patients and professionals’ priorities

 

Agreement at baseline

Agreement at the end of the trial

Change in agreement during the trial

 

r

r

% change (95%CI)

Intervention

0.27

0.69

+42% (+13%, +58%)

Control

0.18

0.19

+1% (−25%, +27%)

Difference between intervention and control

+9%

+ 50%

+41% (+12%, +58%)

p value

0.62

<0.001

<0.01

Agreement between patients and professionals’ priorities at baseline and at the end of the trial (r = 0: no agreement, r = 1: perfect agreement). P value is reported for the difference between intervention and control sites.

Priorities at control sites reflected those selected by professionals at baseline, and remained at odd with patients’ priorities. For example, none of the control sites’ professionals prioritized timely appointments with a primary care provider, although this was the second most important priority identified by patients.

Intention to use

Professionals’ intention to use selected indicators for healthcare improvement scored high in both intervention and control groups (Additional file 1). There was no difference in professionals’ perceptions of the credibility, importance, and feasibility to use selected indicators as local improvement targets.

Costs

The average cost of public involvement was $9,427 per site. Patient involvement increased the cost of the prioritization process by 17% compared with priority-setting by professionals alone. Most of patient involvement costs were incurred by compensation of participants’ time, meal, and travel expenses (34%), coordination of patient recruitment (29%), and hiring of a professional facilitator (15%). Time to reach agreement on common priorities was, on average, 10% longer in intervention sites.

Discussion

To the best of our knowledge, this is the first trial of patient involvement in collective healthcare improvement decisions at the population level [19, 20, 22, 23, 25]. This study shows that patient involvement can change priorities driving healthcare improvement at the community level. The involvement intervention fostered mutual influence and increased agreement between patients and professionals. We found no evidence that patient involvement adversely affected professionals’ intention to use the selected quality indicators for improvement. However, patient involvement required more time and dedicated resources than priority-setting by professionals alone.

The observed impact of patient involvement on priority-setting is clinically significant. Patient involvement shifted healthcare improvement priorities toward core generic components of the Patient-Centered Medical Home and Chronic Care Model, including timely access to a regular primary care provider, self-care support, patient participation in clinical decision-making, and partnership with community organizations [42, 43]. Patients’ ability to move priorities toward access to primary care is meaningful from a North American perspective, as recent international studies rank Canada and the US last among developed countries on this quality dimension, a problem with significant documented impacts on population health [44, 45].

Policy implications

This study is important as it is the first trial to document the impact of patient involvement on strategic healthcare decisions affecting the population [19, 2123]. Such experimental study is unique and makes a significant contribution to the field. While observational studies demonstrated that deliberation influences patients’ opinions, this is the first trial evidence showing that patients can influence professionals and have an actual impact on collective decision-making [29]. Our ability to involve vulnerable patients from disadvantaged socio-economic groups in a real-world priority-setting exercise is also important, and supports the feasibility of effectively involving a broad range of stakeholders in complex policy decisions [31].

These findings have practical implications for the development of effective patient involvement interventions in healthcare. This multi-faceted patient involvement intervention required a structured recruitment strategy, a full-day preparation meeting, consultation of a relatively large number of patients, small-group face-to-face deliberation between professionals and patients, financial compensation of participants, and moderation by an expert facilitator. Setting priorities with patients and professionals required more time and dedicated resources than with professionals alone. Existing patient involvement programs often lack some of these components, which may explain variations of observed effects in practice [21]. For example, it is common for one or two patient representatives to sit on advisory committees chaired by professionals, without structured patient preparation. Results of this trial points toward some ‘key ingredients’ that could support more effective patient involvement in collective healthcare improvement and policy decisions, and those factors have been described in details in a related publication [46].

Study limitations and future research

The study sample size was small, and cluster randomization further reduces statistical power. However, the trial proved sufficient powered to test the impact of patient involvement on priority-setting, because the observed effect size was large. The small sample size may nonetheless have affected our ability to detect differences in professionals’ intention to use quality indicators.

While this study provides strong evidence that patient involvement can change improvement priorities, we cannot conclude that such differences will translate into changes in healthcare delivery. An important area for future research would therefore be to test, over a longer time period, whether patient involvement can transform healthcare services and have an impact on patient-oriented outcomes.

Also, while patient involvement had an important effect in this study, it is unknown if these results can be generalized to other contexts. This involvement intervention benefited from high-level policy support and was implemented within relatively small communities, which may have contributed to its effectiveness [47]. Caution is therefore required before extrapolating the effects observed in this study to other involvement programs and contexts.

Conclusions

In conclusion, this study shows that patient involvement can change priorities driving healthcare improvement at the community level. Effective patient involvement requires time and dedicated resources. Future research should assess the generalizability of these findings to other contexts, and the impact of involvement interventions on patient care.

Declarations

Acknowledgements

This study was funded by an operating grant from the Canadian Health Services Research Foundation and Agence de la Santé et des Services Sociaux de l’Abitibi-Témiscamingue (CHS-2160). AB is supported by a Clinician-Scientist Award from the Canadian Institutes of Health Research. PL holds a Canada Research Chair on Health Innovations.

Celine Hubert, Sylvie Bellot, Suzanne Chartier, and Mélanie Gauthier (Agence de la Santé et des Services Sociaux de l’Abitibi-Témiscamingue) worked as coordinators and research professionals on the project, along with Jolyne Lalonde (Dery and Associés) who acted as our lead moderator. François-Pierre Gauvin, Victoria Thomas, Janet Wale, Richard Osborne, Janusz Kaczorwski, Daniel Marleau, and Julia Sohi offered useful comments and reviews of the manuscript. Anais Lacasse (UQAT) contributed to questionnaire development and the statistical analysis plan. Reinier Akkermans (IQ Healthcare) and Djamal Berbiche (Charles-Lemoyne Hospital Research Center) provided statistical advice and conducted the multi-level analysis. We would like to thank all participants in this study.

Authors’ Affiliations

(1)
Charles-Lemoyne Research Center, Department of Family and Emergency Medicine, Université de Sherbrooke
(2)
Scientific Institute for Quality of Healthcare, Radboud University Nijmegen
(3)
Agence de la santé et des services sociaux de l’Abitibi-Témiscamingue
(4)
Department of Health Administration, IRSPUM, Université de Montréal
(5)
Dutch College of General Practitioners

References

  1. Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A: Improving Chronic Illness Care: Translating Evidence Into Action. Health Aff. 2001, 20 (6): 64-78. 10.1377/hlthaff.20.6.64.View ArticleGoogle Scholar
  2. Barr VJ, Robinson S, Marin-Link B, Underhill L, Dotts A, Ravensdale D, Salivaras S: The expanded Chronic Care Model: an integration of concepts and strategies from population health promotion and the Chronic Care Model. Hosp Q. 2003, 7 (1): 73-82.PubMedGoogle Scholar
  3. Sinaiko AD, Rosenthal MB: Patients' role in accountable care organizations. N Engl J Med. 2010, 363 (27): 2583-2585. 10.1056/NEJMp1011927.View ArticlePubMedGoogle Scholar
  4. VanLare JM, Conway PH, Sox HC: Five next steps for a new national program for comparative-effectiveness research. N Engl J Med. 2010, 362 (11): 970-973. 10.1056/NEJMp1000096.View ArticlePubMedGoogle Scholar
  5. Fleurence R, Selby JV, Odom-Walker K, Hunt G, Meltzer D, Slutsky JR, Yancy C: How the patient-centered outcomes research institute is engaging patients and others in shaping its research agenda. Health Aff (Millwood). 2013, 32 (2): 393-400. 10.1377/hlthaff.2012.1176.View ArticleGoogle Scholar
  6. Developing clinical and health improvement indicators for the Quality and Outcomes Framework (QOF): Interim process guide. [http://www.nice.org.uk]
  7. Boivin A, Currie K, Fervers B, Gracia J, James M, Marshall C, Sakala C, Sanger S, Strid J, Thomas V: Patient and public involvement in clinical guidelines: international experiences and future perspectives. Qual Saf Health Care. 2010, 19 (5): e22-PubMedGoogle Scholar
  8. Chassin MR, Loeb JM, Schmaltz SP, Wachter RM: Accountability measures–using measurement to promote quality improvement. N Engl J Med. 2010, 363 (7): 683-688. 10.1056/NEJMsb1002320.View ArticlePubMedGoogle Scholar
  9. Roseman D, Osborne-Stafsnes J, Amy CH, Boslaugh S, Slate-Miller K: Early lessons from four 'aligning forces for quality' communities bolster the case for patient-centered care. Health Aff (Millwood). 2013, 32 (2): 232-241. 10.1377/hlthaff.2012.1085.View ArticleGoogle Scholar
  10. Scholle SH, Torda P, Peikes D, Han E, Genevro J: Engaging patients and families in the medical home. Agency for Healthcare Research and Quality, US Department of Health and Human Services. 2010Google Scholar
  11. Berwick DM: Launching accountable care organizations–the proposed rule for the Medicare Shared Savings Program. N Engl J Med. 2011, 364 (16): e32-10.1056/NEJMp1103602.View ArticlePubMedGoogle Scholar
  12. Stacey D, Bennett CL, Barry MJ, Col NF, Eden KB, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Legare F, Thomson R: Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2011, 10: CD001431Google Scholar
  13. McAlister FA, Man-Son-Hing M, Straus SE, Ghali WA, Anderson D, Majumdar SR, Gibson P, Cox JL, Fradette M: Impact of a patient decision aid on care among patients with nonvalvular atrial fibrillation: a cluster randomized trial. CMAJ. 2005, 173 (5): 496-501. 10.1503/cmaj.050091.View ArticlePubMedPubMed CentralGoogle Scholar
  14. Grilli R, Magrini N, Penna A, Mura G, Liberati A: Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000, 355 (9198): 103-106. 10.1016/S0140-6736(99)02171-6.View ArticlePubMedGoogle Scholar
  15. Haynes RB, Yao X, Degani A, Kripalani S, Garg A, McDonald HP: Interventions to enhance medication adherence. Cochrane Database Syst Rev. 2005, 4: CD000011Google Scholar
  16. Heneghan C, Ward A, Perera R, Bankhead C, Fuller A, Stevens R, Bradford K, Tyndel S, Alonso-Coello P, Ansell J: Self-monitoring of oral anticoagulation: systematic review and meta-analysis of individual patient data. Lancet. 2012, 379 (9813): 322-334. 10.1016/S0140-6736(11)61294-4.View ArticlePubMedGoogle Scholar
  17. Murray E, Burns J, See TS, Lai R, Nazareth I: Interactive Health Communication Applications for people with chronic disease. Cochrane Database Syst Rev. 2005, 4: CD004274Google Scholar
  18. Warsi A, Wang PS, LaValley MP, Avorn J, Solomon DH: Self-management education programs in chronic disease: a systematic review and methodological critique of the literature. Arch Intern Med. 2004, 164 (15): 1641-1649. 10.1001/archinte.164.15.1641.View ArticlePubMedGoogle Scholar
  19. Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD: Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev. 2006, 3
  20. Conklin A, Morris ZS, Nolte E: Involving the public in healthcare policy. 2010, Cambridge: RAND Europe, 1-83.Google Scholar
  21. Legare F, Boivin A, van der Weijden T, Pakenham C, Burgers J, Legare J, St-Jacques S, Gagnon S: Patient and Public Involvement in Clinical Practice Guidelines: A Knowledge Synthesis of Existing Programs. Med Decis Making. 2011, 31 (6): E45-74. 10.1177/0272989X11424401.View ArticlePubMedGoogle Scholar
  22. Crawford MJ, Rutter D, Manley C, Weaver T, Bhui K, Fulop N, Tyrer P: Systematic review of involving patients in the planning and development of health care. BMJ. 2002, 325 (7375): 1263-10.1136/bmj.325.7375.1263.View ArticlePubMedPubMed CentralGoogle Scholar
  23. Mitton C, Smith N, Peacock S, Evoy B, Abelson J: Public participation in health care priority setting: A scoping review. Health Policy. 2009, 91 (3): 219-228. 10.1016/j.healthpol.2009.01.005.View ArticlePubMedGoogle Scholar
  24. Abelson J, Forest PG, Eyles J, Smith P, Martin E, Gauvin FP: Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Soc Sci Med. 2003, 57 (2): 239-251. 10.1016/S0277-9536(02)00343-X.View ArticlePubMedGoogle Scholar
  25. Coulter A, Ellins J: Patient-focused interventions: A review of the evidence. 2006, United Kingdom: Picker Institute Europe, 277-Google Scholar
  26. Kotter T, Schaefer FA, Scherer M, Blozik E: Involving patients in quality indicator development - a systematic review. Patient Preference and Adherence. 2013, 7: 259-268.PubMedPubMed CentralGoogle Scholar
  27. Chambers S: Deliberative democratic theory. Ann Rev Pol Sci. 2003, 6 (1): 307-326. 10.1146/annurev.polisci.6.121901.085538.View ArticleGoogle Scholar
  28. Davies C, Wetherell M, Barnett E: Citizens at the Centre: Deliberative Participation in Healthcare Decisions. 2006, Bristol: Policy PressGoogle Scholar
  29. Abelson J, Montessanti S, Li K, Gauvin F-P, Martin E: Effective strategies for interactive public engagement in the development of healthcare policies and programs. 2010, Ottawa: Canadian Health Services Research Foundation, 52-Google Scholar
  30. Florin D, Dixon J: Public involvement in health care. BMJ. 2004, 328 (7432): 159-161. 10.1136/bmj.328.7432.159.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Church J, Saunders D, Wanke M, Pong R, Spooner C, Dorgan M: Citizen participation in health decision-making: past experience and future prospects. J Public Health Policy. 2002, 23 (1): 12-32. 10.2307/3343116.View ArticlePubMedGoogle Scholar
  32. Health and Social Services. [http://www.msss.gouv.qc.ca/en/reseau/services.php]
  33. Boivin A, Lehoux P, Lacombe R, Lacasse A, Burgers J, Grol R: Target for Improvement: A cluster randomized trial of public involvement in quality Indicator prioritization (intervention development and study protocol). Implement Sci. 2011, 6: 45-10.1186/1748-5908-6-45.View ArticlePubMedPubMed CentralGoogle Scholar
  34. Green J, Thorogood N: Qualitative methods for health research. 2004, London: SAGEGoogle Scholar
  35. Friedman L, Furberg C, DeMets D: Fundamentals of Clinical Trials (Third Edition). 2006, New York: SpringerGoogle Scholar
  36. Research Randomizer v. 4.0. [http://www.randomizer.org]
  37. Rowe G, Frewer LJ: A Typology of Public Engagement Mechanisms. Sci Technol Humun Values. 2005, 30 (2): 251-10.1177/0162243904271724.View ArticleGoogle Scholar
  38. Campbell SM, Braspenning J, Hutchinson A, Marshall MN: Research methods used in developing and applying quality indicators in primary care. BMJ. 2003, 326 (7393): 816-819. 10.1136/bmj.326.7393.816.View ArticlePubMedPubMed CentralGoogle Scholar
  39. Significance of the Difference Between Two Correlation Coefficients. [http://faculty.vassar.edu/lowry/rdiff.html]
  40. Donner A, Klar N: Design and analysis of cluster randomization trials in health research. 2000, London: ArnoldGoogle Scholar
  41. Burton P, Gurrin L, Sly P: Extending the simple linear regression model to account for correlated responses: an introduction to generalized estimating equations and multi-level mixed modelling. Stat Med. 1998, 17 (11): 1261-1291.View ArticlePubMedGoogle Scholar
  42. Bodenheimer T, Wagner EH, Grumbach K: Improving Primary Care for Patients With Chronic Illness. JAMA. 2002, 288 (14): 1775-1779. 10.1001/jama.288.14.1775.View ArticlePubMedGoogle Scholar
  43. Starfield B, Shi L: The Medical Home, Access to Care, and Insurance: A Review of Evidence. Pediatrics. 2004, 113 (5): 1493-1498.PubMedGoogle Scholar
  44. Schoen C, Osborn R, How SKH, Doty MM, Peugh J: In Chronic Condition: Experiences Of Patients With Complex Health Care Needs, In Eight Countries, 2008. Health Aff. 2009, 28 (1): w1-16. 10.1377/hlthaff.28.1.w1.View ArticleGoogle Scholar
  45. Starfield B, Shi L: Policy relevant determinants of health: an international perspective. Health Policy. 2002, 60 (3): 201-218. 10.1016/S0168-8510(01)00208-1.View ArticlePubMedGoogle Scholar
  46. Boivin A, Burgers J, Grol R, Lehoux P: What are the key ingredients for effective public involvement in healthcare improvement and policy decisions? A randomized trial process evaluation. Milbank Quaterly. Accepted for publication
  47. Abelson J, Forest PG, Eyles J, Casebeer A, Martin E, Mackean G: Examining the role of context in the implementation of a deliberative public participation experiment: Results from a Canadian comparative study. Soc Sci Med. 2007, 64: 2115-2128. 10.1016/j.socscimed.2007.01.013.View ArticlePubMedGoogle Scholar

Copyright

© Boivin et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.