Skip to main content
Download PDF
Download ePub
  • Systematic review
  • Open access
  • Published:

Computerized clinical decision support systems for primary preventive care: A decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes

Implementation Science volume 6, Article number: 87 (2011) Cite this article

Abstract

Background

Computerized clinical decision support systems (CCDSSs) are claimed to improve processes and outcomes of primary preventive care (PPC), but their effects, safety, and acceptance must be confirmed. We updated our previous systematic reviews of CCDSSs and integrated a knowledge translation approach in the process. The objective was to review randomized controlled trials (RCTs) assessing the effects of CCDSSs for PPC on process of care, patient outcomes, harms, and costs.

Methods

We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews Database, Inspec, and other databases, as well as reference lists through January 2010. We contacted authors to confirm data or provide additional information. We included RCTs that assessed the effect of a CCDSS for PPC on process of care and patient outcomes compared to care provided without a CCDSS. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive.

Results

We added 17 new RCTs to our 2005 review for a total of 41 studies. RCT quality improved over time. CCDSSs improved process of care in 25 of 40 (63%) RCTs. Cumulative scientifically strong evidence supports the effectiveness of CCDSSs for screening and management of dyslipidaemia in primary care. There is mixed evidence for effectiveness in screening for cancer and mental health conditions, multiple preventive care activities, vaccination, and other preventive care interventions. Fourteen (34%) trials assessed patient outcomes, and four (29%) reported improvements with the CCDSS. Most trials were not powered to evaluate patient-important outcomes. CCDSS costs and adverse events were reported in only six (15%) and two (5%) trials, respectively. Information on study duration was often missing, limiting our ability to assess sustainability of CCDSS effects.

Conclusions

Evidence supports the effectiveness of CCDSSs for screening and treatment of dyslipidaemia in primary care with less consistent evidence for CCDSSs used in screening for cancer and mental health-related conditions, vaccinations, and other preventive care. CCDSS effects on patient outcomes, safety, costs of care, and provider satisfaction remain poorly supported.

Peer Review reports

Background

Achieving comprehensive and effective primary preventive care (PPC) remains a challenge for healthcare systems worldwide. Despite the existence of clinical guidelines, many preventive care interventions are still underused, for example, the low influenza vaccine rates among children and adolescents with increased-risk conditions [1] and the limited use of prophylaxis against deep vein thrombosis [2].

Interventions to overcome this problem may affect healthcare governance, financial, and delivery arrangements, and may include use of health information technologies such as electronic health records and computerized clinical decision support systems (CCDSSs). CCDSSs have been promoted in many high-income countries as a promising tool for improving PPC [3]. The USA and other nations have accelerated their implementation as part of stimulus packages issued in 2009 [4, 5].

We define CCDSSs for PPC as computerized matching of an individual patient's characteristics with a knowledge base that then provides patient-specific recommendations to healthcare providers about PPC. Despite their promise and expense, definitive evidence of CCDSS effectiveness for process of care (e.g., performance and satisfaction of healthcare providers), patient outcomes (e.g., functional status, disability, major clinical events, quality of life, and death), costs, and safety remain to be established [6–8].

Our previous review showed inconsistent evidence of improvement in providers' adherence to PPC procedures such as screening for breast, cervical, and prostate cancers, and very weak evidence on improvement of patient outcomes [6]. Another review found modest effectiveness for CCDSSs that prompt clinicians for smoking cessation interventions (average increase in delivery of preventive care measure: 23%), cardiac care (average increase: 20%), blood pressure screening (average increase: 16%), vaccinations, diabetes management, and cholesterol (average increase for each measure: 15%), and mammographic screening (average increase: 10%), but only eight (13%) of the included studies tested fully computerized reminders [9]. Jacobson and Szilagyi showed that patient reminder and recall systems in primary care settings are effective in improving immunization rates in developed countries [10]. However, effects of CCDSSs on patient outcomes, costs, and safety have yet to be shown [11, 12].

Many new studies have been published recently, and many health care institutions and clinical practices are considering implementation of this new information technology. We conducted a systematic review of randomized controlled trials (RCTs) assessing the effectiveness of CCDSSs for PPC on process of care, patient outcomes, costs, safety, and provider satisfaction with CCDSS for PPC in partnership with clinical decision makers.

Methods

The detailed methods for this systematic review have been published elsewhere [13] and are available through open access http://www.implementationscience.com/content/5/1/12.

Research questions

This systematic review addressed two questions: Do CCDSSs improve process of care or patient outcomes for PPC, and what are the costs, safety, and provider satisfaction with CCDSS for PPC?

Partnering with decision makers

The review team included a partnership between McMaster University's Health Information Research Unit (HIRU), the senior administration of Hamilton Health Sciences (a large Canadian academic health sciences centre) and Local Health Integration Network (the regional health authority that includes Hamilton), and clinical service chiefs at local hospitals. Decision-maker partners were included in discussions about data extraction for, and interpretation of, factors that might affect implementation. The decision-maker-researcher partnership hypothesized positive effects of CCDSSs in both process of care and patient outcomes regarding PPC, methodological improvement in testing of CCDSSs over time, cost savings, and improved safety and provider satisfaction with CCDSS use.

Search strategy

We previously described our search methods up to 2004 [6] and for this update [13]. Briefly, for the latest update we used a comprehensive search strategy to retrieve potentially relevant RCTs from MEDLINE, EMBASE, Ovid's Evidence-Based Medicine Reviews, and the Inspec bibliographic database from 1 January 2004 to 8 December 2008; a further update was conducted to 6 January 2010. We performed duplicate screening of eligible RCTs and independent data-extraction using piloted forms that were constructed with our decision-maker partners; a third reviewer resolved disagreements. Inter-reviewer agreement on study eligibility was measured using the unweighted Cohen's kappa (κ), and was excellent (κ = 0.93; 95% confidence interval [CI], 0.91 to 0.94) over all applications. Study authors confirmed extracted data for 88% (36/41) of the studies included in the PPC review.

Study selection

We included RCTs (including cluster RCTs) published in any language that compared the effects of care with a CCDSS for PPC, used by healthcare providers, with care without a CCDSS. Outcomes included processes of care and patient outcomes. We only considered RCTs because this method minimizes the risk of biased allocation, and there has been increased publication of RCTs since our 2005 review [6].

For PPC interventions, patients had to be free from the illness to be prevented (e.g., a specific strain of influenza) but could be seen in any setting, including acute healthcare. CCDSSs that provided only computer-aided instruction, performed actions unrelated to clinical decision making (e.g., CCDSSs for diagnostic performance against a gold standard), or evaluated CCDSS users' knowledge or performance in clinical simulations were excluded.

We excluded studies where PPC interventions were merged with a complex set of other interventions (e.g., chronic disease management) and those that did not focus on PPC (e.g., screening of medical errors). We did however include one study that evaluated a CCDSS for influenza vaccination in asthmatic patients because it provided evidence about the independent effects of the intervention on vaccination rates [1].

Data extraction

Independent reviewers extracted key data in duplicate, including study methods, CCDSS and population characteristics, possible sources of bias, and outcomes. Primary authors of each study were asked to review the extracted data for their study and offer comments on the extracted data.

Assessment of study quality

Details of our quality assessment of included RCTs are published elsewhere [13]. RCTs were scored for methodological quality on a 10-point scale (an extension of the Jadad scale [14]) with scores ranging from 0 for the lowest study quality to 10 for the highest quality.

Assessment of CCDSS intervention effects

Researchers and decision-makers selected outcomes that were relevant to PPC from each study before evaluating intervention effects. We used RCTs as the unit of analysis to assess CCDSS effectiveness. A process of care outcome represents the delivered quality of care, while a patient outcome represents the directly measured health status of the patient. We used a dichotomous measure of effect and defined a CCDSS as effective (positive) when there was a significant (p< 0.05) improvement in the endpoint specified as main or primary by the authors or, if no primary endpoint was specified, the endpoint used to estimate study power, or, failing that, ≥50% of multiple pre-specified endpoints. When no clear pre-specified endpoints existed, we considered a CCDSS effective if it improved ≥50% of all reported outcomes. Studies that included ≥1 CCDSS treatment arm were considered effective if any of the treatment CCDSS arms was evaluated as effective. These criteria are more specific than in our 2005 review [6], and the effect assignment was adjusted for some studies from that review.

Data synthesis and analysis

We used descriptive summary measures for data including proportions for categorical variables and means (± standard deviations) for continuous variables. When reporting results from individual studies, we cited the measures of association and p-values as reported in the studies. We considered methodological rigor and scientific quality of the included trials to analyze data and formulate conclusions. We did not pool data or compare studies using effect sizes because of study heterogeneity in populations, settings, interventions, and outcomes. A sensitivity analysis was conducted to assess the possibility of biased results in studies with a mismatch between the unit of allocation (e.g., clinicians) and the unit of analysis (e.g., individual patients without adjustment for clustering). Success rates comparing studies with matched and mismatched analyses were compared using chi-square for comparisons. No differences in reported success were found for either process of care outcomes (Fisher's exact test, 2P = 1.0) or patient outcomes (Fisher's exact test, 2P = 1.0). Accordingly, results have been reported without distinction for mismatch.

Results

We included 46 publications describing 41 trials (Figure 1) [1, 15–59]. We excluded five of the 24 studies included in our previous review [6] because they did not meet our new, stricter inclusion criteria [60–63] or were more relevant for another application [64]. Additionally, we excluded 14 RCTs because reminders were part of a more complex intervention for chronic disease including diabetes [65–69], hypertension [70, 71], heart failure and/or ischemic heart disease [72], asthma or chronic obstructive pulmonary disease [73], or the CCDSS screened for medical errors [74, 75] including those caused by drug-drug interaction and adverse drug effects [76], reported on advanced clinical directives [77], or compared two CCDSSs [78]. Twelve included studies contribute outcomes to this review as well as other CCDSS applications in the series; two studies [27, 28] to four reviews, five studies [18, 19, 29, 31, 42, 59] to three reviews, and five studies [1, 43–45, 47, 50, 56] to two reviews; but we focused here on PPC-relevant outcomes.

Figure 1
figure 1

Flow diagram of included and excluded studies for the update 1 January 2004 to 6 January 2010 with specifics for primary preventive care*. *Details provided in: Haynes RB et al. [13]. Two updating searches were performed, for 2004 to 2009 and to 6 January 2010 and the results of the search process are consolidated here.

Full size image

Summary outcome data are reported in Table 1. The methodological quality of included studies is summarized in Additional file 1 Table S1; CCDSS characteristics in Additional file 2 Table S2; study characteristics in Additional file 3 Table S3; detailed outcome data in Additional file 4 Table S4; and other CCDSS process-related outcomes in Additional file 5 Table S5.

Table 1 Summary of results of CCDSS trials of primary preventive care
Full size table

Study quality

Additional file 1 Table S1 shows an overall increase of methodological quality of RCTs over time, although this could be due, in part, to improved reporting. Eighteen of 41 (44%) studies [1, 15, 18–22, 24, 27, 29, 30, 35, 36, 42, 48, 49, 54–56, 59] scored at least 8 of 10 points (i.e., high quality) including six trials with perfect scores [27, 29, 30, 35, 42, 56]. The main methodological limitations in low-score trials were lack of allocation concealment and cluster randomization, and incomplete follow-up. The correlation of study methodological quality with CCDSSs effects on process of care was non-significant (Pearson 0.142, 95% CI -0.18 to 0.43). The same analysis could not be undertaken for patient outcomes due to the small number of studies that evaluated patients outcomes (n = 14) and that showed a positive effect (n = 4).

CCDSS and study characteristics

Additional file 2 Table S2 shows that 20/41 (49%) CCDSSs were integrated with an electronic medical record [1, 17, 25, 27, 29, 31, 32, 34–36, 39, 41–46, 49, 50, 52, 56, 59] including at least five also integrated with a computerized order entry system [1, 32, 42, 49, 56] and 21/41 (51%) were stand-alone computer systems [15, 16, 18–22, 24, 26, 28, 30, 33, 37, 38, 40, 47, 48, 51, 53–55, 57, 58]. The data entry method varied across systems, with a non-practitioner decision-maker entering data on 29/39 (74%) studies [1, 15, 17, 21, 23–25, 27, 29, 31, 32, 34–55, 59] and automatic entry through electronic health records in 15/39 (38%) cases [1, 17, 27, 29, 31, 34–36, 41, 42, 46, 49, 50, 56, 59]. In all but one study [26], physicians used all PPC CCDSSs, either solely or shared with other healthcare providers including trainees [1, 25, 28, 29, 39, 41, 42, 46–48, 52], advanced practice nurses [1, 17–19, 30, 50, 59], physician assistants [18, 19, 33], and social workers [26]. No single study completely described the CCDSSs interface. Delivery methods for CCDSS recommendations varied: 17/40 studies (43%) reported use of a desktop or laptop computer [1, 26–32, 34–36, 42, 49, 50, 56–59]; 10/40 (25%) used existing non-prescribing staff [17, 28, 33, 40, 41, 43–46, 52, 53, 59]; 8/40 (20%) used research project staff [15, 20–22, 24, 38, 39, 47]; and the remaining studies used other methods, including personal digital assistants [18, 19] and paper reports [50]. CCDSSs were pilot tested in 15/33 studies (45%), providers received training on the CCDSS in 23/35 trials (66%), and the CCDSS provided suggestions at the time of care in 36/41 studies (88%). Investigators also developed the CCDSS in 28/35 studies (80%).

Twenty-nine of 41 trials (71%) were conducted in the USA [1, 16–26, 28, 29, 32, 33, 36, 39–45, 47–50, 52–55, 59], 5/41 (12%) in the UK [30, 34, 37, 51, 57, 58], 3/41 (7%) in Canada [15, 38, 46], and 1/41 (2%) each in Italy [31], New Zealand [35], Spain [27], and The Netherlands [56]. Forty-four percent (18/41) of trials were published after the year 2001 including 14/41 (34%) published after the year 2005. Eighty percent (33/41) of trials reported a public funding source [1, 15–24, 28–30, 33–35, 37, 39–47, 49–59], 7% (3/41) a private source [27, 36, 48], 2% (1/41) both public and private [38], and 10% (4/41) did not report these data [25, 26, 31, 32]. Twenty-two trials (54%) took place mainly in primary care settings [1, 18–20, 22, 23, 27, 30, 31, 33–38, 40, 49–51, 53–58] while 19 trials (46%) were undertaken in a combination of hospitals, specialist clinics, and primary care, or in academic centres [15–17, 21, 22, 24–26, 28, 29, 32, 36, 39, 41–48, 52, 59]. In all but one [1] of the 41 trials, the patients were adults or elderly.

Many CCDSS interventions for PPC were tested in the included studies. Twenty-two (54%) studies evaluated multifaceted interventions with ≥3 preventive care components [15, 18–23, 28, 30, 34, 35, 37, 39–41, 46, 47, 49–51, 53–55, 57, 58], including educational sessions on preventive interventions and the CCDSS, supply of materials to clinicians and/or to patients, assessments of patient and clinician attitude towards health conditions and/or the CCDSS, audit and feedback of clinician performance, academic detailing, telephone reminder to patients, elimination of out-of-pocket expenses to patients, and use of local clinician leaders. Eleven (27%) trials assessed two components [1, 16, 24, 27, 31, 33, 36, 38, 42, 48, 52], and the remaining eight (21%) assessed the effectiveness of a CCDSS with one component, typically a reminder (e.g., printed, audio, or visual) [17, 25, 26, 29, 32, 43–45, 56, 59].

CCDSSs effectiveness

Table 1 (see Additional file 4 Table S4 for detailed information) shows that all trials assessed the effects of CCDSSs on processes of care. Twenty-five of 40 (61%) studies showed an improved process of care using our dichotomous measure; three of those studies also included CCDSS treatment arms that did not improve process of care [26, 41, 56]. Four of 14 (29%) studies showed improved patient outcomes. Only 13 (32%) studies reported both process of care and patient outcomes.

Cancer screening (10 trials)

CCDSSs improved the screening or referral of patients with breast, cervical, ovarian, colorectal, and prostate cancers in 5/10 (50%) trials [20, 22, 30, 39, 40]. Emery et al.[30] showed improved rate of appropriate referrals to regional genetics clinics by primary care clinicians regarding familial cancers (i.e., breast, ovarian, and colorectal cancers). Conversely, Burack et al. demonstrated no effects for reminders for mammography screening [21] and screening mammography and pap smears tests in primary care [24]. Only three studies assessed patient outcomes, and none demonstrated effects [30, 49, 57, 58].

Multiple preventive care activities (10 trials)

In rural and urban primary care settings and hospitals, clinicians received CCDSS recommendations for various interventions in adult and geriatric patients including cancer screening, cardiovascular (CV) risk assessment, vaccination, tuberculosis skin tests, counselling, patient education, prophylactic antacids, calcium supplements, and screening for functional independency. Six (60%) trials reported improved process of care [28, 29, 33, 41, 46, 52] including one trial demonstrating higher ordering rates for pneumococcal vaccination (35.8% of patients in the intervention group versus 0.8% of those in the control group, p<0.001), influenza vaccination (51.4% versus 1.0%, p<0.001), prophylactic heparin (32.2% versus 18.9%, p<0.001), and prophylactic aspirin at discharge (36.4% versus 27.6%, p<0.001) in a teaching USA hospital [29]. Conversely, Overhage et al.[42] showed that a CCDSS for 22 preventive care measures in hospital inpatients did not change clinicians' actions for such measures. Only two studies assessed patient outcomes, but neither showed effects [16, 34].

Screening and management of CV risk factors (9 trials)

CCDSSs helped clinicians detect and treat dyslipidaemia, diabetes, smoking, obesity, hypertension, and renal diseases as well as calculating coronary risk factor profiles. All nine trials reported improved process of care of which three targeted screening and treatment of dyslipidaemia in primary care [18, 19, 27, 56]. Five trials reported patient outcomes; three showed positive effects [17, 38, 43–45], and two [27, 54, 55] showed no effects.

Screening and management of mental health-related conditions (6 trials)

Studies in this category covered various CCDSSs for screening and management of mental health conditions in primary, secondary, and tertiary care settings. Only one trial [47] used cluster randomization (see Additional file 1 Table S1) and all but one trial [51] were conducted in a single site. In all six trials, the CCDSSs were stand-alone systems (see Additional file 2 Table S2), and four trials included patient-completed computer-based instruments [15, 37, 48] or paper-based post intervention surveys [47]. Two trials showed positive effects in process of care, including Ahmad et al. [15] who reported that a CCDSS increased opportunities to discuss intimate partner violence in primary care (adjusted relative risk [RR], 1.4; 95% CI, 1.1 to 1.9) and increased its detection (adjusted RR, 2.0; 95% CI, 0.9 to 4.1). Three trials reported on patient outcomes including one with positive [51] and two with no effects [37, 47].

Vaccination (3 trials)

CCDSSs for tetanus, hepatitis, pneumococcal, measles, and influenza vaccinations in children, adults, and the elderly in primary care only improved influenza vaccination among the elderly in one trial [25]. All trials compared 'usual care' with CCDSS alone [25, 32] or in addition to an educational session [1], and no trials assessed patient outcomes.

Other preventive care activities (3 trials)

Two trials reported improved process of care [36], including one that also assessed patient outcomes but found no effects [59]. All studies in this category compared CCDSSs with 'usual care' although in one study, all providers were educated on the importance of HIV screening and trained on CCDSS functions [50]. Lafata et al.[36] showed that, among insured women 65 to 89 years of age, reminders mailed to patients, either alone or with physician prompts, improved osteoporosis screening and treatment rates. Zanetti et al.[59] showed improved intraoperative redose of prophylactic antibiotic, but it was underpowered to demonstrate effects on patient outcomes.

Costs and practical process related outcomes (see Additional file 5, Table S5)

Costs of developing, implementing, and maintaining a CCDSS were partly reported in 6/41 (15%) trials [16, 27, 33, 46, 54, 55, 57, 58]. Among these six studies, when a CCDSS was used, two found costs of care were significantly less [27, 46], three yielded increased cost of care [16, 33, 54, 55], and one showed varied cost minimization data [57, 58]. Rosser et al.[46] did not report detailed costs, although the physician reminder was reported to be the most cost-effective method of improving preventive services, followed by letter reminder, and telephone reminder. Cobos et al.[27] showed that a CCDSS for management of patients with dyslipidaemia including those without coronary heart disease had no effects on lipid profiles, but saved 24.9% in treatment cost per patient and 20.8% in total costs, including costs for physician visits, laboratory analyses, and lipid-lowering drugs. Apkon et al.[16] showed a difference of US $91 more patient resource usage (ambulatory visits, laboratory tests, diagnostic imaging, and pharmacy use) for multiple preventive care procedures in the CCDSS group than usual-care group. Frame et al. showed that a CCDSS for multiple preventive procedures did not increase revenue generation or the number of office visits to a fee-for-service clinic despite its positive effects on provider compliance to such activities [33], and Unrod et al. found implementation costs for CCDSS, including equipment, training, and staff costs, increased costs for smoking cessation counselling [54, 55]. Wilson et al. presented software development costs and the marginal cost for each additional compact disc [57, 58].

Only two (5%) trials reported CCDSS adverse events; one demonstrated greater risk for over treatment than for under treatment in dyslipidaemia because all patients were screened, including low-risk patients who would not normally be screened [18, 19]. Zanetti et al.[59] reported four in 449 (1%) inappropriate alerts to redose prophylactic antibiotics during cardiac surgery and one unnecessary intraoperative redosing [59].

Six (15%) trials reported on provider satisfaction with CCDSSs [15, 16, 30, 40, 49, 50] including two trials [30, 49] on cancer screening where most providers were satisfied with CCDSSs use. Only Apkon et al.[16] reported provider and patient satisfaction when a CCDSS was used, but showed no significant differences between groups in patient satisfaction results and mixed providers' satisfaction within the group using a CCDSS for 12 preventive care interventions.

Discussion

We added 17 new trials to our previous review [6], and synthesised the evidence from 41 RCTs of CCDSSs for PPC. Forty of 41 trials examined process of care for which the majority of CCDSSs, 25/40 (63%), were effective using a dichotomous measure of effect. Recent trials more often reported patient outcomes (14/41 (34%) versus 1/24 (4%) study in our 2005 review [6]), but these outcomes were mostly surrogates (e.g., cholesterol level) rather than major patient outcomes.

For CCDSSs showing positive effect, it is important to be cautious about ascribing positive effects solely to CCDSSs [79] because most interventions included multiple components, such as educational sessions for clinicians and outreach to patients, and all trials were unblinded. For CCDSSs showing no effect, control-group clinicians often received training on the condition and recommended care. These 'educated' participants may have diluted intervention effects. Moreover, the reality of clinical practice, such as patients' varied adherence to recommendations, deficient follow-through by healthcare services, and long waiting lists for preventive care procedures [34], may have reduced intervention effects. In short, interventions directed at provider behaviors are bound to have limited effect on actions that also require patient adherence and service support to realize such actions.

Our review found that CCDSSs for PPC rarely reported cost-effectiveness and harm assessments. Within the 6/41 (15%) RCTs reporting costs, the majority only performed cost comparisons of interventions, not cost-effectiveness analysis [80]. Reporting was often incomplete, focused mainly on the CCDSS operating expenses, and varied substantially in methods of calculating costs and items included in analyses. There also was limited reporting on CCDSS-caused harm. The paucity of cost-effectiveness and harm analyses in PPC-related CCDSS studies is consistent with the current literature [9, 81].

Findings in this review may not be generalised to low- and middle-income countries because all included trials were conducted in high-income countries, the CCDSS costs and context-related data were incompletely reported, and many CCDSSs were integrated with electronic health records. These factors may hinder implementation or scaling up of CCDSSs in resource scarce settings, and it remains unclear if and how such settings might achieve similar benefits and at what costs. Moreover, patients' and organizational culture and values may influence implementation of CCDSSs' recommendations in different settings. That said, until CCDSSs show more reliable and substantial effects, delays in studies and implementation in resource-limited settings may be fortunate.

Our review endorsed the shift that trials of CCDSS have been making since 1976 [82] from single university-based practices, with medical residents as users, small numbers of patients, and covering a few interventions, to multiple settings, used by physicians and multi-professional teams, encompassing larger numbers of patients with multiple health conditions and interventions. It also supported that assessment of patient outcomes [83], associated costs, and safety have seen limited increases [6, 84, 85].

Study strengths and limitations

We built on our previous review by including only RCTs published in any language, and using duplicate study identification, data abstraction, and study evaluation. Our current focus on RCTs provided a more scientifically robust estimate of CCDSS effectiveness, although the potential for publication bias was not assessed. We confirmed our abstractions with primary authors. We collaborated with clinical decision-makers in extracting and analyzing data, and formulating and disseminating findings. We considered the methodological rigor of trials. We could not use meta-analysis to pool effect sizes because included RCTs presented a considerable variety of systems and outcomes. The vote counting approach that we used to summarize study results does not take into account the size or quality of individual studies.

Our decision-maker partners indicated concerns regarding insufficient reporting on infrastructure and contextual factors in which CCDSSs were evaluated, including impact on clinician workflow and the interoperability across different systems. An assessment of available data across all studies in the review set (166 RCTs) is in progress.

Although trial methods improved over time, our review was hampered by the limitations of the primary studies. CCDSSs should target processes of care that have already been shown to be validly related to improved patient outcomes, but not all studies reported the validity of the targeted processes. In addition, most trials did not assess patient outcomes, and even the trials that did were too small to detect clinically important effects. Further, information on study duration was often missing, limiting our ability to assess sustainability of CCDSS effects.

Conclusions

Our review found a growing number of RCTs that assessed a wide variety of CCDSSs designed to improve PPC. To date, the included trials showed good evidence for the effectiveness of CCDSSs for screening and treatment/management of dyslipidaemia in primary care, and mixed evidence for CCDSSs in screening of cancer and mental health-related conditions, multiple preventive care activities, vaccination, and other preventive care interventions. Although CCDSSs for PPC did not seem to cause any serious adverse effects and may reduce some costs of care, most trials did not assess or report these findings. Despite the cumulative knowledge of CCDSSs, it is still not possible to draw definite conclusions on their effectiveness, especially for patient outcomes, because of heterogeneity in systems, settings, and outcomes assessed.

References

  1. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, Bell LM, Alessandrini EA: Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009, 124 (1): 159-169. 10.1542/peds.2008-2823.

    Article  PubMed  Google Scholar 

  2. Kucher N, Koo S, Quiroz R, Cooper JM, Paterno MD, Soukonnikov B, Goldhaber SZ: Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med. 2005, 352 (10): 969-977. 10.1056/NEJMoa041533.

    Article  CAS  PubMed  Google Scholar 

  3. Desai N, Krause B, Gemmil-Toyama M: Health Information Technology in the United States: can planning lead to reality?. EuroHealth. 2009, 15 (2): 26-28.

    Google Scholar 

  4. Davis K: Investing in health care reform. N Engl J Med. 2009, 360 (9): 852-855. 10.1056/NEJMp0900309.

    Article  CAS  PubMed  Google Scholar 

  5. Fontaine P, Ross S, Zink T, Schilling L: Systematic review of health information exchange in primary care practices. J Am Board Fam Med. 2010, 23 (5): 655-670. 10.3122/jabfm.2010.05.090192.

    Article  PubMed  Google Scholar 

  6. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005, 293 (10): 1223-1238. 10.1001/jama.293.10.1223.

    Article  CAS  PubMed  Google Scholar 

  7. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG: Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006, 144 (10): 742-752.

    Article  PubMed  Google Scholar 

  8. Starfield B: Toward international primary care reform. CMAJ. 2009, 180 (11): 1091-1092. 10.1503/cmaj.090542.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Dexheimer JD, Talbot TR, Sanders DL, Rosenbloom ST, Aronsky D: Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. J Am Med Inform Assoc. 2008, 15 (3): 311-320. 10.1197/jamia.M2555.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Jacobson VJ, Szilagyi P: Patient reminder and recall systems to improve immunization rates. Cochrane Database Syst Rev. 2005, CD003941-3

  11. Chaudhry B: Computerized clinical decision support: will it transform healthcare?. J Gen Intern Med. 2008, 23 (Suppl 1): 85-87.

    Article  PubMed  Google Scholar 

  12. Mollon B, Chong J, Holbrook AM, Sung M, Thabane L, Foster G: Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials. BMC Med Inf Decis Mak. 2009, 9: 11-10.1186/1472-6947-9-11.

    Article  Google Scholar 

  13. Haynes RB, Wilczynski N, the Computerized Clinical Decision Support System (CCDSS) Systematic Review Team: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: Methods of a decision-maker-researcher partnership systematic review. Implement Sci. 2010, 5: 12-

    Article  PubMed  PubMed Central  Google Scholar 

  14. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJM, Gavaghan DJ, McQuay HJ: Assessing the quality of reports of randomized clinical trials: Is blinding necessary?. Control Clin Trials. 1996, 17 (1): 1-12. 10.1016/0197-2456(95)00134-4.

    Article  CAS  PubMed  Google Scholar 

  15. Ahmad F, Hogg-Johnson S, Stewart DE, Skinner HA, Glazier RH, Levinson W: Computer-assisted screening for intimate partner violence and control: a randomized trial. Ann Intern Med. 2009, 151 (2): 93-102.

    Article  PubMed  Google Scholar 

  16. Apkon M, Mattera JA, Lin Z, Herrin J, Bradley EH, Carbone M, Holmboe ES, Gross CP, Selter JG, Rich AS, Krumholz HM: A randomized outpatient trial of a decision-support information technology tool. Arch Intern Med. 2005, 165 (20): 2388-2394. 10.1001/archinte.165.20.2388.

    Article  PubMed  Google Scholar 

  17. Barnett GO, Winickoff RN, Morgan MM, Zielstorff RD: A computer-based monitoring system for follow-up of elevated blood pressure. Med Care. 1983, 21 (4): 400-409. 10.1097/00005650-198304000-00003.

    Article  CAS  PubMed  Google Scholar 

  18. Bertoni AG, Bonds DE, Chen H, Hogan P, Crago L, Rosenberger E, Barham AH, Clinch CR, Goff DC: Impact of a multifaceted intervention on cholesterol management in primary care practices: guideline adherence for heart health randomized trial. Arch Intern Med. 2009, 169 (7): 678-686. 10.1001/archinternmed.2009.44.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Rosenberger EL, Goff DC, Blackwell CS, Williams DT, Crago OL, Ellis SD, Bertoni AG, Bonds DE: Implementing a palm pilot intervention for primary care providers: lessons learned. Contemp Clin Trials. 2009, 30 (4): 321-325. 10.1016/j.cct.2009.03.009.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Burack RC, Gimotty PA, George J, Stengle W, Warbasse L, Moncrease A: Promoting screening mammography in inner-city settings: a randomized controlled trial of computerized reminders as a component of a program to facilitate mammography. Med Care. 1994, 32 (6): 609-624. 10.1097/00005650-199406000-00006.

    Article  CAS  PubMed  Google Scholar 

  21. Burack RC, Gimotty PA, George J, Simon MS, Dews P, Moncrease A: The effect of patient and physician reminders on use of screening mammography in a health maintenance organization. Results of a randomized controlled trial. Cancer. 1996, 78 (8): 1708-1721. 10.1002/(SICI)1097-0142(19961015)78:8<1708::AID-CNCR11>3.0.CO;2-1.

    Article  CAS  PubMed  Google Scholar 

  22. Burack RC, Gimotty PA: Promoting screening mammography in inner-city settings. The sustained effectiveness of computerized reminders in a randomized controlled trial. Med Care. 1997, 35 (9): 921-931. 10.1097/00005650-199709000-00005.

    Article  CAS  PubMed  Google Scholar 

  23. Burack RC, Gimotty PA, George J, McBride S, Moncrease A, Simon MS, Dews P, Coombs J: How reminders given to patients and physicians affected pap smear use in a health maintenance organization: results of a randomized controlled trial. Cancer. 1998, 82 (12): 2391-2400. 10.1002/(SICI)1097-0142(19980615)82:12<2391::AID-CNCR13>3.0.CO;2-K.

    Article  CAS  PubMed  Google Scholar 

  24. Burack RC, Gimotty PA, Simon M, Moncrease A, Dews P: The effect of adding Pap smear information to a mammography reminder system in an HMO: results of randomized controlled trial. Prev Med. 2003, 36 (5): 547-554. 10.1016/S0091-7435(02)00062-2.

    Article  PubMed  Google Scholar 

  25. Chambers CV, Balaban DJ, Carlson BL, Grasberger DM: The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center. J Am Board Fam Pract. 1991, 4 (1): 19-26.

    CAS  PubMed  Google Scholar 

  26. Cannon DS, Allen SN: A comparison of the effects of computer and manual reminders on compliance with a mental health clinical practice guideline. J Am Med Inform Assoc. 2000, 7 (2): 196-203. 10.1136/jamia.2000.0070196.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Cobos A, Vilaseca J, Asenjo C, Pedro-Botet J, Sanchez E, Val A, Torremade E, Espinosa C, Bergonon S: Cost effectiveness of a clinical decision support system based on the recommendations of the European Society of Cardiology and other societies for the management of hypercholesterolemia: Report of a cluster-randomized trial. Dis Manag Health Out. 2005, 13 (6): 421-432. 10.2165/00115677-200513060-00007.

    Article  Google Scholar 

  28. Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, Nichol K, Woolliscroft J, Henderson WG: Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000, 284 (11): 1411-1416. 10.1001/jama.284.11.1411.

    Article  CAS  PubMed  Google Scholar 

  29. Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ: A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001, 345 (13): 965-970. 10.1056/NEJMsa010181.

    Article  CAS  PubMed  Google Scholar 

  30. Emery J, Morris H, Goodchild R, Fanshawe T, Prevost AT, Bobrow M, Kinmonth AL: The GRAIDS Trial: a cluster randomised controlled trial of computer decision support for the management of familial cancer risk in primary care. Br J Cancer. 2007, 97 (4): 486-493. 10.1038/sj.bjc.6603897.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Filippi A, Sabatini A, Badioli L, Samani F, Mazzaglia G, Catapano A, Cricelli C: Effects of an automated electronic reminder in changing the antiplatelet drug-prescribing behavior among Italian general practitioners in diabetic patients: an intervention trial. Diabetes Care. 2003, 26 (5): 1497-1500. 10.2337/diacare.26.5.1497.

    Article  PubMed  Google Scholar 

  32. Flanagan JR, Doebbeling BN, Dawson J, Beekmann S: Randomized study of online vaccine reminders in adult primary care. Proc AMIA Symp. 1999, 755-759. [http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2232874/pdf/procamiasymp00004-0792.pdf]

    Google Scholar 

  33. Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW: Computer-based vs manual health maintenance tracking. A controlled trial. Arch Fam Med. 1994, 3 (7): 581-588. 10.1001/archfami.3.7.581.

    Article  CAS  PubMed  Google Scholar 

  34. Harari D, Iliffe S, Kharicha K, Egger M, Gillmann G, von Renteln-Kruse W, Beck J, Swift C, Stuck A: Promotion of health in older people: a randomised controlled trial of health risk appraisal in British general practice. Age Ageing. 2008, 37 (5): 565-571. 10.1093/ageing/afn150.

    Article  PubMed  Google Scholar 

  35. Kenealy T, Arroll B, Petrie KJ: Patients and computers as reminders to screen for diabetes in family practice. Randomized-controlled trial. J Gen Intern Med. 2005, 20 (10): 916-921. 10.1111/j.1525-1497.2005.0197.x.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Lafata JE, Kolk D, Peterson EL, McCarthy BD, Weiss TW, Chen YT, Muma BK: Improving osteoporosis screening: results from a randomized cluster trial. J Gen Intern Med. 2007, 22 (3): 346-351. 10.1007/s11606-006-0060-9.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Lewis G, Sharp D, Bartholomew J, Pelosi AJ: Computerized assessment of common mental disorders in primary care: effect on clinical outcome. Fam Pract. 1996, 13 (2): 120-126. 10.1093/fampra/13.2.120.

    Article  CAS  PubMed  Google Scholar 

  38. Lowensteyn I, Joseph L, Levinton C, Abrahamowicz M, Steinert Y, Grover S: Can computerized risk profiles help patients improve their coronary risk? The results of the Coronary Health Assessment Study (CHAS). Prev Med. 1998, 27 (5 Pt 1): 730-737.

    Article  CAS  PubMed  Google Scholar 

  39. McPhee SJ, Bird JA, Jenkins CN, Fordham D: Promoting cancer screening. A randomized, controlled trial of three interventions. Arch Intern Med. 1989, 149 (8): 1866-1872. 10.1001/archinte.149.8.1866.

    Article  CAS  PubMed  Google Scholar 

  40. McPhee SJ, Bird JA, Fordham D, Rodnick JE, Osborn EH: Promoting cancer prevention activities by primary care physicians. Results of a randomized, controlled trial. JAMA. 1991, 266 (4): 538-544. 10.1001/jama.266.4.538.

    Article  CAS  PubMed  Google Scholar 

  41. Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A: Computer-generated physician and patient reminders. Tools to improve population adherence to selected preventive services. J Fam Pract. 1991, 32 (1): 82-90.

    CAS  PubMed  Google Scholar 

  42. Overhage JM, Tierney WM, McDonald CJ: Computer reminders to implement preventive care guidelines for hospitalized patients. Arch Intern Med. 1996, 156 (14): 1551-1556. 10.1001/archinte.156.14.1551.

    Article  CAS  PubMed  Google Scholar 

  43. Rogers JL, Haring OM, Goetz JP: Changes in patient attitudes following the implementation of a medical information system. QRB Qual Rev Bull. 1984, 10 (3): 65-74.

    CAS  PubMed  Google Scholar 

  44. Rogers JL, Haring OM, Wortman PM, Watson RA, Goetz JP: Medical information systems: assessing impact in the areas of hypertension, obesity and renal disease. Med Care. 1982, 20 (1): 63-74. 10.1097/00005650-198201000-00005.

    Article  CAS  PubMed  Google Scholar 

  45. Rogers JL, Haring OM: The impact of a computerized medical record summary system on incidence and length of hospitalization. Med Care. 1979, 17 (6): 618-630. 10.1097/00005650-197906000-00006.

    Article  CAS  PubMed  Google Scholar 

  46. Rosser WW, McDowell I, Newell C: Use of reminders for preventive procedures in family medicine. CMAJ. 1991, 145 (7): 807-814.

    CAS  PubMed  PubMed Central  Google Scholar 

  47. Rubenstein LV, McCoy JM, Cope DW, Barrett PA, Hirsch SH, Messer KS, Young RT: Improving patient quality of life with feedback to physicians about functional status. J Gen Intern Med. 1995, 10 (11): 607-614. 10.1007/BF02602744.

    Article  CAS  PubMed  Google Scholar 

  48. Schriger DL, Gibbons PS, Langone CA, Lee S, Altshuler LL: Enabling the diagnosis of occult psychiatric illness in the emergency department: a randomized, controlled trial of the computerized, self-administered PRIME-MD diagnostic system. Ann Emerg Med. 2001, 37 (2): 132-140. 10.1067/mem.2001.112255.

    Article  CAS  PubMed  Google Scholar 

  49. Sequist TD, Zaslavsky AM, Marshall R, Fletcher RH, Ayanian JZ: Patient and physician reminders to promote colorectal cancer screening: a randomized controlled trial. Arch Intern Med. 2009, 169 (4): 364-371. 10.1001/archinternmed.2008.564.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Sundaram V, Lazzeroni LC, Douglass LR, Sanders GD, Tempio P, Owens DK: A randomized trial of computer-based reminders and audit and feedback to improve HIV screening in a primary care setting. Int J STD AIDS. 2009, 20 (8): 527-533. 10.1258/ijsa.2008.008423.

    Article  CAS  PubMed  Google Scholar 

  51. Thomas HV, Lewis G, Watson M, Bell T, Lyons I, Lloyd K, Weich S, Sharp D: Computerised patient-specific guidelines for management of common mental disorders in primary care: a randomised controlled trial. Br J Gen Pract. 2004, 54 (508): 832-837.

    PubMed  PubMed Central  Google Scholar 

  52. Tierney WM, Hui SL, McDonald CJ: Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care. 1986, 24 (8): 659-666. 10.1097/00005650-198608000-00001.

    Article  CAS  PubMed  Google Scholar 

  53. Turner RC, Peden JG, O'Brien K: Patient-carried card prompts vs computer-generated prompts to remind private practice physicians to perform health maintenance measures. Arch Intern Med. 1994, 154 (17): 1957-1960. 10.1001/archinte.154.17.1957.

    Article  CAS  PubMed  Google Scholar 

  54. Unrod M, Smith M, Spring B, DePue J, Redd W, Winkel G: Randomized controlled trial of a computer-based, tailored intervention to increase smoking cessation counseling by primary care physicians. J Gen Intern Med. 2007, 22 (4): 478-484. 10.1007/s11606-006-0069-0.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Smith MY, Cromwell J, DePue J, Spring B, Redd W, Unrod M: Determining the cost-effectiveness of a computer-based smoking cessation intervention in primary care. Manag Care. 2007, 16 (7): 48-55.

    PubMed  Google Scholar 

  56. van Wyk JT, van Wijk MA, Sturkenboom MC, Mosseveld M, Moorman PW, van der Lei J: Electronic alerts versus on-demand decision support to improve dyslipidemia treatment: a cluster randomized controlled trial. Circulation. 2008, 117 (3): 371-378. 10.1161/CIRCULATIONAHA.107.697201.

    Article  PubMed  Google Scholar 

  57. Wilson BJ, Torrance N, Mollison J, Wordsworth S, Gray JR, Haites NE, Grant A, Campbell MK, Miedyzbrodzka Z, Clarke A, Watson MS, Douglas A: Improving the referral process for familial breast cancer genetic counselling: findings of three randomised controlled trials of two interventions. Health Technol Assess. 2005, 9 (3): iii-iv. 1-126

    Article  CAS  Google Scholar 

  58. Wilson BJ, Torrance N, Mollison J, Watson MS, Douglas A, Miedzybrodzka Z, Gordon R, Wordsworth S, Campbell M, Haites N, Grant A: Cluster randomized trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk. Fam Pract. 2006, 23 (5): 537-544. 10.1093/fampra/cml026.

    Article  PubMed  Google Scholar 

  59. Zanetti G, Flanagan HL, Cohn LH, Giardina R, Platt R: Improvement of intraoperative antibiotic prophylaxis in prolonged cardiac surgery by automated alerts in the operating room. Infect Control Hosp Epidemiol. 2003, 24 (1): 13-16. 10.1086/502109.

    Article  PubMed  Google Scholar 

  60. Bonevski B, Sanson-Fisher RW, Campbell E, Carruthers A, Reid AL, Ireland M: Randomized controlled trial of a computer strategy to increase general practitioner preventive care. Prev Med. 1999, 29 (6): 478-486. 10.1006/pmed.1999.0567.

    Article  CAS  PubMed  Google Scholar 

  61. Tang PC, LaRosa MP, Newcomb C, Gorden SM: Measuring the effects of reminders for outpatient influenza immunizations at the point of clinical opportunity. J Am Med Inform Assoc. 1999, 6 (2): 115-121. 10.1136/jamia.1999.0060115.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Tape TG, Campbell JR: Computerized medical records and preventive health care: success depends on many factors. Am J Med. 1993, 94 (6): 619-625. 10.1016/0002-9343(93)90214-A.

    Article  CAS  PubMed  Google Scholar 

  63. Turner BJ, Day SC, Borenstein B: A controlled trial to improve delivery of preventive care: physician or patient reminders?. J Gen Intern Med. 1989, 4 (5): 403-409. 10.1007/BF02599691.

    Article  CAS  PubMed  Google Scholar 

  64. McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, McCabe GP: Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984, 100 (1): 130-138.

    Article  CAS  PubMed  Google Scholar 

  65. Nilasena DS, Lincoln MJ: A computer-generated reminder system improves physician compliance with diabetes preventive care guidelines. Proc Annu Symp Comput Appl Med Care. 1995, 640-645.

    Google Scholar 

  66. Lobach DF, Hammond W: Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med. 1997, 102 (1): 89-98. 10.1016/S0002-9343(96)00382-8.

    Article  CAS  PubMed  Google Scholar 

  67. Meigs JB, Cagliero E, Dubey A, Murphy-Sheehy P, Gildesgame C, Chueh H, Barry MJ, Singer DE, Nathan DM: A controlled trial of web-based diabetes disease management: the MGH diabetes primary care improvement project. Diabetes Care. 2003, 26 (3): 750-757. 10.2337/diacare.26.3.750.

    Article  PubMed  Google Scholar 

  68. Quinn CC, Clough SS, Minor JM, Lender D, Okafor MC, Gruber-Baldini A: WellDocâ„¢ mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther. 2008, 10 (3): 160-168. 10.1089/dia.2008.0283.

    Article  PubMed  Google Scholar 

  69. Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H: Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ. 2009, 181 (1-2): 37-44. 10.1503/cmaj.081272.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Borbolla D, Giunta D, Figar S, Soriano M, Dawidowski A, de Quiros FG: Effectiveness of a chronic disease surveillance systems for blood pressure monitoring. Stud Health Technol Inform. 2007, 129 (Pt 1): 223-227.

    PubMed  Google Scholar 

  71. Montgomery AA, Fahey T, Peters TJ, MacIntosh C, Sharp DJ: Evaluation of computer based clinical decision support system and risk chart for management of hypertension in primary care: randomised controlled trial. BMJ. 2000, 320 (7236): 686-690. 10.1136/bmj.320.7236.686.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD: Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003, 18 (12): 967-976. 10.1111/j.1525-1497.2003.30635.x.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD: Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005, 40 (2): 477-497. 10.1111/j.1475-6773.2005.0t369.x.

    Article  PubMed  PubMed Central  Google Scholar 

  74. Javitt JC, Rebitzer JB, Reisman L: Information technology and medical missteps: evidence from a randomized trial. J Health Econ. 2008, 27 (3): 585-602. 10.1016/j.jhealeco.2007.10.008.

    Article  PubMed  Google Scholar 

  75. Javitt JC, Steinberg G, Locke T, Couch JB, Jacques J, Juster I, Reisman L: Using a claims data-based sentinel system to improve compliance with guidelines: results of a randomized prospective study. Am J Manag Care. 2005, 11 (2): 93-102.

    PubMed  Google Scholar 

  76. Gurwitz JH, Field TS, Rochon P, Judge J, Harrold LR, Bell CM, Lee M, White K, LaPrino J, Erramuspe-Mainard J, DeFlorio M, Gavendo L, Baril JL, Reed G, Bates DW: Effect of computerized provider order entry with clinical decision support on adverse drug events in the long-term care setting. J Am Geriatr Soc. 2008, 56 (12): 2225-2233. 10.1111/j.1532-5415.2008.02004.x.

    Article  PubMed  Google Scholar 

  77. Dexter PR, Wolinsky FD, Gramelspacher GP, Zhou XH, Eckert GJ, Waisburd M, Tierney WM: Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms, a randomized controlled trial. Ann Intern Med. 1998, 128: 102-110.

    Article  CAS  PubMed  Google Scholar 

  78. Dexter PR, Perkins SM, Maharry KS, Jones K, McDonald CJ: Inpatient computer-based standing order vs physician reminders to increase influenza and pneumococcal vaccination rates: a randomized trial. JAMA. 2004, 292 (19): 2366-2371. 10.1001/jama.292.19.2366.

    Article  CAS  PubMed  Google Scholar 

  79. Grant RW, Middleton B: Improving primary care for patients with complex chronic diseases: can health information technology play a role?. CMAJ. 2009, 181 (1-2): 37-44. 10.1503/cmaj.081272.

    Article  Google Scholar 

  80. Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB: Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA. 1996, 276 (15): 1253-1258. 10.1001/jama.276.15.1253.

    Article  CAS  PubMed  Google Scholar 

  81. Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD: Improving preventive care by prompting physicians. Arch Intern Med. 2000, 160 (3): 301-308. 10.1001/archinte.160.3.301.

    Article  CAS  PubMed  Google Scholar 

  82. McDonald CJ: Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976, 295 (24): 1351-1355. 10.1056/NEJM197612092952405.

    Article  CAS  PubMed  Google Scholar 

  83. Neumann PJ, Tunis SR: Medicare and medical technology -- the growing demand for relevant outcomes. N Engl J Med. 2010, 362 (5): 377-379. 10.1056/NEJMp0912062.

    Article  CAS  PubMed  Google Scholar 

  84. Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005, 330 (7494): 765-10.1136/bmj.38398.500764.8F.

    Article  PubMed  PubMed Central  Google Scholar 

  85. Randell R, Mitchell N, Dowding D, Cullum N, Thompson C: Effects of computerized decision support systems on nursing performance and patient outcomes: a systematic review. J Health Serv Res Policy. 2007, 12 (4): 242-249. 10.1258/135581907782101543.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The research was funded by a Canadian Institutes of Health Research Synthesis Grant: Knowledge Translation KRS 91791. The members of the Computerized Clinical Decision Support System (CCDSS) Systematic Review Team included the Principal Investigator, Co-Investigators, Co-Applicants/Senior Management Decision-makers, Co-Applicants/Clinical Service Decision-Makers, and Research Staff. The following were involved in collection and/or organization of data: Jeanette Prorok, MSc, McMaster University; Nathan Souza, MD, MMEd, McMaster University; Brian Hemens, BScPhm, MSc, McMaster University; Robby Nieuwlaat, PhD, McMaster University; Shikha Misra, BHSc, McMaster University; Jasmine Dhaliwal, BHSc, McMaster University; Navdeep Sahota, BHSc, University of Saskatchewan; Anita Ramakrishna, BHSc, McMaster University; Pavel Roshanov, BSc, McMaster University; Tahany Awad, MD, McMaster University; Emma Iserman, BA, McMaster University. Nicholas Hobson Dip.T., Chris Cotoi BEng, EMBA, and Rick Parrish Dip.T., at McMaster University provided programming and information technology support.

Author information

Authors and Affiliations

  1. Health Research Methodology Program, McMaster University, 1280 Main Street West, Hamilton, ON, Canada

    Nathan M Souza

  2. Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, ON, Canada

    Rolf J Sebaldt & R Brian Haynes

  3. Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, ON, Canada

    Jean A Mackay, Jeanette C Prorok, Lorraine Weise-Kelly, Tamara Navarro, Nancy L Wilczynski & R Brian Haynes

  4. Hamilton Health Sciences, 1200 Main Street West, Hamilton, ON, Canada

    R Brian Haynes

Authors
  1. Nathan M Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rolf J Sebaldt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean A Mackay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeanette C Prorok
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lorraine Weise-Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tamara Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nancy L Wilczynski
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. R Brian Haynes
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

the CCDSS Systematic Review Team

Corresponding author

Correspondence to R Brian Haynes.

Additional information

Competing interests

RBH, NLW, RJS, JAM, NMS, LWK, TN, JP received support through the Canadian Institutes of Health Research Synthesis Grant: Knowledge Translation KRS 91791for the submitted work; RJS is the owner of Fig.P Software Incorporated, which develops and sells a chronic disease management system that is not a subject of this review. RBH is acquainted with several CCDSS developers and researchers, including authors of papers included in this review.

Authors' contributions

RBH was responsible for study conception and design; acquisition, analysis and interpretation of data; drafting and critical revision of the manuscript; obtaining funding; and study supervision. He is the guarantor. NMS acquired, analysed, and interpreted data; drafted and critically revised the manuscript; and provided statistical analysis. RJS analysed and interpreted data; and drafted and critically revised the manuscript. JAM acquired, analysed, and interpreted data; drafted the manuscript; and provided administrative, technical or material support. JP acquired data; drafted the manuscript; and provided administrative, technical, or material support. LWK and TN acquired data and drafted the manuscript. NLW acquired, analysed, and interpreted data; drafted the manuscript; provided administrative, technical, or material support; and provided study supervision. All authors have read and approved the final manuscript.

Electronic supplementary material

13012_2011_406_MOESM1_ESM.DOCX

Additional File 1: Study methods scores for trials of primary preventive care. Methods scores for the included studies. (DOCX 23 KB)

13012_2011_406_MOESM2_ESM.DOCX

Additional File 2: CCDSS characteristics for trials of primary preventive care. CCDSS characteristics of the included studies. (DOCX 40 KB)

13012_2011_406_MOESM3_ESM.DOCX

Additional File 3: Study characteristics for trials of primary preventive care. Study characteristics of the included studies. (DOCX 46 KB)

13012_2011_406_MOESM4_ESM.DOCX

Additional File 4: Results for CCDSS trials of primary preventive care. Details results of the included studies. (DOCX 89 KB)

13012_2011_406_MOESM5_ESM.DOCX

Additional File 5: Costs and CCDSS process-related outcomes for trials of primary preventive care. Cost and CCDSS process-related outcomes for the included studies. (DOCX 27 KB)

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Souza, N.M., Sebaldt, R.J., Mackay, J.A. et al. Computerized clinical decision support systems for primary preventive care: A decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implementation Sci 6, 87 (2011). https://doi.org/10.1186/1748-5908-6-87

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1748-5908-6-87

Keywords

Implementation Science

ISSN: 1748-5908