- Research article
- Open Access
- Open Peer Review
Designing an automated clinical decision support system to match clinical practice guidelines for opioid therapy for chronic pain
- Jodie A Trafton†1Email author,
- Susana B Martins†1, 2,
- Martha C Michel†1,
- Dan Wang1,
- Samson W Tu3,
- David J Clark4,
- Jan Elliott4,
- Brigit Vucic1,
- Steve Balt1,
- Michael E Clark5,
- Charles D Sintek6, 7,
- Jack Rosenberg8,
- Denise Daniels8 and
- Mary K Goldstein2, 1, 9
https://doi.org/10.1186/1748-5908-5-26
© Trafton et al; licensee BioMed Central Ltd. 2010
- Received: 21 July 2009
- Accepted: 12 April 2010
- Published: 12 April 2010
Abstract
Background
Opioid prescribing for chronic pain is common and controversial, but recommended clinical practices are followed inconsistently in many clinical settings. Strategies for increasing adherence to clinical practice guideline recommendations are needed to increase effectiveness and reduce negative consequences of opioid prescribing in chronic pain patients.
Methods
Here we describe the process and outcomes of a project to operationalize the 2003 VA/DOD Clinical Practice Guideline for Opioid Therapy for Chronic Non-Cancer Pain into a computerized decision support system (DSS) to encourage good opioid prescribing practices during primary care visits. We based the DSS on the existing ATHENA-DSS. We used an iterative process of design, testing, and revision of the DSS by a diverse team including guideline authors, medical informatics experts, clinical content experts, and end-users to convert the written clinical practice guideline into a computable algorithm to generate patient-specific recommendations for care based upon existing information in the electronic medical record (EMR), and a set of clinical tools.
Results
The iterative revision process identified numerous and varied problems with the initially designed system despite diverse expert participation in the design process. The process of operationalizing the guideline identified areas in which the guideline was vague, left decisions to clinical judgment, or required clarification of detail to insure safe clinical implementation. The revisions led to workable solutions to problems, defined the limits of the DSS and its utility in clinical practice, improved integration into clinical workflow, and improved the clarity and accuracy of system recommendations and tools.
Conclusions
Use of this iterative process led to development of a multifunctional DSS that met the approval of the clinical practice guideline authors, content experts, and clinicians involved in testing. The process and experiences described provide a model for development of other DSSs that translate written guidelines into actionable, real-time clinical recommendations.
Keywords
- Electronic Medical Record
- Decision Support System
- Knowledge Management
- Clinical Practice Guideline
- Opioid Prescribe
Background
Promoting use of good care practices is necessary for safe and effective use of opioid therapy for chronic non-cancer pain, but achieving provider adherence to clinical practice guideline (CPG) recommended care practices has proven difficult in most primary health care settings [1–3]. Increased attention to the importance of pain management has led to increased prescribing of analgesic medications [4]. Opioid analgesics are among the most prescribed medications in the US today [5, 6] and, as of 2008, hydrocodone was the top prescribed medication in the country [4]. However, increased use of these powerful and potentially addictive medications has had negative consequences. Rates of opioid overdose, prescription opioid misuse and addiction, diversion of prescribed medications toward illicit use, and opioid-related legal suits against physicians have all increased to a disturbing extent [4, 5]. Use of recommended care practices is considered essential for minimizing these negative consequences without reversing gains made in improving pain management in clinical settings.
In 2003, the Veterans Administration (VA)/Department of Defense (DOD) published a CPG for use of opioid therapy for the treatment of chronic non-cancer pain [7]. The goals included using evidence-based recommendations to improve analgesia, promote uniformity of care, and decrease related morbidity of patients with non-cancer chronic pain in the primary care setting. This guideline provides detailed information about appropriate dosing, including protocols for initiation, titration, and cessation of the most commonly used opioid medications. It provides information on potential contraindications for opioid therapy for chronic pain and suggestions for opioid management in patients at higher risk of misuse, diversion, adverse effects, overdose, and/or lack of efficacy. A substantial portion of the guideline focuses on processes of care. For example, the guideline encourages clinicians to: regularly conduct assessments of pain and functioning; use urine drug screening protocols to discourage and detect medication misuse and diversion; obtain written agreement on the parameters and responsibilities of the patient regarding the opioid prescription; provide clear education on both the risks and realistic level of benefit from opioid analgesics; and carefully document and follow treatment plans. This framework can increase clinician's confidence in appropriately prescribing opioid therapy.
Despite expert consensus on the importance of adherence to these care guidelines, there is little evidence that they are consistently followed in actual clinical practice [8]. Numerous barriers to providing guideline-adherent care exist [9]. Clinicians report lack of training in both pain management and addiction medicine and are uncomfortable assessing and treating these conditions. Moreover, patient-provider communication about opioids is complicated by: the subjective nature of pain experience, which prevents physicians from objectively verifying the severity of the pain condition; the reinforcing effects of opioid drugs, which may lead to either deliberate or unknowing attempts by the patient to obtain opioid medications; provider and patient fears about the consequences of either prescribing a potentially addictive medication or under-managing pain; and stigma associated with substance use disorders [10–12]. Because of these communication difficulties, providers may be hesitant to prescribe opioid medications initially, to discontinue medication when there is no clear sign of benefit, and to address the addictive nature of opioid analgesics and the possibility of misuse. In all cases, these behaviors lead to suboptimal care. Additionally, poor care coordination within the health care system contributes to poor opioid management [13]. Lack of clear documentation of pain management plans and opioid use agreements and lack of communication between providers can lead to inconsistent treatment and poor prescribing decisions that contribute to misuse and poor pain management. Lastly, good care practices take time, and time limitations and competing demands during outpatient visits in primary care may limit clinician adherence to guidelines.
Developing health services interventions that address these barriers is essential for improving opioid management in chronic pain. A computerized decision support system (DSS) may provide such an intervention [14, 15], and some DSSs have been shown to increase adherence to guideline recommended care [16]. Hunt and colleagues systematically reviewed randomized controlled trials of DSSs, defined as 'any electronic or non-electronic system designed to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians for consideration' [17]. Kawamoto and colleagues identified features that were independently associated with improved clinical practice in a multiple regression analysis. These included: automatic delivery, presentation of the DSS when and where clinical decision making occurs, provision of concrete recommendations of how to proceed, and computer-based generation of decision support [18]. A model computerized DSS (ATHENA-DSS) that links with the electronic medical record (EMR) system used by the VA Health Care System (VistA) was designed to provide these key features [19–22]. ATHENA-DSS, developed using the EON guideline decision-support technology [23, 24], accesses patient information in the EMR, evaluates this information in terms of a knowledge base consisting of encoded CPG recommendations, generates patient-specific recommendations, and presents a graphical user interface with these recommendations along with tools and information support to clinicians when they open the EMR of a relevant patient at the time of the clinic visit.
Model of CPG translation and revision. This figure describes the products, review processes and reviewers for the three main products of the ATHENA-OT CPG translation project.
Methods
The patient safety features and a thorough description of the ATHENA-OT graphical user interface have been described previously [25]. This study was approved and overseen by the Stanford University Human Research Protection Program and the VA Palo Alto Health Care System Research and Development Committee.
In the process described, the team started with the CPG and translated it into three primary products: an operationalized algorithm in Protégé/EON, a matching written Rules Document, plus a set of clinical tools (Figure 1). We based our guideline translation process on experience gained in development of ATHENA-Hypertension as well as general principles from medical informatics literature encouraging iterative design based on interim evaluation and testing (for example, the ADDIE (Analysis, Design, Development, Implementation and Evaluation) process [27]. The accuracy testing procedures for both the Protégé/EON algorithm and the clinical tools were adapted from those initially designed and successfully used by the ATHENA-Hypertension development team [28]. The Rules Document validation process was designed for ATHENA-OT and has not been previously described, and thus we report this process and findings in greater detail.
A knowledge management team (KM) consisting of the study managers, knowledge modelers (SBM and MM, medical informaticists with expertise in translation of clinical knowledge into encoded computer-interpretable formats using a knowledge acquisition program called Protégé [29]), and system software experts drafted, revised, and managed the review of these 3 products. Each of these three products were reviewed and revised through separate procedures and distinct, but overlapping teams. Revisions to the Protégé/EON algorithm and the Rules Document were made in tandem to maintain consistency, based on feedback from the accuracy and rules validation testing. These processes occurred iteratively during ATHENA-OT development. Each of the processes, as well as major revisions, are described below.
Drafting a Rules Document and operationalized algorithm in Protégé/EON
To create an encoded guideline, one must specify details that are not explicitly included in the CPG [30]. For example, the CPG for opioid therapy states: 'long-acting agents are effective for continuous, chronic pain'. This statement fails to specify which medications should be considered 'long-acting agents' and the definition of continuous, chronic pain. In order for the computer to be able to use this information, the definitions of 'long-acting agents' and 'continuous, chronic pain' must be explicitly defined or operationalized.
To operationalize the 2003 VA/DOD 'Clinical Practice Guideline for Opioid Therapy for Chronic Non-Cancer Pain', the KM, the medical director, and clinical nurse specialist who direct the VA Palo Alto Health Care System Pain Management Clinic worked collaboratively to create a draft of the guideline knowledge to be encoded in Protégé and specify concepts that were not clearly defined. The process involved the KM reviewing the CPG and attempting to translate the contained recommendations into well-defined concepts that could be encoded in terms of a computer-interpretable model of CPGs [23]. The KM referred questions to clinical experts to iteratively refine the encoded guideline. In addition to encoding the knowledge in Protégé/EON, a 'Rules Document' was created that provided a written description in simple but highly-specified English of the included concepts and rules that the developers intended to encode. The Rules Document serves as a format for review by clinicians and CPG authors [28].
Review processes
Accuracy testing of the Protégé/EON algorithm
Experts in opioid therapy for chronic pain, including the clinical nurse specialist at the VA Palo Alto Pain Management Clinic, a Ph.D. researcher specializing in opioid pharmacology and behavior, a primary-care physician, and a psychiatrist, pilot tested the encoded guideline iteratively during the development and refinement of the operationalized algorithm. Accuracy testing involved examination of ATHENA-OT recommendations for real patient cases with recent primary care visits selected from the VA Palo Alto's EMR. ATHENA-OT generated definitions and recommendations that were compared to information in the EMR and to expert assessment of the patient case in the EMR using the CPG recommendations. Straightforward errors in the generated recommendations were noted and sent to the KM for immediate correction (e.g., miscoding of a diagnosis or minor wording changes). Concerns involving clinical recommendations were first discussed by the expert reviewers, and final suggestions for changes to the encoded guideline were sent to the KM. When suggestions were outside the boundaries of the DSS, the KM met again with the expert reviewers to discuss options and insure that the boundaries were made clear to clinical users to avoid false expectations on the part of the user about the system's capabilities.
Validation of the draft Rules Document by authors of the CPG
Once the encoded guideline had been pilot tested for accuracy and the Rules Document updated to match the current content of the encoded guideline, the Rules Document was sent to three authors of the 2003 VA/DOD Opioid Therapy for Chronic Non-Cancer Pain CPG (MC, JR, CS). For each clinical rule, the authors were asked to consider the CPG and indicate first whether the clinical rule agreed with the intent of the CPG as written or was incorrect based upon the intent of the CPG. Second, they were asked to comment when the Rules Document was not clear and further clarification of intent of the encoded guideline was required. The guideline-authors' comments included details regarding clinical rules with which they disagreed or that they thought needed refinement. This feedback was used to revise the Rules Document and Protégé/EON algorithm to address the guideline authors' concerns.
Clinical tool design
The CPG contained many recommendations to support good clinical care practices that were best shared with primary care clinicians through easily accessible tools (links within the DSS). In discussion with VA Palo Alto clinicians in the Pain Management and Primary Care Clinics, the KM developed information sheets and other clinical tools within ATHENA-OT to facilitate adherence to the CPG recommendations. These tools were vetted and, where appropriate, pilot tested for accuracy by the clinical staff at the pain management clinic and opioid experts on the project team.
User interface design
A final step in translating the CPG into ATHENA-OT was determining how to present patient-specific recommendations and clinical tools to the clinicians most effectively. Accordingly, in consultation with a graphic design firm, we used an iterative design and evaluation process to optimize the graphical user interface. This process is described elsewhere [26]. While it is difficult to completely dissociate the development of the user interface from the process of translating the guideline, here we focus only on development of clinical tools and patient-specific recommendations suggested in the CPG.
Revision of the Rules Document and Protégé/EON algorithm
Based upon feedback from the review processes, a substantial redesign of the algorithm was conducted. Following system redesign, in depth re-testing of the accuracy the Protégé/EON algorithm was conducted, and the revised Rules Document was again sent out to the three CPG authors for a second round of validation. The CPG authors indicated additional areas of disagreement or requirements for clarification. In this round, the exact wording of DSS recommendations was provided for review. Final consensus on the Rules Document was obtained by conducting follow-up phone calls and emails with the CPG authors where remaining changes were planned, specified, and approved.
Results
Drafting of a Rules Document and operational algorithm in Protégé/EON
The KM and clinical experts met approximately 30 times over the course of nine months, and had extensive email communication. The encoded guideline in Protégé/EON included: operationalized definitions of all the concepts included in the guideline (e.g., the ICD-9 codes corresponding to a named diagnosis, or the pharmacy codes for medications of a specified class); an algorithm that operationalized guideline recommendations in terms of the relevant patient scenarios, management decisions, and alternative actions; a collection of situations that warrant warning messages; and declarative specification of the indications, contraindications, and dose ranges of classes of opioids.
Examples of Boundaries of ATHENA-Opioid Therapy
Issue | Solution |
---|---|
Lack of expert consensus on specific criteria for judging an opioid trial as failed and thus appropriate to discontinue. | The determination of whether to discontinue an opioid medication was left to clinical judgment and always presented as an option. Detailed instructions on how but not when to discontinue the opioid medication were provided. |
CPG was written to guide prescribing for non-cancer pain. Some patients have cancer plus pain from non-cancer-related causes, making it unclear whether the CPG was appropriate to apply. | ATHENA-OT issued a warning when the patient had cancer and indicated that system recommendations may not be appropriate if the patient's pain was caused by the cancer. |
Determination of the severity of illness requires clinical assessment during the current visit. | ATHENA-OT issued a warning about potentially concerning diagnoses and recommended that the clinician assess the patient's current status to clarify if opioid dose adjustments were necessary. |
In the electronic medical record (VistA), allergies are not distinguished from adverse events. | As a conservative measure, any record of an allergy/adverse event was considered an allergy, and recommendations were generated based on this assumption. This definition was clarified in clinician training sessions. |
Round one review
Accuracy testing of the Protégé/EON algorithm
Accuracy testing of the Protégé/EON algorithm identified numerous errors in CPG coding that were subsequently corrected. Commonly identified technical errors included omissions of important medical record data in the ATHENA-OT data extract and miscoding of concepts such that recommendations were not produced as planned. Less commonly, clinical cases that had not been anticipated previously by the KM and clinical experts were identified that required refinement of recommendations to align with the assumed intent of the CPG.
Clinical practice guideline author agreement with Rules Document
Rule Category | Agreement (%) | Clarification (%) | ||
---|---|---|---|---|
Round one | Round two | Round one | Round two | |
Drug recommendations overall | 82 | 79 | 24 | 21 |
1) Initiation dosing | 93 | 43 | 7 | 64 |
2) Titration dosing | 89 | 89 | 11 | 0 |
3) Switching dosing | 100 | 100 | 14 | 0 |
4) Cessation dosing | 28 | 100 | 86 | 0 |
5) Medication choice | - | 100 | - | 0 |
Contraindications/warnings overall | 84 | 86 | 44 | 12 |
1) Medical contraindications | 80 | 69 | 75 | 0 |
2) Psychiatric contraindications | 94 | 89 | 30 | 15 |
3) Psychosocial contraindications | 63 | 100 | 47 | 0 |
Patient eligibility and exclusion | 66 | 100 | 44 | 0 |
Thus, CPG authors' comments in round one Rules Document assessment suggested problems with an overall decision support strategy of providing clinicians with a single actionable recommendation for opioid prescribing (e.g., 'increase dose of medication [X] by [Y] mg'). Guideline author comments made it clear that clinician judgment, patient preferences, and information not available in the EMR were crucial to providing CPG-adherent opioid therapy, and that a decision support strategy providing greater clinical flexibility would be more appropriate.
Revision of the Rules Document and Protégé/EON algorithm
A substantial redesign was conducted. This redesign addressed several concerns that had not previously been solved because of lack of consensus or detail in the CPG or lack of information in the EMR. Instead of displaying our best 'guess' about the recommended course for opioid prescribing, we decided to display all possible therapeutic options for the provider to select from based on clinical judgment. Specifically, we switched from presenting clinicians with detailed procedural or dosing recommendations for the system's one best guess regarding the appropriate strategy for dosing change (i.e., start medication, increase dose, decrease dose, switch to a different medication, or stop medication) to providing detailed procedural or dosing recommendations for all possible options with presentation of indications and contraindications for each choice. This modification emphasized the fact that clinical decisions about overall strategy for opioid therapy require assessment of physical and social functioning and the patients' goals and preferences for treatment as well as clinical judgement. Thus, this clinical decision requires clinician-patient discussion during the visit and cannot be made based on information solely in the EMR. This design choice allowed the team to focus ATHENA-OT on insuring safe and informed implementation of treatment strategies following a shared clinical decision-making model [11].
Round two review of the Protégé/EON algorithm and Rules Document
Accuracy testing was conducted again. Errors in Protégé/EON coding were identified and corrected, and wording of recommendations was edited as recommended by the expert testers. After the Rules Document was updated based on the comments from the initial assessment and the system redesign, CPG authors re-evaluated the clinical rules (See Appendix 2 for the Rules Document for round two review). Notably, in round two, the wording of clinical recommendations was included in the Rules Document for approval and comment. In this second round review, CPG authors indicated increased consensus on the Rules Document used for ATHENA-OT. The CPG authors agreed with a higher proportion of the clinical rules as written, and they requested clarification of fewer clinical rules, with the notable exception of several suggestions for detailed modifications which affected many of the rules for initiation dosing and contraindication warnings (Table 2). These exceptions are discussed below. As the Rules Document became more defined and refined, CPG author concerns and comments became more detailed. Suggested changes and clarifications became more minor, although the number of suggestions did not decline in every category. While major redesign of ATHENA-OT was required to address guideline author comments in round one, round two revisions were minor enough to be resolved with small wording changes in existing patient-specific recommendations or slight modification of diagnostic definitions. These changes were discussed and approved in follow-up telephone calls with the CPG authors.
Examples of areas of disagreement in the Rules Document and revisions
Round | Examples of disagreement | Guideline author's reasoning | Revisions made in response |
---|---|---|---|
Round one | Strict discontinuation messages based on substance abuse or psychiatric diagnosis | Need to evaluate current status of diagnosis | Updated algorithm to generate all therapeutic options with contraindications for provider to consider and apply based on clinical judgement |
Round one | Warning for patients that live >200 miles from VA. | Patients may have continuity of care even if living far away. | Recommendation to not provide opioid therapy and refer patient for care with a local provider was removed |
Round two | Dose recommendation for short acting opioids combined with NSAIDs/acetaminophen | Concern about dose of NSAIDs/acetaminophen with medication combinations | Wording of message updated |
Round two | Warning about patients with personality disorder | Warn specifically about anti-social and borderline personality disorders | Restricted warnings to persons with these specific personality disorder diagnoses. |
Clinical tools development
The CPG provided information regarding definitions, assessment and documentation requirements, patient education materials, clinical referral needs, and dosing conversion tables that could not be efficiently presented as patient-specific recommendations. To include these elements of the guideline in ATHENA-OT, we developed clinical tools and information sheets that were incorporated into menus on the graphical user interface. These tools were derived from the CPG with additional input from pain experts and primary care providers.
These clinical tools included interactive systems, such as a conversion calculator (Additional file 1) that was created based on tables in the guideline to facilitate dose conversion when switching between opioids. Interactive assessment instruments were also developed to improve CPG adherence. The CPG recommends that clinicians conduct and document a comprehensive biopsychosocial assessment of pain before opioid prescribing and at subsequent visits. Discussion with local primary care clinicians in usability testing revealed that such assessments were not completed in a standardized manner and that clinicians were uncertain about the detailed elements that should be included in their assessments. Experts and target users agreed that a standardized pain assessment form that could be written back into the EMR would be helpful for clinicians conducting these pain assessments and reassessments. We designed such a pain assessment (Additional file 2) based upon existing tools and the recommendations of the local pain clinic staff. This tool provided check boxes to record patient information that could be written back into a structured progress note (Additional file 3) in the patient EMR for clinician review and signature.
Some information sheets were created to provide locally tailored versions of information for providing guideline-adherent therapy. For example, a contact list for local referral sources for pain, addiction, rehabilitation, and behavioral therapy was created. Similarly, state legal requirements for documentation of opioid management were outlined in another information sheet. We also included simple pre-existing information sheets and patient education documents to ensure that these were readily accessible to clinicians.
Discussion
Collaboration on an iterative design process between the ATHENA-OT developers, local content experts, and the authors of the CPG for management of opioid therapy for chronic pain identified issues that arise in encoding a DSS and provided a mechanism for their resolution, resulting in an automated DSS that fulfils the intentions of the CPG authors.
Clinicians who provide care for patients based on guidelines must operationalize them to carry out care. Operationalizing CPGs for automated DSS highlights the context, assumptions, ambiguities, and gaps that are inherent in the usual formats of CPGs [30]. Encoding the DSS requires interpreting the context, specifying concepts, clarifying embedded assumptions, spanning gaps, and resolving ambiguities in source documents. Evidence sources for recommendations can sometimes serve as sources for specification of concepts (e.g., which specific diagnoses were included in the original study that forms the evidence base for a particular recommendation) or target population assumptions, but do not fill all the gaps. Consensus-based recommendations may present a particular challenge for translation into computerized DSS. These recommendations typically do not reference scientific studies on which definitions of concepts or cut-points for decisions could be based [30]. Without participation of the CPG authors, who have extensive expert knowledge beyond what is written into the CPG document, the intent or specificity of the guideline may be altered as it is translated into a computerized DSS. In developing ATHENA-OT, we used a two-phase process including accuracy testing plus a review of the guideline rules by three authors of the CPG, followed by a final round of telephone and email communications to arrive at a final version meeting approval of the CPG authors.
This process clarified details of the recommendations and better specified patient populations for whom recommendations were relevant. However, the process was made more challenging by the consensus nature of the guidelines. There was not perfect agreement among the CPG authors for all recommendations. Moreover, in some cases, it seemed apparent that some vague areas of the guideline were purposefully written to be vague, an option that could not be directly incorporated in the computerized algorithm beyond careful wording of the recommendations. Encoding the guideline into a DSS revealed areas of the guideline that lacked specificity and therefore might be difficult to implement both in the DSS and clinically. We note that the opioid therapy guidelines may have been more challenging to operationalize than other guidelines in this regard, due to the extreme heterogeneity of patients and underlying health conditions, limited evidence-base for many criteria, and controversies surrounding treatment.
Operationalizing the opioid therapy for chronic pain guideline was further complicated by the fact that good opioid prescribing decisions require consideration of behavioral, mental health, and psychosocial conditions and functioning. Moreover, patient preferences and goals necessarily influence decisions about use of opioid therapy. Prescribing decisions involve striking a balance between pain control, adverse-effect management, physical and emotional function, and addiction risk. This information can be efficiently obtained only in discussion between patient and provider; there is little to no computable information recorded in the medical record to indicate biopsychosocial functioning and patient preferences to guide such decisions. What information is available in the medical record is typically contained in free-text notes rather than structured data fields and would require advanced text mining algorithms to access, an option that may be available in the future. To design a functional DSS, the expert team was required to consider how the algorithm could usefully support prescribing decisions while not having access to crucial information required to guide opioid prescribing. To overcome these limitations, the expert team designed a DSS that would highlight important clinical information available in the medical record, provide many tools to support shared decision making, facilitate appropriate documentation, and present a checklist of important items to review when considering opioid therapy.
An issue that led to difficulties in getting expert consensus was the problem of differentiating between accurate information and information that was of high clinical priority. There was disagreement among the expert team and CPG authors about whether all accurate information should be provided by the DSS, or whether messages should be limited to issues that were clearly important enough to warrant taking primary care clinicians' time. The process of obtaining consensus on the system could be streamlined by clarifying whether the goal of rules development was to identify all clinically accurate recommendations that could be made from EMR data, or to identify important clinical recommendations for typical primary care practice. The process of analysing the encoded guideline rules presented the guideline information in a different way to guideline authors leading to new insight on their part. This insight made explicit some implicit assumptions by showing varying ways the guideline could be interpreted and applied, which may not have been the intention of the guideline authors.
The wording of recommendations and design of clinical tools evolved alongside the operationalization for the guideline. Striking a balance between detail, accuracy, and clinical utility was difficult, with disagreements among the diverse members of the development and evaluation team leading to dynamic changes in the content of recommendations in terms of prioritization, tone, and wordiness. The CPG suggested tools that would facilitate guideline recommended care processes, but expert and end-user input was needed to optimize content and design. For example, based on feedback from clinician members of the team and usability testing [26], we implemented a write back capacity of the pain assessment template.
- 1.
Including an explicit focus on defining the boundaries or limits of the DSS and how they would be handled at the start of the translation process. Outlining these in the Rules Document for review would help ensure that they are considered and addressed thoroughly during initial DSS design.
- 2.
Collecting information from CPG authors regarding prioritization of DSS recommendations during review of the Rules Document. Both CPG authors and end-users participating in usability testing [31] brought up issues regarding distinguishing the clinical importance of recommendations.
- 3.
Presenting specific wording of DSS recommendations as well as an indication of whether the wording should be presented on the main screen or only after end-user interaction with the system (e.g., presented following a mouse click) to CPG authors to review as part of the Rules Document. Subtle wording changes in the DSS recommendations could alter author consensus on recommendations.
Summary
An iterative process of drafting, testing, reviewing, and revising the DSS content enabled us to develop a DSS that usefully operationalized the written CPG for opioid therapy for chronic pain into a system that could provide real-time decision support in clinical settings. Including guideline authors alongside the local expert team in the iterative development of the content resulted in a product that was considered consistent with the intent of the guideline and amenable to implementation with the local EMR and patient population. The process and experiences described here provide a model for development of the content other DSSs attempting to translate written CPGs into actionable, real-time clinical recommendations and tools.
Notes
Declarations
Acknowledgements
We would like to thank Nadeem Riaz for his participation in initial attempts to operationalize the guideline and create a test environment to evaluate the encoded algorithm, Phil Alper for drafting the initial wording of recommendations, and Naquell Johnson for facilitating communication between study team members. This project was funded by Grant # TRX 04-402 entitled,'Decision Support for the Management of Opioid Therapy in Chronic Pain' and Grant # IMA 04-156, entitled 'Building Expertise to Develop and Implement Decision Support Systems' from the VA Health Service Research and Development (HSR&D) Program, and built on work conducted in Grant # VA HSR&D CPI 99-275 entitled 'Guidelines for Drug Therapy of Hypertension: Multi-Site Implementation'. The VA Health Service Research and Development Program provided funding and reviewed the original grant proposal and annual project progress. They did not participate in the collection, analysis or interpretation of data, the writing of the manuscript or the decision to submit the manuscript for publication. The views expressed are those of the authors and not necessarily those of the Department of Veterans Affairs.
Authors’ Affiliations
References
- Farmer AP, Legare F, Turcot L, Grimshaw J, Harvey E, McGowan JL, Wolf F: Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2008, CD004398-Google Scholar
- Maviglia SM, Zielstorff RD, Paterno M, Teich JM, Bates DW, Kuperman GJ: Automating complex guidelines for chronic disease: lessons learned. J Am Med Inform Assoc. 2003, 10: 154-165. 10.1197/jamia.M1181.View ArticlePubMedPubMed CentralGoogle Scholar
- Worrall G, Chaulk P, Freake D: The effects of clinical practice guidelines on patient outcomes in primary care: a systematic review. CMAJ. 1997, 156: 1705-1712.PubMedPubMed CentralGoogle Scholar
- Manchikanti L, Singh A: Therapeutic opioids: a ten-year perspective on the complexities and complications of the escalating use, abuse, and nonmedical use of opioids. Pain Physician. 2008, 11: S63-88.PubMedGoogle Scholar
- Kuehn BM: Opioid prescriptions soar: increase in legitimate use as well as abuse. Jama. 2007, 297: 249-251. 10.1001/jama.297.3.249.PubMedGoogle Scholar
- Pletcher MJ, Kertesz SG, Kohn MA, Gonzales R: Trends in opioid prescribing by race/ethnicity for patients seeking care in US emergency departments. Jama. 2008, 299: 70-78. 10.1001/jama.2007.64.PubMedGoogle Scholar
- VA/DoD Opioid Therapy for Chronic Pain Working Group: The Management of Opioid Therapy for Chronic Pain. 2003, Washington DC: The Office of Quality and Performance, VA and Quality Management Directorate, US Army MEDCOMGoogle Scholar
- Yanni LM, Weaver MF, Johnson BA, Morgan LA, Harrington SE, Ketchum JM: Management of chronic nonmalignant pain: a needs assessment in an internal medicine resident continuity clinic. J Opioid Manag. 2008, 4: 201-211.PubMedGoogle Scholar
- Glajchen M: Chronic pain: treatment barriers and strategies for clinical practice. J Am Board Fam Pract. 2001, 14: 211-218.PubMedGoogle Scholar
- Ballantyne JC: Opioid analgesia: perspectives on right use and utility. Pain Physician. 2007, 10: 479-491.PubMedGoogle Scholar
- Frantsve LM, Kerns RD: Patient-provider interactions in the management of chronic pain: current findings within the context of shared medical decision making. Pain Med. 2007, 8: 25-35. 10.1111/j.1526-4637.2007.00250.x.View ArticlePubMedGoogle Scholar
- Savage SR, Kirsh KL, Passik SD: Challenges in using opioids to treat pain in persons with substance use disorders. Addict Sci Clin Pract. 2008, 4: 4-25. 10.1151/ascp08424.View ArticlePubMedPubMed CentralGoogle Scholar
- Redmond K: Organizational barriers in opioid use. Support Care Cancer. 1997, 5: 451-456. 10.1007/s005200050113.View ArticlePubMedGoogle Scholar
- Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, Spurr C, Khorasani R, Tanasijevic M, Middleton B: Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003, 10: 523-530. 10.1197/jamia.M1370.View ArticlePubMedPubMed CentralGoogle Scholar
- Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, Campbell E, Bates DW: Grand challenges in clinical decision support. J Biomed Inform. 2008, 41: 387-392. 10.1016/j.jbi.2007.09.003.View ArticlePubMedGoogle Scholar
- Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. Jama. 2005, 293: 1223-1238. 10.1001/jama.293.10.1223.View ArticlePubMedGoogle Scholar
- Hunt DL, Haynes RB, Hanna SE, Smith K: Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. Jama. 1998, 280: 1339-1346. 10.1001/jama.280.15.1339.View ArticlePubMedGoogle Scholar
- Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005, 330: 765-10.1136/bmj.38398.500764.8F.View ArticlePubMedPubMed CentralGoogle Scholar
- Goldstein MK, Coleman RW, Tu SW, Shankar RD, O'Connor MJ, Musen MA, Martins SB, Lavori PW, Shlipak MG, Oddone E: Translating research into practice: organizational issues in implementing automated decision support for hypertension in three medical centers. J Am Med Inform Assoc. 2004, 11: 368-376. 10.1197/jamia.M1534.View ArticlePubMedPubMed CentralGoogle Scholar
- Goldstein MK, Hoffman BB, Coleman RW, Tu SW, Shankar RD, O'Connor M, Martins S, Advani A, Musen MA: Patient safety in guideline-based decision support for hypertension management: ATHENA DSS. J Am Med Inform Assoc. 2002, 9: S11-S16. 10.1197/jamia.M1218.View ArticlePubMed CentralGoogle Scholar
- Goldstein MK, Hoffman BB, Coleman RW, Musen MA, Tu SW, Advani A, Shankar R, O'Connor M: Implementing clinical practice guidelines while taking account of changing evidence: ATHENA DSS, an easily modifiable decision-support system for managing hypertension in primary care. Proc AMIA Symp. 2000, 300-304.Google Scholar
- Goldstein M, Hoffman B: Hypertension Recordkeeping and Electronic Management Systems. Hypertension Primer: the essentials of high blood pressure. Edited by: Izzo J, Sica D, Black H. 2008, Dallas, TX: American Heart Association with Lippincott Williams & Wilkins, 428-431.Google Scholar
- Tu SW, Musen MA: A flexible approach to guideline modeling. Proc AMIA Symp. 1999, 420-424.Google Scholar
- Tu SW, Musen MA: From guideline modeling to guideline execution: defining guideline-based decision-support services. Proc AMIA Symp. 2000, 863-867.Google Scholar
- Michel MC, Trafton JA, Martins SB, Wang DW, Tu S, Johnson NA, Goldstein MK: Improving Patient Safety using ATHENA-Decision Support System Technology: Opioid Therapy for Chronic Pain. Advances in Patient Safety: From Research to Implementation. Edited by: Henriksen K, Battles J, Keyes M, Grady M. 2008, Rockville, MD: Agency for Healthcare Research and Quality, 1-12.Google Scholar
- Trafton J, Martins S, Michel M, Lewis E, Johnson N, Combs A, Wang D, Tu S, Goldstein M: Evaluation of the acceptability of a decision support system to encourage safe and effective use of opioid therapy for chronic pain by primary care providers. Pain Medicine.Google Scholar
- Dick W, Carey L: The Systematic Design of Instruction. 1996, New York: Harper Collins, 4Google Scholar
- Martins S, Lai S, Tu S, Shankar R, Hastings S, Hoffman BN, Goldstein M: Offline Testing of the ATHENA Hypertension Decision Support System Knowledge Base to Improve the Accuracy of Recommendations. Proc AMIA Symp. 2006Google Scholar
- Gennari J, Musen M, Ferguson R, Grosso W, Crubezy M, Erikkson H, Noy N, Tu S: The Evolution of Protege: An Environment for Knowledge-Based Systems Development. Int J Hum Comput Stud. 2003, 58: 89-123. 10.1016/S1071-5819(02)00127-1.View ArticleGoogle Scholar
- Tierney WM, Overhage JM, Takesue BY, Harris LE, Murray MD, Vargo DL, McDonald CJ: Computerizing guidelines to improve care and patient outcomes: the example of heart failure. J Am Med Inform Assoc. 1995, 2: 316-322.View ArticlePubMedPubMed CentralGoogle Scholar
- Miller RA: Evaluating evaluations of medical diagnostic systems. J Am Med Inform Assoc. 1996, 3: 429-431.View ArticlePubMedPubMed CentralGoogle Scholar
- Shiffman RN, Michel G, Essaihi A, Thornquist E: Bridging the guideline implementation gap: a systematic, document-centered approach to guideline implementation. J Am Med Inform Assoc. 2004, 11: 418-426. 10.1197/jamia.M1444.View ArticlePubMedPubMed CentralGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.