- Open Access
- Open Peer Review
Toward a policy ecology of implementation of evidence-based practices in public mental health settings
Implementation Sciencevolume 3, Article number: 26 (2008)
Mental health policymaking to support the implementation of evidence-based practices (EBPs) largely has been directed toward clinicians. However, implementation is known to be dependent upon a broader ecology of service delivery. Hence, focusing exclusively on individual clinicians as targets of implementation is unlikely to result in sustainable and widespread implementation of EBPs.
Policymaking that is informed by the implementation literature requires that policymakers deploy strategies across multiple levels of the ecology of implementation. At the organizational level, policies are needed to resource the added marginal costs of EBPs, and to assist organizational learning by re-engineering continuing education units. At the payor and regulatory levels, policies are needed to creatively utilize contractual mechanisms, develop disease management programs and similar comprehensive care management approaches, carefully utilize provider and organizational profiling, and develop outcomes assessment. At the political level, legislation is required to promote mental health parity, reduce discrimination, and support loan forgiveness programs. Regulations are also needed to enhance consumer and family engagement in an EBP agenda. And at the social level, approaches to combat stigma are needed to ensure that individuals with mental health need access services.
The implementation literature suggests that a single policy decision, such as mandating a specific EBP, is unlikely to result in sustainable implementation. Policymaking that addresses in an integrated way the ecology of implementation at the levels of provider organizations, governmental regulatory agencies, and their surrounding political and societal milieu is required to successfully and sustainably implement EBPs over the long term.
Mental health policymaking in the past several decades has explicitly encouraged the adoption and implementation of specific evidence-based practices (EBPs). Purchasers (such as Medicaid agencies) and regulators (such as departments of mental health) have established lists of preferred therapies, have conducted provider profiling, and have provided training and technical assistance – all in an attempt to ensure that best available interventions are being delivered by clinicians to their clients. State agencies, therefore, largely seem to be taking a clinical approach to the implementation of EBPs. Conversely, the emerging literature within implementation science suggests that implementation requires a systemic, or ecological, approach . By ignoring this ecology, current policymaking to support the implementation of EBPs is itself not evidence-based.
In this article – directed toward policymakers and implementation researchers in public mental health settings – we argue that mandating the use of EBPs by individual clinicians and provider organizations, or narrowly focusing on effecting change within individual organizations, is unlikely to result in their successful and sustainable implementation unless the broader ecology within which these interventions are delivered is also supported. Following a brief overview of this implementation ecology, we present a framework to operationalize this ecology and illustrate it in Figure 1. We end by highlighting potential strategies at each level of this framework that policymakers can deploy in order to support implementation of EBPs and summarize these strategies in Table 1.
The march toward evidence
Quality improvement within mental health services has long been a goal of policy. Operationalization of quality improvement efforts has occurred largely at the level of individual clinicians and their clients through the development and deployment of specific interventions backed by research evidence, clinical judgment, and client preferences. Pioneered at McMaster University as 'evidence-based medicine' [2, 3], applications of this approach to mental health have resulted in various EBPs [4, 5]. These practices are often packaged with manuals and other materials suitable for demonstrating a particular practice to clinicians [6–9].
Policymaking directed toward mental health quality improvement evolved to support these clinical efforts. Government agencies have supported the development and use of clinical guidelines to standardize care [10, 11]. They also have released reports on various aspects of quality  and have supported the widespread dissemination and use of evidence-based mental health interventions . States have also incentivized (i.e., created a reward structure for using) EBPs – Oregon's passage of SB 267 in 2003 requiring the state to spend an increasing share of its budget in purchasing specified interventions is one example . States also have required the use of particular clinical protocols (e.g. Texas' emphasis on the use of medication algorithms) . The District of Columbia's Department of Mental Health has adopted a policy to support 'evidence-based psychotherapy,' which requires that all psychotherapy provided to clients in the District appear on a list maintained by the department . Today, the number of individuals receiving interventions that are evidence-based is one of the Substance Abuse and Mental Health Services Administration's (SAMHSA) National Outcome Measures . Efforts by governmental agencies to improve quality in Britain , Brazil , and Germany , to name a few examples, have also been largely focused at the clinical level.
The ecology of implementing EBPs
In FY 2005, SAMHSA began funding mental health transformation state incentive grants [20, 21]. In terms of policy, these grants marked a shift in thinking from incentivizing the development and deployment of specific interventions, to incentivizing the infrastructure necessary for appropriate service delivery. This shift in policy was to some degree influenced by the wealth of literature that has accumulated in recent years on institutional frameworks [22, 23] and organizational factors that can guide implementation efforts [24–26]. This literature suggests that, although the clinical encounter is where EBPs are delivered, efforts to promote the implementation of such practices should focus on the wider context of service delivery [24, 27, 28]. Collectively, this literature articulates an ecology of mental health intervention ranging from the clinical encounter to the social context of mental health service delivery. As articulated by Vijay Ganju :
This perspective goes beyond the adoption of the EBP by an individual practitioner or organization and includes the notion of broader, systemwide availability of EBPs and their integration into existing systems of care. ... The model that emerges related to the ultimate, broad based adoption depends on the nature of the EBP ..., the consumer ..., the practitioner ..., the organizational matrix within which the practitioner operates ... and the public mental health authority, or purchaser .... The implication is that each of these levels must be adequately addressed for sustained, systemwide uptake of an EBP.
Implementation spans the set of activities necessary to successfully and sustainably apply with high fidelity an intervention of known efficacy within community-based clinical settings. These activities are contextual, involving the organization within which services are delivered, the regulatory and funding environment operant upon the organization, a political milieu that supports mental health service delivery, and societal norms and subcultures that affect consumers' access to EBPs. Therefore, policymaking that is focused exclusively toward clinicians is unlikely to be sustainable; instead, policymakers need to align the effects of policy action across all of these contexts in order to produce 'sustained, systemwide uptake' of EBPs. Implementation researchers designing and conducting implementation studies even at the level of a single organization need to be cognizant of influences at multiple levels of the organization that can affect their chances of success. In the remainder of this article, we describe a policy ecology framework for EBP implementation, and identify policy levers (i.e. strategies that policymakers can deploy) at each of these contexts – other than at the level of the individual client/practitioner encounter – that, when addressed, can result in sustainable uptake of EBPs (Figure 1).
Policy levers at the organizational level
Organizations – ranging from small mental health practice associations to large, multidisciplinary mental health facilities – form the immediate context within which most clinicians deliver mental health interventions to consumers. Attempting to deliver new evidence-based interventions within organizations is associated with several challenges [7, 30]; however, once organizations determine that a particular function, such as delivering EBPs to a particular population, is part of their core mission, they tend to protect these technical functions from what Robert Rosenheck describes as the 'stormy sea of organizational process' . In addition to protecting core functions, some 'learning organizations'  actively seek out ways to improve their core missions , through promoting inquiry, connection, and opportunities to learn and grow . Policymakers can incentivize, and researchers can find ways to enhance, both of these organizational actions – protecting the delivery of EBPs, and actively engaging in quality improvement efforts – in two ways.
First, most EBPs are associated with higher marginal costs that need to be reimbursed. These costs are generated by providers, organizations, and state agencies, and several of these costs accrue to organizations. Examples include the costs of training in EBPs, costs of ongoing supervision and case consultation, productivity losses as novice clinicians gain mastery in new interventions, and the costs of documentation and regulatory compliance. Existing reimbursement strategies rarely cover these higher costs, which are currently borne largely by organizations. By developing enhanced reimbursement strategies  that cover the marginal costs of implementation, mental health purchasers can ensure a more sustainable organizational context for EBPs.
Second, much of organizational learning occurs through continuing education and related professional development activities by its licensed professionals. Regulations surrounding mandated continuing education units (CEUs) offer policymakers the ability to shape professional practice toward EBPs. State licensing board regulators, or their interagency partners, can assume all costs of, or subsidize, certain CEUs, provide direct technical assistance in developing courses and programs, or disallow certain courses for licensing credit. However, in order to promote an EBP environment, licensing boards will need to reconsider the structure of the CEU. Because single-shot training and didactic approaches are usually ineffective in shaping provider behavior, licensing boards will need to support quality improvement approaches that are rooted in the literature on provider behavioral change [35–38]. This literature suggests that provider education needs to be combined with auditing and feedback, as well as reminder systems and real-time decision support in order to be truly effective . Re-engineering CEUs can help address an issue raised in a recent report issued by a National Institute of Mental Health (NIMH) workgroup, which noted a dearth of professionals adequately trained to provide EBPs .
Policy levers at the regulatory and purchasing agency level
Regulatory and purchasing agencies form the immediate policy context for organizational activity in mental health, and have a long history of quality improvement. First-generation efforts undertaken by purchasers focused largely on profiling providers and hospitals. For example, Congress required the establishment of the National Provider Data Bank (which assists credential review of providers by state licensing boards, hospitals, and other health care entities) , and over 30 states make available physician profiles to the public . Second-generation efforts directed at quality improvement largely involved managing quantity. For example, the rise of Medicaid managed care saw the increasing use of utilization review (or management) covering a variety of inpatient and ambulatory services .
Third-generation efforts, currently underway in almost all states, are designed to enhance the appropriateness of care by monitoring the type of care that is delivered, and can serve as models to support the implementation of EBPs. States can influence the type of care they pay for by using contractual requirements during the bidding process for purchase of services. In general, states tend to use five principal types of fiscal incentives – pay-for-performance or other payment incentive mechanisms, reduction in oversight and other regulatory requirements, fast-tracking or providing other advantages in the competitive bidding process, paying for infrastructure (such as free training in EBPs), and some sort of public recognition or award for providing EBPs . The Ohio Departments of Mental Health and Alcohol and Drug Addiction Services have established coordinating centers to provide training, supervision, consultation, and other types of information sharing to support implementation of EBPs . How a state structures its contracts to purchase EBPs is likely to be highly idiosyncratic, requiring a mix of financing and regulatory change, addressing issues of leadership and organizational politics, and ensuring training and data management efforts .
Second, some states have undertaken quality improvement within a disease management framework, defined as '... a system of coordinated healthcare interventions and communications for populations with conditions in which patient self-care efforts are substantial' . While the specific components of a disease management program vary, common elements of all such programs include a systematic way of identifying patients; matching the intervention with their needs; ensuring the availability of EBP guidelines for not only physicians but also for all other providers involved in the care of the disease; developing an individualized treatment plan for the unique needs of the patient; designing services that promote patient adherence to this individualized treatment plan through such mechanisms as patient education, monitoring and reminders, and behavior modification programs; systematically collecting process and outcome measures; and providing a way for physicians, other providers, and the patient to obtain ongoing feedback on how care is progressing and what outcomes are being met . Successful implementation of disease management programs in commercial health plans led to the establishment of the first Medicaid disease management program – Virginia's program for asthma – in the early 1990s . Florida established the nation's first disease management program for a mental disorder (major depression), and the Centers for Medicare and Medicaid Services (CMS) have funded a national disease management demonstration project . Disease management programs in public mental health settings are most likely to be successful for conditions that are stable over time, that can be reliably identified through screening instruments, that have well-developed and tested interventions suitable for implementation, and that require a comprehensive array of services. Conditions such as childhood trauma, for example, offer an opportunity for the construction of disease management programs, which can serve as a vehicle for the implementation of EBPs for this condition [49, 50].
Third, prior authorization – the requirement that providers obtain approval for the use of a particular intervention or drug – is an existing approach used by over 30 states . Originally developed as a cost-containment measure to control pharmacy costs, prior authorization is often combined with formularies to restrict the variety of medications available to beneficiaries. Although little experience exists with prior authorization for behavioral interventions, such programs could be used to restrict potentially harmful interventions being delivered to enrollees. Conversely, eliminating evidence-based interventions from restrictions on session limits, modifying designated patient regulations that restrict who can receive the service, and modifying existing regulations governing session lengths can all serve to promote EBPs using this approach.
Fourth, many states have experience with provider profiling, which can be used to promote an EBP agenda. Attempts to improve the appropriateness of psychotropic medication use, for example, have seen the use of provider-level audits of prescriptions, a practice called prescriber profiling. While attempts are underway to reduce sharing of prescriber data with pharmaceutical companies , state purchasers can, and do, access prescriber-level records to identify individuals engaging in aggressive pharmacotherapy, and to monitor compliance with established medication algorithms . However, few states seem to have implemented 'psychotherapy profiling' systems, which may allow therapists with specialized training to be identified and reimbursed at a higher rate. Psychotherapy profiling will require states to develop specialized billing codes that would permit comprehensive assessments [54, 55], which could guide deployment of EBPs. States may also need to modify their billing requirements to accommodate EBPs that differ structurally from individual or group therapy.
Finally, states will need to find a way to link all such procedural efforts with individual client-level outcomes. Outcome measures in mental health are difficult and expensive to administer, which is why most quality improvement efforts focus on performance or process indices. However, federal policymaking seems to be moving toward requiring comprehensive outcome measures as a condition of payment – such as the requirement that all home health agencies seeking Medicare certification report on a set of common measures contained in the Outcome and Assessment Information Set (OASIS) . SAMHSA's national outcome measures (NOMs) also outline a series of client-level as well as systemic outcomes across ten domains , although there is not yet a link between purchasing and attainment of these measures. To date, few states have established mechanisms to collect data on performance indicators statewide , and none have established statewide client-level outcomes monitoring. Developing or adopting outcomes assessment, either statewide or within a given system (e.g. children's mental health) is a necessary first step for any outcomes-based reimbursement approach as a tool to support EBP implementation.
Policy levers at the political level
We define the political context of EBP implementation as involving all legislative and advocacy efforts that support such a goal. While few laws are directed specifically at implementation efforts, legislation often forms the enabling resource for EBP implementation. For example, following Oregon's 2003 legislation (discussed earlier), the Iowa legislature passed in 2004 a law that extended an EBP mandate beyond state agencies, requiring community mental health centers to spend an increasing share of Mental Health Block Grant dollars in purchasing EBPs . Policymakers will need to carefully weigh the nature of the evidence, the availability of local resources to deliver the EBP with fidelity, and the unintended consequences of micromanaging care before considering such legislative strategies.
While much of the focus of EBP activists has been on such targeted laws, other laws, such as mental health parity laws, also require attention by state policymakers. These laws have a broader objective – ensuring that mental health services can be adequately resourced and delivered – which is a necessary requisite for providing EBPs. The issue of mental health parity extends far beyond providing necessary resources, however, in being intimately related to social-level issues of stigma, misunderstanding, and discrimination toward people with mental health diagnoses . Parity legislation, therefore, is necessary but insufficient to drive implementation efforts due to these societal factors, which we consider in greater detail below.
Third, another way in which state legislatures can incentivize EBP implementation is by passing laws that reduce or forgive educational debts for professionals who adopt and promote EBPs. Debt forgiveness is currently used to promote health sciences, counseling, and social work practice in high-need areas with particularly vulnerable populations, based on federal legislation [59–61]. By re-conceptualizing an underserved area as extending beyond geography to types of practice, legislation could provide forgiveness of educational debt to clinicians delivering EBPs to defined populations, and make EBP-trained clinicians more financially attractive to employers.
Fourth, policymakers can begin to reduce or eliminate structural stigma. Structural stigma refers to state policies or legislation that deliberately deprive certain groups of individuals (in this case, the mentally ill) from the rights and privileges that accrue to other groups. A 1999 survey of state laws revealed that 44 states imposed some restrictions on the rights of mentally ill individuals to serve on a jury, 37 imposed restrictions on their voting, 23 imposed restrictions on their holding elective office, and 27 imposed restrictions on their parental rights . An analysis of the 968 mental health bills introduced into legislatures nationwide in calendar year 2002 revealed that although most states were legislating to protect the rights of mentally ill individuals, some states were expanding restrictions of parental rights for the mentally ill . While we are unaware of examples of structural stigma that affect EBPs, legislators will need to pay close attention to the effects of their EBP-related lawmaking in order to avoid discriminating against mental health consumers.
Fifth, policymakers who look to support for an EBP agenda from consumer-focused advocacy groups will find that these groups have not been universally supportive of the emergence of EBPs. On one hand, there is an appreciation of a model of practice built upon documented results rather than on the opinions of experts , and organizations such as the National Alliance for the Mentally Ill have supported some EBP implementation efforts . Professional organizations are generally strong supporters of EBPs, have testified before Congress , and have set up task forces to promote such practices , among other efforts. Conversely, consumer-directed advocacy groups, such as the National Mental Health Consumer's Association, have advanced equivocal positions . Several family/consumer fears have been documented in the literature , and include concerns that the EBP movement is insufficiently aligned with the recovery model (in which mental health consumers are presumed capable of making considerable progress toward independence ), that EBPs may replace other needed services, that there may be a lack of availability of providers able to deliver EBPs, that EBPs may be unduly prescriptive and cause consumers to lose control over their care, and that they may not be sufficiently culturally competent [6, 69].
Policymakers who confront such issues may benefit from an approach to engaging consumers and families proposed by Birkel and colleagues , who suggest that EBP implementation efforts should actively build collaborative relationships at the beginning of the implementation process; should find ways to integrate recovery in the development and deployment of EBPs; should pay special attention to racial/ethnic, geographical, cultural and linguistic diversity in all implementation efforts; and should develop and disseminate resources to support not just the EBP but also its advocacy. Such an inclusive approach to policymaking that takes into account diverse consumer and family needs may be necessary to assure widespread acceptability of EBPs.
Policy levers at the social level
Several efforts discussed above, including those focused on consumer advocacy, cultural competence, and consensus-building, lie at the interface between political and social contexts of EBP implementation. In this section, we focus on how policymakers can mitigate the effects of stigma in preventing access to EBPs.
Combating mental health-related stigma is a goal of the President's New Freedom Commission , and is essential because access to EBPs is conditional upon access to mental health services. Several EBPs also require greater amounts of adherence to protocols, which can be compromised in the presence of stigma. However, empirical guidance for policymakers on ways to reduce stigma is still emerging. The NIMH established a Stigma Working Group in 1999, and has issued a program announcement to fund research projects on stigma reduction . SAMHSA established its Eliminating Barriers Initiative in 2003, and funds a Resource Center to Address Discrimination and Stigma . These efforts aim to identify effective approaches to reduce stigma and discrimination, such as those involving public educational programming, producing educational materials, contact-based approaches, and public service announcements. Other efforts at ending stigma use social marketing principles. For example, the World Health Organization's 2001 World Health Day launched an international campaign to reduce stigma associated with mental illnesses among youth, which included organization of school contests addressing stigma, development of teacher guides, production of a book on stigma through children's eyes, and a curriculum for use in health education programs worldwide [74, 75].
While reducing stigma may be an educational endeavor, eliminating discrimination is usually a legal process. Policy approaches that can serve more proximal goals of ending discrimination include ensuring appropriate treatment and care for individuals with mental illnesses, supporting effective public and professional education, and preserving mental health allotments in health, welfare, and research budgets . While none of these approaches directly serve an EBP implementation agenda, they create the contexts for improved access to mental health services, within which an EBP agenda can be supported.
Efforts to improve the quality of mental health services should consider the larger ecology that is known to affect implementation instead of solely focusing on specific interventions and the specific locations of their delivery. An integrated approach to policymaking at several levels of this ecology – as summarized in Table 1 – can support a more sustainable, and ultimately more successful, implementation process. In addition, implementation researchers need to be aware of influences at multiple levels of this ecology; absent a conducive environment, gains from even the best-designed approaches targeted solely at individual providers or organizations are unlikely to persist over the long term. Implementation researchers will also need to build in the systematic collection of data at multiple levels of the implementation ecology while designing their studies in order to identify and test change strategies that are likely to succeed. Practice leaders, such as executive directors of mental health organizations, are usually highly attuned to the environment within which their organizations operate, and will be important sources of information on, and change agents in, this ecological approach to implementation. We recommend that all individuals involved with implementation efforts consider these strategies as they collectively strive to increase the availability of EBPs to vulnerable populations.
Ganju V: Implementation of evidence-based practices in state mental health systems: implications for research and effectiveness studies. Schizophrenia bulletin. 2003, 29: 125-131.
Evidence-Based Medicine Working Group: Evidence-Based Medicine. A New Approach to Teaching the Practice of Medicine. JAMA. 1992, 268: 2420-2425. 10.1001/jama.268.17.2420.
Guyatt GH: Evidence-Based Medicine. Annals of Internal Medicine. 1991, 114: A-16.
Hoagwood K, Burns BJ, Kiser L, Ringeisen H, Schoenwald SK: Evidence-Based Practice in Child and Adolescent Mental Health Services. Psychiatric Services. 2001, 52: 1179-1189. 10.1176/appi.ps.52.9.1179.
Tanenbaum S: Evidence-Based Practice in Mental Health: Practical Weaknesses Meet Political Strengths. Journal of Evaluation in Clinical Practice. 2003, 9: 287-301. 10.1046/j.1365-2753.2003.00409.x.
Drake RE, Goldman HH, Leff HS, Lehman AF, Dixon L, Mueser KT, Torrey WC: Implementing Evidence-Based Practices in Routine Mental Health Service Settings. Psychiatric Services. 2001, 52: 179-182. 10.1176/appi.ps.52.2.179.
Ganju V: Implementation of Evidence-Based Practices in State Mental Health Systems: Implications for Research and Effectiveness Studies. Schizophrenia Bulletin. 2003, 29: 125-131.
U.S. Department of Health and Human Services Substance Abuse and Mental Health Services Administration: About Evidence-Based Practices: Shaping Mental Health Services Towards Recovery. 2005, Rockville, MD: USDHHS, Substance Abuse and Mental Health Services Administration
McBroom K: The Development of an Evidence-based Toolkit to Prevent Meningococcal Disease in College Students. Family & Community Health. 2007, 30: 93-111.
U.S. Department of Health and Human Services Agency for Health Care Policy and Research: Using Clinical Practice Guidelines to Evaluate Quality of Care. 1995, Rockville, MD: U.S. Department of Health and Human Services
Huttin C: The Use of Clinical Guidelines to Improve Medical Practice: Main Issues in the United States. Int J Qual Health Care. 1997, 9 (3): 207-214. 10.1093/intqhc/9.3.207.
U.S. Department of Health and Human Services Agency for Healthcare Research and Quality: 2006 National Healthcare Quality Report. 2006, Rockville, MD: U.S. Department of Health and Human Services
Oregon Office of Mental Health and Addiction Services: Project Plan to Promote the Adoption of Evidence-Based Practices. 2004
Texas Medication Algorithm Project. [http://www.dshs.state.tx.us/mhprograms/TMAP.shtm]
Tanenbaum SJ: Evidence-Based Practice as Mental Health Policy: Three Controversies and a Caveat. Health Affairs. 2005, 24: 163-173. 10.1377/hlthaff.24.1.163.
U.S. Department of Health and Human Services Substance Abuse and Mental Health Services Administration: National Outcome Measures. 2007, Rockville, MD: USDHHS, Substance Abuse and Mental Health Services Administration
Ferlie EB, Shortell SM: Improving the Quality of Health Care in the United Kingdom and the United States: A Framework for Change. Milbank Quarterly. 2001, 79: 281-315. 10.1111/1468-0009.00206.
Malik AM: Quality Improvement Issues in Brazil. Joint Commission Journal on Quality and Safety. 1997, 23: 55-59.
Paeger A: Quality Improvement in Germany. Joint Commission Journal on Quality and Safety. 1997, 23: 38-46.
U.S. Department of Health and Human Services Substance Abuse and Mental Health Services Administration: FY 2005 SAMHSA Grant Awardees – CMHS. 2005, Rockville, MD: USDHHS, Substance Abuse and Mental Health Services Administration
U.S. Department of Health and Human Services Substance Abuse and Mental Health Services Administration: FY 2006 SAMHSA Grant Awardees. 2006, Rockville, MD: USDHHS, Substance Abuse and Mental Health Services Administration
Schoenwald SK, Hoagwood K: Effectiveness, Transportability, and Dissemination of Interventions: What Matters When?. Psychiatric Services. 2001, 52: 1190-1197. 10.1176/appi.ps.52.9.1190.
Schoenwald SK, Sheidow AJ, Letourneau EJ, Liao JG: Transportability of Multisystemic Therapy: Evidence for Multilevel Influences. Mental Health Services Research. 2003, 5: 223-239. 10.1023/A:1026229102151.
Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F: Implementation Research: A Synthesis of the Literature. 2005, Tampa, FL: National Implementation Research Network, Louis de la Parte Florida Mental Health Institute, University of South Florida
Glisson C: The Organizational Context of Children's Mental Health Services. Clinical Child and Family Psychology Review. 2002, 5: 233-253. 10.1023/A:1020972906177.
Glisson C, Green P: The Effects of Organizational Culture and Climate on the Access to Mental Health Care in Child Welfare and Juvenile Justice Systems. Administration and Policy in Mental Health and Mental Health Services Research. 2006, 33: 433-448. 10.1007/s10488-005-0016-0.
Pincus HA, Hough L, Houtsinger JK, Rollman BL, Frank RG: Emerging models of depression care: multi-level ('6 P') strategies. International Journal of Methods in Psychiatric Research. 2003, 12: 54-63. 10.1002/mpr.142.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations. The Milbank Quarterly. 2004, 82: 581-629. 10.1111/j.0887-378X.2004.00325.x.
Ganju V: Mental Health Quality and Accountability: The Role of Evidence-Based Practices and Performance Measurement. Administration and Policy in Mental Health and Mental Health Services Research. 2006, 33: 659-665. 10.1007/s10488-006-0071-1.
Rosenheck RA: Organizational Process: A Missing Link between Research and Practice. Psychiatric Services. 2001, 52: 1607-1612. 10.1176/appi.ps.52.12.1607.
Senge PM: The Fifth Discipline: The Art and Practice of the Learning Organization. 1990, New York: Doubleday/Currency
Birleson P, Brann P: Reviewing the learning organisation model in a child and adolescent mental health service. Australian Health Review. 2006, 30: 181-194.
Noble J: Develop a learning organisation. Education for Primary Care. 2007, 18: 220-222.
Ogrod ESI, Doherty RB: Enhanced Reimbursement is Becoming a Reality. Consultant. 1988, 28: 89-90. 93–95
Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O'Brien MA: Changing Provider Behavior: An Overview of Systematic Reviews of Interventions. Medical Care. 2001, 39: I12-45. 10.1097/00005650-200108002-00002.
Mazmanian P: Exploring Relationships of Knowledge to Action. Journal of Continuing Education in the Health Professions. 2006, 26: 98-10.1002/chp.56.
Mazmanian PE, Davis DA: Continuing Medical Education and the Physician as a Learner: Guide to the Evidence. JAMA. 2002, 288: 1057-1060. 10.1001/jama.288.9.1057.
Shojania KG, Grimshaw JM: Evidence-based Quality Improvement: The State of the Science. Health Affairs. 2005, 24: 138-150. 10.1377/hlthaff.24.1.138.
National Advisory Mental Health Council's Workgroup on Services and Clinical Epidemiology Research: The Road Ahead: Research Partnerships to Transform Services. 2006, Bethesda, MD: U.S. Department of Health and Human Services, National Institutes of Health
Fact Sheet on the National Practitioner Data Bank. [http://www.npdb-hipdb.hrsa.gov/pubs/fs/Fact_Sheet-National_Practitioner_Data_Bank.pdf]
Sutton JH: Physician Data Profiling Proliferates. Bulletin of the American College of Surgeons. 2001, 86: 20-24.
Buck JA, Silverman HA: Use of Utilization Management Methods in State Medicaid Programs. Health Care Financing Review. 1996, 17: 77-86.
Necochea L: Identifying State Purchasing Levers for Promoting the Use of Evidence-Based Practice in Substance Abuse Treatment (RWJF Research Meeting Highlight). 2007, Washington, DC: Robert Wood Johnson Foundation
Biegel DE, Kola LA, Ronis RJ, Boyle PE, Delos Reyes CM, Wieder B, Kubek P: The Ohio Substance Abuse and Mental Illness Coordinating Center of Excellence: Implementation Support for Evidence-Based Practice. Research on Social Work Practice. 2003, 13: 531-545. 10.1177/1049731503013004007.
Isett KR, Burnam MA, Coleman-Beattie B, Hyde PS, Morrissey JP, Magnabosco J, Rapp CA, Ganju V, Goldman HH: The State Policy Context of Implementation Issues for Evidence-Based Practices in Mental Health. Psychiatric Services. 2007, 58: 914-921. 10.1176/appi.ps.58.7.914.
Gillespie JL, Rossiter LF: Medicaid Disease Management Programs: Findings from Three Leading US State Programs. Disease Management & Health Outcomes. 2003, 11: 345-361. 10.2165/00115677-200311060-00001.
Todd WE, Nash D: Disease management: A systems approach to improving patient outcomes. 1997, San Francisco, CA: Jossey-Bass
Brown R, Chen A: Disease Management Options: Issues for State Medicaid Programs to Consider (Issue Brief). 2004, Washington, DC: Mathematica Policy Research
Cohen JA, Mannarino AP, Deblinger E: Child and Parent Trauma-Focused Cognitive Behavioral Therapy Treatment Manual. 2006, Philadelphia, PA: Drexel University College of Medicine
Cohen JA, Mannarino AP, Deblinger E: Treating Trauma and Traumatic Grief in Children and Adolescents. 2006, New York: Guilford Press
Tilly J, Elam L: Prior Authorization for Medicaid Prescription Drugs in Five States: Lessons for Policy Makers. 2003, Washington, DC: Kaiser Family Foundation
Greene JA: Pharmaceutical Marketing Research and the Prescribing Physician. Annals of Internal Medicine. 2007, 146: 742-748.
Ning A, Dubin WR, Parks JJ: State Mental Health Policy: Pharmacy Costs: Finding a Role for Quality. Psychiatric Services. 2005, 56: 909-911. 10.1176/appi.ps.56.8.909.
Goldberg RJ: Billing for Psychiatric Evaluations: Options for Coding and Reimbursement. General Hospital Psychiatry. 2004, 26: 296-301. 10.1016/j.genhosppsych.2004.02.005.
Worley LLM, Kunkel EJS, Hilty DM: How C-L Services Can Comply with New HCFA Guidelines. General Hospital Psychiatry. 1998, 20: 160-169. 10.1016/S0163-8343(98)00011-5.
Iowa Consortium for Mental Health: Legislation Update. 2006
Hinshaw SP: The mark of shame: Stigma of mental illness and an agenda for change. 2007, New York: Oxford University Press
Student Loan Assistance Programs for Counselors. [http://www.counseling.org/]
Loan Forgiveness Programs Available to Social Workers. [http://www.socialworkers.org/advocacy/updates/2007/072707.asp]
Zimmerman K: Working in an Undeserved Area?. American Journal of Nursing. 2001, 101: 24-
Hemmens C, Miller M, Burton VS, Milner S: The Consequences of Official Labels: An Examination of the Rights Lost by the Mentally Ill and Mentally Incompetent Ten Years Later. Community Mental Health Journal. 2002, 38: 129-140. 10.1023/A:1014543104471.
Corrigan PW, Watson AC, Heyrman ML, Warpinski A, Gracia G, Slopen N, Hall LL: Structural Stigma in State Legislation. Psychiatric Services. 2005, 56: 557-563. 10.1176/appi.ps.56.5.557.
Youngclarke D, Ramos K: Evidence-Based Practice: What role should research play in our decision making?. California Court Appointed Special Advocate Association. 2006
Frese FJ, Stanley J, Kress K, Vogel-Scibilia S: Integrating Evidence-Based Practices and the Recovery Model. Psychiatric Services. 2001, 52: 1462-1468. 10.1176/appi.ps.52.11.1462.
Testimony: FY 2003 Appropriations for Mental Health Research and Services, Subcommittee on Labor, Health and Human Services and Education of the House Committee on Appropriates. [http://www.aagpgpa.org/advocacy/testimony.asp?viewfull=7]
American Psychological Association: Report of the 2005 Presidential Task Force on Evidence-Based Practice. 2005
Ramon S, Healy B, Renouf N: Recovery from mental illness as an emergent concept and practice in Australia and the UK. International Journal of Social Psychiatry. 2007, 53: 108-122. 10.1177/0020764006075018.
Scheyett A, McCarthy E, Rausch C: Consumer and Family Views on Evidence-Based Practices and Adult Mental Health Services. Community Mental Health Journal. 2006, 42: 243-257. 10.1007/s10597-005-9027-2.
Birkel RC, Hall LL, Lane T, Cohan K, Miller J: Consumers and Families as Partners in Implementing Evidence-Based Practice. Psychiatric Clinics of North America. 2003, 26: 867-881. 10.1016/S0193-953X(03)00065-0.
New Freedom Commission on Mental Health: Achieving the Promise: Transforming Mental Health Care in America. Final Report. 2003, Rockville, MD
U.S. Department of Health and Human Services National Institutes of Health: Reducing Mental Illness Stigma and Discrimination PAR-07-156. 2006, Bethesda, MD: U.S. Department of Health and Human Services, National Institutes of Health
ADS Center: Resource Center to Address Discrimination and Stigma. [http://www.adscenter.org]
Jenkins R: World Health Day 2001: Minding the World's Mental Health. Social Psychiatry and Psychiatric Epidemiology. 2001, 36: 165-168. 10.1007/s001270170059.
World Health Organization: Child and Adolescent Mental Health Initiatives of the Department of Mental Health and Substance Abuse. 2003
Graham N, Lindesay J, Bertolote CKJM, Camus V, Copeland JRM, Lima CAdM, Gaillard M, Nargeot MCG, Gray J, Jacobsson L: Reducing stigma and discrimination against older people with mental disorders: A technical consensus statement. International Journal of Geriatric Psychiatry. 2003, 18: 670-678. 10.1002/gps.876.
Dr. Raghavan is an investigator with the Center for Mental Health Services Research, at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (5P30 MH068579).
The authors declare that they have no competing interests.
RR conceived this manuscript, and led the writing. CLB assisted with the literature review, and wrote parts of this manuscript. ALS participated in the conceptualization and writing of this manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.