- Open Access
- Open Peer Review
Factors associated with the impact of quality improvement collaboratives in mental healthcare: An exploratory study
© Versteeg et al; licensee BioMed Central Ltd. 2012
- Received: 7 November 2010
- Accepted: 9 January 2012
- Published: 9 January 2012
Quality improvement collaboratives (QICs) bring together groups of healthcare professionals to work in a structured manner to improve the quality of healthcare delivery within particular domains. We explored which characteristics of the composition, participation, functioning, and organization of these collaboratives related to changes in the healthcare for patients with anxiety disorders, dual diagnosis, or schizophrenia.
We studied three QICs involving 29 quality improvement (QI) teams representing a number of mental healthcare organizations in the Netherlands. The aims of the three QICs were the implementation of multidisciplinary practice guidelines in the domains of anxiety disorders, dual diagnosis, and schizophrenia, respectively. We used eight performance indicators to assess the impact of the QI teams on self-reported patient outcomes and process of care outcomes for 1,346 patients. The QI team members completed a questionnaire on the characteristics of the composition, participation in a national program, functioning, and organizational context for their teams. It was expected that an association would be found between these team characteristics and the quality of care for patients with anxiety disorders, dual diagnosis, and schizophrenia.
No consistent patterns of association emerged. Theory-based factors did not perform better than practice-based factors. However, QI teams that received support from their management and both active and inspirational team leadership showed better results. Rather surprisingly, a lower average level of education among the team members was associated with better results, although less consistently than the management and leadership characteristics. Team views with regard to the QI goals of the team and attitudes towards multidisciplinary practice guidelines did not correlate with team success.
No general conclusions about the impact of the characteristics of QI teams on the quality of healthcare can be drawn, but support of the management and active, inspirational team leadership appear to be important. Not only patient outcomes but also the performance indicators of monitoring and screening/assessment showed improvement in many but not all of the QI teams with such characteristics. More studies are needed to identify factors associated with the impact of multidisciplinary practice guidelines in mental healthcare.
- Anxiety Disorder
- Performance Indicator
- Organizational Context
- Mental Healthcare
Healthcare providers worldwide are looking to improve the quality of healthcare delivery. Quality improvement collaboratives (QICs) are currently being established on a widespread basis for this purpose. QICs bring together groups of healthcare providers to work in a structured manner to improve the quality of healthcare delivery in a specific domain. Ideal domains for QICs involve interventions derived from evidence-based guidelines or, when these are not available, other best-practice information .
There is considerable variation in how QICs are structured and run . A well-known approach consists of the adoption of the Breakthrough Series (BTS) developed and promoted by the Institute for Healthcare Improvement in the United States [3, 4]. Although QICs vary in their approach, generally five essential features are involved. There is: a specific topic; clinical and quality improvement experts provide ideas and support for improvement; multi-professional teams from multiple organizations participate; a specific model is used for improvement (e.g., set targets, collect data, test for change) ; and a series of structured collaborative activities is undertaken (e.g., national conferences) .
The establishment of QICs is a popular strategy with positive but limited evidence regarding its effectiveness [6, 7]. The domain of the QIC, the methods used, the organizational context, and the general healthcare setting can all influence the effectiveness of a particular QIC [1, 2, 8, 9]. In addition, the effects of QICs in mental healthcare settings are largely unknown. Insight into the QI process in the field of mental health care is therefore needed to identify which factors promote the success of QICs and thereby improved mental healthcare [6, 7].
In the present study, we evaluated the impact of QICs that are part of a nationwide program to improve mental health care in the Netherlands. The domains of the QICs were the implementation of evidence-based multidisciplinary practice guidelines for anxiety disorders, dual diagnosis, and schizophrenia [10–12]. We also examined if such factors as the composition, participation, functioning, and organizational context for QI teams related to reported quality of healthcare for patients with anxiety disorders, dual diagnosis, or schizophrenia [1, 2, 9, 13–26].
Setting and study population
In 2004, QICs were introduced into mental healthcare in the Netherlands. The domains of depression and schizophrenia were targeted . Given positive evaluations, a nationwide QI program was introduced in the field of mental healthcare in the Netherlands between 2006 and 2009. At the same time a study was designed to evaluate the outcomes of the program.
In this one-year study, three QICs involving a total of 29 QI teams from different mental healthcare organizations—including eight primary healthcare organizations—in the Netherlands were evaluated. Participation in the QICs was voluntary. The participating organizations paid a fee for external support.
The QI teams involved 5 to 13 people, including the team leader. They targeted improved implementation of one of the three multidisciplinary practice guidelines. The QI teams consisted of psychiatrists or general practitioners (only in the domain of anxiety disorders), psychologists or psychotherapists, social workers, physiotherapists, social psychiatric nurses or mental health nurses, and practice assistants.
Aims of the QICs and national QI program
In promoting the implementation of multidisciplinary practice guidelines, the QICs specifically aimed to improve the delivery of care, monitoring of clinical status, and—in the end—health outcomes. The interventions and instruments used to do this were derived from the evidence reported in the multidisciplinary practice guidelines.
As part of the national QI program, each QIC had one multidisciplinary practice guideline as its topic. This meant that the QI teams in the QIC for anxiety disorders, for example, implemented a stepped care model to achieve better screening and treatment for patients with anxiety disorders. A BTS approach was adopted to do this.
Performance indicators as predefined by clinical experts with the standard QI teams had to meet at the end of the study
After six months of treatment, 50% of the patients with severe anxiety disorders have a score of less than three on the Clinical Global Impression- Severity (CGI-S) scale.
Experienced quality of life, as measured by the Manchester Short Assessment of Quality of Life (MANSA) scale, improved by 15 points for 75% of patients with a dual diagnosis.
Social functioning, as measured by the Health of the Nation Outcome Scale (HONOS), improved by 2 points for 100% of patients with schizophrenia.
At least 80% of the patients with anxiety disorders are monitored (every six weeks from the start of study and at least two times) using the Clinical Global Impression- Severity (CGI-S) scale.
The quality of life of 100% of the patients with a dual diagnosis is monitored (every 12 weeks and at least two times during the study) using the Manchester Short Assessment of Quality of Life (MANSA) scale.
The social functioning of 100% of the patients with schizophrenia is monitored at least two times using the Health of the Nation Outcome Scale (HONOS).
Every month at least 4 out of 1000 patients suspected of anxiety disorders, are screened using the Four Dimensional Symptom Questionnaire (4DSQ).
Of all patients with dual diagnosis, 100% are screened for psychopathology with the Health of the Nation Outcome Scale (HONOS).
During a period of one year, clinical experts and quality improvement experts provided QI support via team visits, conferences, and an internet forum. At least four conferences were organized to provide explicit instruction on techniques for QI and the measurement of QI. The conferences also gave the QI teams an opportunity to exchange knowledge and experiences.
Each QI team collected information on patient outcomes (see Data Analysis) and such process of care outcomes as monitoring and screening/assessment (see Data Analysis) in order to determine the impact of their guideline implementation. The QI teams used standardized spreadsheets to register individual patient data across a one-year period. The number of measuring occasions needed to calculate a performance indicator varied. For example, the performance indicator for screening/assessment of patients with dual diagnosis could be calculated on the basis of a single measurement occasion per patient. In contrast, patient outcomes and monitoring rates required at least two measurement occasions per patient. All of the QI teams worked to improve patient outcomes and monitoring rates. The teams in the domains of anxiety disorders and dual diagnosis also worked to improve screening/assessment rates (Table 1).
Factors hypothesized (i.e., average per QI team) to be associated with successful team performance
QI teams will perform better if they:
are more professional; have a higher level of education, more specialized training, more years of practice on the current job and more years of working in the present organization .
spend more time on the improvement .
have prior experience with quality improvement .
have a higher attendance of recommended national conferences .
have more positive attitudes towards the method for improvement (BTS approach), more positive opinions about how their contribution to the improvement is valued by other team members, and more positive opinions about the team's improvement efficacy .
have a more positive attitude towards change .
have a positive team climate; a positive view about communication with regard to the innovation; agree on the goals of the QIC; and have a positive opinion with regard to the openness of the working method used [19–22].
have a more positive attitude towards evidence-based practice guidelines .
Organizational context characteristics
receive support from the management of the organization .
Composition of the QI team
We collected data on personal characteristics like age  and profession. The QI team members were asked about their professional experience in terms of level of education [13, 14] and specialized knowledge . These were measured using a nominal scale. Education was divided into: associate degree (general psychiatric nurses, practice assistants), bachelor degree (specialized psychiatric nurses, social workers, physiotherapists), or master's degree (psychiatrists, general practitioners, psychologists, psychotherapists). We also collected data on years of experience with current job and years of employment in present organization . These were continuous variables. The QI team members were further asked about the time spent on improvement (continuous variable) . We measured prior experience with quality improvement  by asking QI team members dichotomous questions (yes/no) about: their involvement in other quality improvement activities; their experience with the organization of refresher courses; the development of guidelines; and in the organization of activities for their professional group.
Participation of the QI team in the national program
To measure participation of the QI teams in the national conferences, they were asked to indicate how many conferences they had attended . Conference attendance by team leaders versus team members was distinguished because the leaders were expected to attend more conferences.
Functioning of the QI team
The Attitude Social influence Efficacy model (ASE model)  was used to examine the perceptions and opinions of the QI team members with regard to their social influence, efficacy and the BTS approach. The Evidence-based Practice Attitude Scale (EBPA)  was used to assess the attitudes of the QI team members towards changes in their professional routines. The answers on a five-point Likert scale could range from strongly disagree (= 1) to strongly agree (= 5).
To assess team climate (TCI) [19–22], the QI team members were asked about: communication and readiness for innovation within the team; their views on the goals of the QI team; and their use of the team's approach and/or working method. Response options varied from strongly disagree (= 1) to strongly agree (= 5) for the team's communication and goals; from sometimes (= 1) to often (= 5) for the team's approach or working methods.
Finally, the QI team members were asked about their attitudes towards the evidence-based practice guideline targeted by them, in particular the openness of the guideline to interpretation . The responses on this five-point Likert scale could range from strongly disagree (= 1) to strongly agree (= 5).
Organizational context of the QI team
Perceptions of the organizational conditions for the QI teams were probed and, in particular, perceptions of the presence of organizational conditions needed for healthcare improvement efforts like time, workforce, sponsoring, and skills [1, 2, 9]. In addition, we asked if the QI team members experienced support from their managements . A seven-point Likert scale was used, which meant that responding could range from strongly disagree (= 1) to strongly agree (= 7).
Finally, we asked the QI team members about the type of leadership they received from the QI team leader. Did they perceive the leader as inspirational? transformational? passive? The responses on the Multi Factorial Leadership Questionnaire (MFLQ) [24, 26] could vary from sometimes (= 1) to often (= 5).
For the QIC in the domain of anxiety disorders and the QIC in the domain of dual diagnosis, three performance indicators were selected to assess the impact of improvement efforts: one for patient outcomes, one for monitoring, and one for screening/assessment. For the QIC in the domain of schizophrenia, two performance indicators were selected: one for patient outcomes and one for monitoring. Only QI teams with at least two months of data registration were included in the analysis. If a second measurement occasion was needed for the indicator but was missing for a patient, the patient was excluded from the analysis. The mean scores for the indicators were calculated for each QI team.
After a period of one year, only a few of the QI teams met the standards that the clinical experts had pre-defined. We therefore decided to measure improvement relative to baseline and assumed the baseline scores for the performance indicators to be zero, as the multidisciplinary guidelines had not yet been implemented at baseline. Patient outcomes and process of care outcomes were measured using the Clinical Global Impression-Severity (CGI-S), Manchester Short Assessment of Quality of Life (MANSA), Health of the Nation Outcome Scale (HONOS), and the Four Dimensional Symptom Questionnaire (4DSQ) (Table 1). We then identified those QI teams with the most and least improvement for the different performance indicators within each QIC: Those teams with change scores in the lowest quartile (≤ 25%) were considered least successful; those teams with change scores in the highest quartile (≥ 75%) were considered most successful. Depending on the number of QI teams participating in the QIC, two or three teams could be identified as least and most successful, respectively. Next we calculated the mean impact of the most and least successful QI teams.
Finally we calculated the mean scores of the characteristics (related to the composition, participation, functioning, and organizational context) for the QI teams in the lowest and highest quartiles.
Because our numbers were too low for statistical testing and the sampling was not random, we decided to consider a factor as possibly associated with the impact of a team when the following three conditions were met: the mean scores for the factor from the most successful QI teams did not fall within the range of the scores from the least successful QI teams; the mean scores for the factors from the least successful QI teams did not fall within the range of the scores from the most successful QI teams; and both these conditions were met for at least two of the performance indicators or in at least two of the three QICs.
Major associations between QI team factors (i.e., composition, participation, functioning and the 0rganizational context) and performance indicators (i.e., patient outcomes, monitoring, screening/assessment) for successful teams
Participation in the national program
Associations between characteristics of QI teams (i.e., composition, participation, functioning and the organizational context) and performance indicators (i.e., patient outcomes, monitoring, screening/assessment) for successful teams
Number of team members
Number of different professionals
Level of education:
% Master degree
% Bachelor degree
% Associate degree
Years of practice in this job
Years of practice in this organization
Number of team members with specialized knowledge
Time spent on improvement
% Involvement in quality improvement
Participation in the national program
% Attendance conferences QI team members
% Attendance conferences by QI team leaders
Attitude quality improvement
Attitude guidelines, factor innovation (specificity/flexibility)
Organizational conditions present
Type of leadership:
Final study population
Characteristics of the QI teams per QIC (N = 26)
Number of QI teams
Number of organizations represented by QI teams
Number of questionnaires sent
Respondents/known non- respondents#
Mean number of members per QI team (range amongst QI teams)
6.2 (5 to 8)
7.5 (3 to 13)
10 (7 to 13)
7.5 (3 to 13)
Mean age per QI team (range amongst QI teams)
44.2 (25 to 62)
42.8 (25 to 59)
38.4 (22 to 57)
42.0 (22 to 62)
Number of patients reached (range amongst QI teams)
571 (10 to 165)
467 (38 to 82)
308 (11 to 87)
1346 (1 to 165)
Impact of the QI teams on healthcare improvement
The actual measurement period for the teams varied from 2 to 12 months. In the QIC for anxiety disorders, the period ranged from 7 to 12 months; in the QIC for schizophrenia, six teams of the eight teams registered patient data for more than eight months; and in the QIC for dual diagnosis, the period ranged from 2 to 8 months. As the measurement period proceeded, the number of registrations per patient declined in all of the QI teams.
In the end, four QI teams were successful on at least two of the performance indicators: For dual diagnosis one QI team was successful on all three performance indicators; two QI teams for anxiety disorders and another dual diagnosis QI team were successful on patient and monitoring outcomes.
Impact of the QI teams on performance indicators in the QIC domains of anxiety disorders, dual diagnosis, and schizophrenia
Range amongst QI teams
Range amongst QI teams
Range amongst QI teams
(N = 12)
0 to 23.9%
7.9% to 78.6%
0.9‰ to 2.3-‰
(N = 6)
0 to 27.3%
0 to 28.9%
9.1 to 82.8%
(N = 8)
2.1% to 23.3%
52.6% to 98.0%
After six months of treatment, 8% of the patients with severe anxiety disorders were less anxious (range 0 to 23.9% amongst QI teams). The social functioning of the patients with schizophrenia improved with 13.3% (range 2.1% to 23.3%). And 8.6% of the patients with a dual diagnosis experienced an improved quality of life (range 0 to 27.3%).
Process of care outcomes: monitoring
The QI teams for anxiety disorders monitored the severity of condition for 35.9% of their patients (range 7.9% to 78.6% amongst QI teams). The social functioning of 48.8% of the patients suffering from schizophrenia was systematically monitored (range 52.6 to 98.0%). The QI teams for dual diagnosis monitored the quality of life for 5.8% of their patients (range 0 to 28.9%).
Process of care outcomes: screening/assessment
Eight QI teams of anxiety disorders executed goals on screening and assessment with the 4DSQ. These primary care teams screened 1.4‰ of patients per practice (range 0.9‰ to2.3‰ amongst QI teams). The QI teams for dual diagnosis screened 33.7% of patients for psychopathology using the HONOS (range 9.1% to 82.8%). The QIC for schizophrenia had no screening/assessment goals.
Questionnaire response rate
Cronbach's α of the validated questionnaire items measuring characteristics of the QI teams
Attitude quality improvement (EBPA)
(= attitude towards change in working routine)
Team Climate Inventory (TCI)
Type of leadership
Associations between QI team factors and performance indicators
In Table 3, the major associations between QI team factors (i.e., composition, participation in national program, functioning, and organizational context) and the performance indicators we measured (i.e., patient outcome, monitoring, screening/assessment) are presented. The direction of the influence of the factors is noted. The results are summarized below.
All the possible associations in the composition of the successful QI teams were either mixed or absent.
Participation the national program
Only the participation of the most successful QI teams from the dual diagnosis QIC showed a possible positive association with patient outcomes and monitoring. The participation of the most successful QI teams from the schizophrenia QIC appeared to negatively affect monitoring.
The team functioning of the most successful QI teams appeared to be associated with the following outcomes: the functioning of the QI teams within the dual diagnosis QIC appeared to promote patient outcomes and monitoring; for anxiety disorders, team functioning appeared to negatively affect patient outcomes and screening/assessment; and the functioning of the QI teams within the schizophrenia QIC also appeared to negatively affect patient outcomes. Other associations were either mixed or absent.
The organizational context for the most successful QI teams appeared to be associated with better patient outcomes and greater monitoring in the domain of schizophrenia and greater monitoring in the domain of dual diagnosis. Other associations were either mixed or absent.
Associations between characteristics of the QI teams and performance indicators
In Table 4, we report the possible associations between 22 characteristics of the QI teams and the performance indicators: patient outcome, monitoring, and screening/assessment. The direction of the associations is also indicated, and the results are discussed further below. The findings for the least and most successful QI teams in the domains of dual diagnosis, depression, and schizophrenia are available in additional files (Additional Files 1, 2, and 3).
Out of nine characteristics of QI team composition, two appeared to relate to team success: years of employment in the organization and the level of education.
QI teams that had worked fewer years in the organization they represented showed better results for screening/assessment in the domains of anxiety disorders (average 5.3 years, range 4.7 to 5.8 years amongst QI teams) and dual diagnosis (average 3.9 years, range 3.9 to 3.9 years).
QI teams that counted more members with an associate degree and fewer members with a master's degree had better results. They showed better patient outcomes, greater monitoring, and screening/assessment within the domain of dual diagnosis. The successful teams consisted of fewer psychiatrists, general practitioners, psychologists, and psychotherapists (average 12.3%, range 9.1 to 15.4%, average 22% amongst QI teams, range 15.4 to 28.6% amongst QI teams, respectively). A similar pattern was found for the most successful QI teams within the domain of schizophrenia (average 16.1%, range 15.4 to 16.7%). They showed greater monitoring.
The successful QI teams in the domain of dual diagnosis also consisted of more mental health nurses and practice assistants (average 34.3%, range 23.1 to 45.5%) they showed better patient outcomes and greater monitoring.
Characteristics of team participation in national program
A higher average attendance of the recommended conferences appeared to be associated with better results in the domain of dual diagnosis in particular (average attendance 77.5%, range 67.5 to 87.5% amongst QI teams). The successful QI teams showed better patient outcomes and greater monitoring.
Characteristics of team functioning
Out of eight characteristics of QI team functioning, three appeared to be associated with improvement: views about the team's QI efficacy; the attitudes towards change; and views about communication with regard to the innovation within the QI team.
Those QI teams with less positive perceptions about their QI efficacy showed better performance results. Better patient outcomes were found for the QI teams with less positive efficacy expectations in the domains of anxiety disorders (average item score 3.2, range 3.2 to 3.3 amongst QI teams) and schizophrenia (average item score 3.1, range 3.0 to 3.2).
Those QI teams with less positive attitudes towards change also showed better results in some domains. In the domain of schizophrenia, those QI teams with less positive attitudes towards change (average item score 3.5, range 3.5 to 3.5 amongst QI teams) showed better patient outcomes. In the domain of dual diagnosis, the QI teams with less positive attitudes towards change (average item score 3.7, range 3.4 to 3.9) showed greater screening/assessment. Finally, the QI teams with positive views about communication with regard to the innovation in the domain of dual diagnosis (average item score3.5, range 3.4 to 3.6) showed better patient outcomes; those in the domain of schizophrenia (average item score 4.1, range 4.0 to 4.1) showed greater monitoring.
Characteristics of the organizational context
All characteristics—the presence of organizational conditions for improvement, support from the management, and type of leadership—appeared to be associated with better patient outcomes and process of care outcomes.
QI teams with enough time, workforce, sponsoring, and skills generally showed better results. The QI teams with good organizational conditions in the domain of dual diagnosis (average item score 4.6, range 4.4 to 4.8 amongst QI teams) showed better patient outcomes and greater monitoring.
Similarly, those that felt supported by their management in the domain of dual diagnosis (average item score 5.6, range 5.3 to 5.8 amongst QI teams) showed relatively better patient outcomes and greater monitoring. Also in the domain of anxiety disorders, the successful QI teams that felt supported by the management of the organization (average item score 4.4, range 4.0 to 4.9, average item score 4.4, range 4.2 to 4.5, respectively) showed greater monitoring and screening/assessment.
QI teams with an inspiring leader showed better results . The successful teams with particularly inspirational leadership in the domain of anxiety disorders (average item score 3.6, range 3.4 to 3.9, average item score 3.9, range 3.4 to 4.3 amongst QI teams, respectively) showed better patient outcomes and greater monitoring. The successful teams in the domain of schizophrenia (average item score 4.2, range 3.8 to 4.5) showed greater monitoring. And the successful QI teams in the domain of dual diagnosis (average item score 4.0, range 3.9 to 4.1) showed better patient outcomes.
Finally, those QI teams with a more active leader (i.e., a lower average item score) showed better results as well. The successful QI teams with active leadership in the domain of anxiety disorders (average item score 1.8, range 1.0 to 2.2 and an average item score 1.8, range 1.5 to 2.0 amongst QI teams, respectively) showed improvement on all performance indicators (i.e., patient outcomes, monitoring, and screening/assessment). The successful QI teams with active leadership in the domain of dual diagnosis (average item score 1.8, range 1.7 to 1.8) showed relatively better patient outcomes.
The successful QI teams in the domain of schizophrenia had an average item score of 2.8 (range 2.5 to 3.1), which matched the minimum score needed to be considered a team with an active leader as proposed by Den Hartog ; they showed better patient outcomes.
In this study, we explored which aspects of the composition, participation, functioning, and organizational contexts for QI teams appeared to be associated with improved quality of mental healthcare. The impact of QI teams improvements efforts to promote the implementation of multidisciplinary evidence-based guidelines and recommendations were examined in the domains of anxiety disorders, dual diagnosis, and schizophrenia.
A general clear pattern of results could not be detected. Factors derived from theory did not perform better than factors derived from practice. Only modest support was found for a few select factors but not on a widespread basis. Support from the management of the organization and active, inspirational QI team leadership related to somewhat more than 50% or the performance outcomes for the most successful QI teams in the domains we studied. In other research evaluating a national program aimed at quality improvement in Dutch hospitals , support from the management was also found to be an important factor for success.
Less consistently, factors like average level of education in the QI team, number of years at current job, and previous involvement in QI efforts appeared to be associated with improved quality of mental healthcare.
QI teams with more members with an associate degree and fewer members with a master's degree surprisingly had better results. It is possible that QI teams need members from a variety of backgrounds and thus not only members with strong theoretical backgrounds (i.e., members with a master's degree)  to be successful. QI teams need professionals who are able to take control of the improvement initiative, supervise the improvement process, and record data or, in other words, members with an associate degree who tend to be general nurses .
The number of years of practice (i.e., on the current job and prior involvement in QI efforts showed possible associations with at least 50% of the performance indicators for the successful QI teams. Although the directions of the associations were mixed, purely the number of associations suggests that these factors play a role in the efforts of teams to improve the quality of mental healthcare.
The present findings are in keeping with the findings summarized in reviews by Hulscher and Schouten [6, 7] who reported only a few factors to appear to influence the effectiveness of QICs. Comparison of QI teams within specific QICs (limited to one domain) is therefore called for, using a mixed approach that thus combines both quantitative and qualitative methods to better understand improvement processes in particularly the field of mental healthcare [6, 29].
Strengths and weaknesses of the study
A strong point of the present study is that it was conducted close to daily mental healthcare practice. The findings are thus likely to reflect real people during real practice.
Most of the evaluations of QICs to date have been based on general hospitals and nursing homes, which makes the present findings a welcome contribution. That is, the present results contribute to our knowledge of the impact of QICs in general and within the field of mental healthcare in particular.
Limitations on the present study should be mentioned as well. The number of participating QI teams was small and only QI teams with a minimum of two months of data registration were included in the analysis.
Moreover, the amount of registration (i.e., data) per patient declined as the study proceeded. No relation, however, was found between QI team success and the amount of data available per team. How we handled missing data (if a second measurement occasion was needed to calculate the performance indicator but was missing for a patient, the patient was excluded from the analysis) produced a conservative estimate of team success, which may mean that the true impact of the QI teams has been underestimated. Also, the range in the number of patients reached and the different number of registered measurements per patient may have affected the decision about the most and least successful QI teams.
With the exception of the screening/assessment rates for patients with anxiety disorders, it is not known whether the denominator for the performance indicator scores corresponds to the number of patients who could be expected in the participating mental healthcare organizations and general practices. The absence of baseline measurements and control groups is yet another limitation on the present study. For some of the improvement measures, the internal consistency—as expressed by the Cronbach's alpha—was only moderate.
In the present research, we opted to study a wide range of factors in conjunction with the efforts of QICs to improve the quality of mental healthcare. To discern a relationship between these factors and improved mental healthcare outcomes, we used a qualitative cross-sectional study design. Because the numbers proved too low for statistical testing and sampling was not random, we were only able to identify possible associations. Further research is therefore needed to confirm the positive findings of our study while addressing limitations.
The QICs in this study aimed to improve patient care by implementing the recommendations provided by multidisciplinary practice guidelines in three separate domains of mental healthcare. The success of the QICs appears to depend on a range of largely still unknown factors. On the basis of the present research, however, it can be concluded that QI teams should be given active and inspiring team leadership. Support from management of the organization is also important. QI teams should have a variety of members: In addition to professionals with heavy theoretical backgrounds, the teams should include professionals who are willing (and able) to take control of the improvement process, supervise the improvement process, and record the necessary data to do all this. The views of QI team members regarding the goals of the team or the QI effort appear to be of less importance. But the number of years of practice on the current job and prior involvement in QI efforts do appear to be important for successful QI team performance. Additional study is nevertheless needed to identify factors associated with the impact of QICs on the implementation of multidisciplinary practice guidelines. We recommend a mixed approach that thus combines both quantitative and qualitative methods to better understand improvement processes in particularly the field of mental healthcare
Juliette Cruijsberg for participation in the conduction of the study. Gera Nagelhout for participation in the design of the study. The Trimbos Institute for its cooperation and the opportunity to access the data from the QICs that participated in the national quality improvement program in mental health care. The participating QI teams for their registration and provision of patient data. ZonMw, the Netherlands Organization of Health Research and Development funded the research.
Ethics committee approval
The research ethics committee CMO of the region Arnhem-Nijmegen (letter of 3 April 2008; SE/CMO 2008/061) granted approval for the study. The committee concludes in this letter that in compliance with the law on Medical-scientific research (WMO), the study does not need approval. With the exception of the questionnaires for healthcare professionals, patient outcomes and QI team performance data were anonymously extracted using standardized spreadsheets.
- Ovretveit J, Bate P, Cleary P, Cretin S, Gustafson D, McInnes K, Mc Leod H, Molfenter T, Plsek P, Robert G, Shortell S, Wilson T: Quality collaboratives: lessons from research. Qual Saf Health Care. 2002, 11: 345-351. 10.1136/qhc.11.4.345.View ArticlePubMedGoogle Scholar
- Wilson T, Berwick DM, Cleary PD: What do collaborative improvement projects do? Experience from seven countries. Jt Comm J Qual Saf. 2003, 29: 85-93.PubMedGoogle Scholar
- Kilo CM: A framework for collaborative improvement: lessons from the Institute for Healthcare Improvement's Breakthrough Series. Qual Manag Health Care. 1998, 6: 1-13.View ArticlePubMedGoogle Scholar
- Plsek PE: Collaborating across organizational boundaries to improve the quality of care. Am J Infect Control. 1997, 25: 85-95. 10.1016/S0196-6553(97)90033-X.View ArticlePubMedGoogle Scholar
- Berwick DM: Developing and testing changes in delivery of care. Ann Intern Med. 1998, 128: 651-656.View ArticlePubMedGoogle Scholar
- Hulscher M, Schouten LM, Grol R: Collaboratives. 2009, London, QQIP, The Health FoundationGoogle Scholar
- Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol RP: Evidence for the impact of quality improvement collaboratives: systematic review. BMJ. 2008, 336: 1491-1494. 10.1136/bmj.39570.749884.BE.View ArticlePubMedPubMed CentralGoogle Scholar
- Strating MMH, Zuiderent-Jerak T, Nieboer AP, Bal RA: Evaluating the Care for Better collaborative. Results of the first year of evaluation. 2008Google Scholar
- Kilo CM: Improving care through collaboration. Pediatrics. 1999, 103: 384-393.PubMedGoogle Scholar
- Trimbos-instituut: Landelijke Stuurgroep Multidisciplinaire Richtlijnontwikkeling in de GGZ. Multidisciplinaire Richtlijn Angststoornissen. Richtlijn voor de diagnostiek, behandeling en begeleiding van volwassen cliënten met een angststoornis (Multidisciplinary guideline for diagnosis, treatment and monitoring of adult clients with anxiety disorders). Utrecht. 2003Google Scholar
- Parnassia Addiction Research Center: Dubbele Diagnose, Dubbele hulp. Richtlijnen voor diagnostiek en behandeling (Dual diagnosis, dual help. Guideline for diagnosis and treatment). Den Haag. 2003Google Scholar
- Trimbos-instituut: Landelijke Stuurgroep Multidisciplinaire Richtlijnontwikkeling in de GGZ. Richtlijn voor de diagnostiek, zorgorganisatie en behandeling van volwassen mensen met schizofrenie (Multidisciplinary guideline for diagnosis, organization of care, and treatment for adult clients with schizophrenia). Utrecht. 2005Google Scholar
- Routhieaux RL, Gutek BA: TQM/CQI effectiveness at team and departmental levels. Journal of Quality Management. 1998, 3: 39-62. 10.1016/S1084-8568(99)80103-5.View ArticleGoogle Scholar
- Kimberly JR, Evanisko MJ: Organizational innovation: the influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. Acad Manage J. 1981, 24: 689-713.View ArticlePubMedGoogle Scholar
- Priebe S, Fakhoury W, White I, Watts J, Bebbington P, Billings J, Burns T, Johnson S, Muijen M, Ryrie I, Wright C: Characteristics of teams, staff and patients: associations with outcomes of patients in assertive outreach. Br J Psychiatry. 2004, 185: 306-311. 10.1192/bjp.185.4.306.View ArticlePubMedGoogle Scholar
- Alexander JA, Lichtenstein R, Jinnett K, Wells R, Zazzali J, Liu D: Cross-functional team processes and patient functional improvement. Health Serv Res. 2005, 40: 1335-1355. 10.1111/j.1475-6773.2005.00418.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Damoiseaux V, van der Molen HT, Kok GJ: Gezondheidsvoorlichting en gedragsverandering. 1993, Assen: Van GorcumGoogle Scholar
- Aarons GA: Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004, 6: 61-74.View ArticlePubMedPubMed CentralGoogle Scholar
- Strating M, Nieboer A: Psychometric test of the Team Climate Inventory-short version investigated in Dutch quality improvement teams. BMC Health Services Research. 2009, 9: 126-10.1186/1472-6963-9-126.View ArticlePubMedPubMed CentralGoogle Scholar
- Ouwens M, Hulscher M, Akkermans R, Hermens R, Grol R, Wollersheim H: The Team Climate Inventory: application in hospital teams and methodological considerations. Quality and Safety in Health Care. 2008, 17: 275-280. 10.1136/qshc.2006.021543.View ArticlePubMedGoogle Scholar
- Wheelan SA, Burchill CN, Tilin F: The link between teamwork and patients' outcomes in intensive care units. Am J Crit Care. 2003, 12: 527-534.PubMedGoogle Scholar
- Anderson NR, West MA: Measuring climate for work group innovation: development and validation of the team climate inventory. Journal of Organizational Behavior. 1998, 19: 235-258. 10.1002/(SICI)1099-1379(199805)19:3<235::AID-JOB837>3.0.CO;2-C.View ArticleGoogle Scholar
- Peters MAJ, Harmsen M, Laurant MGH, Wensing M: Ruimte voor verandering? Knelpunten en mogelijkheden voor verbeteringen in de patiëntenzorg (Constraints and possiblities in the improvement of patient care). Afdeling Kwaliteit van zorg (WOK), UMC St Radboud. Nijmegen. 2003Google Scholar
- Den Hartog DN, Van Muijen JJ, Koopman PL: Transactional Versus Transformational Leadership: An Analysis of the MLQ. Journal of Occupational and Organizational Psychology. 1997, 70: 19-34. 10.1111/j.2044-8325.1997.tb00628.x.View ArticleGoogle Scholar
- Damanpour V: Organizational Innovation: A Meta-Analysis of Effects of Determinants and Moderators. The Academy of Management Journal. 1993, 34: 555-590.View ArticleGoogle Scholar
- Antonakis J, Avolio BJ, Sivasubramaniam N: Context and leadership: an examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. The Leadership Quarterly. 2003, 14: 261-295. 10.1016/S1048-9843(03)00030-4.View ArticleGoogle Scholar
- Van Splunteren P, van Everdingen J, Janssen , Minkman M, Rouppe van der Voort M, Schouten LM: Doorbreken met resultaten. Verbetering van de patiëntenzorg met de Doorbraakmethode (Improvement of patient care with Quality Improvement Collaboratives). 2003Google Scholar
- Duckers MLA, Wagner C, Groenewegen PP: Evaluating the implementation and effects of a multilevel quality collaborative in hospital care. Quality and safety improvement research: methods and research practice from the international quality improvement research network. 2008, 105-126.Google Scholar
- Mittman BS: Creating the Evidence Base for Quality Improvement Collaboratives. Ann Intern Med. 2004, 140: 897-901.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.