- Research
- Open Access
- Open Peer Review
Efficiency of the implementation of cardiovascular risk management in primary care practices: an observational study
- Received: 24 September 2015
- Accepted: 4 May 2016
- Published: 13 May 2016
Abstract
Background
This study aimed to document the variation in technical efficiency of primary care (PC) practices in delivering evidence-based cardiovascular risk management (CVRM) and to identify associated factors.
Methods
This observational study was based on the follow-up measurements in a cluster randomized trial. Patients were recruited from 41 general practices in the Netherlands, involving 106 GPs and 1671 patients. Data on clinical performance were collected from patient records. The analysis focused on PC practices and used a two-stage data envelopment analysis (DEA) approach. Bias-corrected DEA technical efficiency scores for each PC practice were generated, followed by regression analysis with practice efficiency as outcomes and organizational features of general practice as predictors.
Results
Not all PC practices delivered recommended CVRM with the same technical efficiency; a significant difference from the efficient frontier was found (p < .000; 95 % CI 1.018–1.041). The variation in technical efficiency between PC practices was associated with training practice status (p = .026). Whether CVRM clinical tasks were performed by a practice nurse or a GP did not influence technical efficiency in a statistical significant way neither did practice size.
Conclusions
Technical efficiency in delivering evidence-based CVRM increased with having a training practice status. Nurse involvement and practice size showed no statistical impact.
Keywords
- Primary care
- Cardiovascular risk management
- Technical efficiency
- Data envelopment analysis
Background
Across the world, health-care systems experience challenges due to ageing populations and increases in chronic and lifestyle-related diseases [1] and rising health-care costs [2]. Many policymakers believe that a strong primary care is a crucial component of the health-care system [3]. Primary care can only fulfil this expectation, if evidence-based prevention and treatment is provided in all eligible patients at reasonable costs. This study focused on the delivery of evidence-based prevention of cardiovascular disease in primary care. Cardiovascular diseases (CVD) are common in the Netherlands; the number of CVD-related hospital admissions was almost 375,000 in 2012 [4]. Many programmes have focused on prevention of CVD and related disease, but still, they remain an important cause of mortality and morbidity [5]. Cardiovascular risk management (CVRM) is largely provided in primary care (PC) in the Netherlands. Although these efforts have resulted in improved cardiovascular prevention care over recent years, there is a substantial part of CVD patients at risk in the Netherland that do not receive CVRM as recommended by prevailing evidence-based guidelines [6]. Little is known about the variation across general practice organizations regarding the efficiency of delivering evidence-based CVRM. In case of improvement of CRVM, substantial health benefits can be gained, while containing costs of healthcare at the same time [7]. Decision makers (both clinicians and policymakers) would be supported by insight into the efficiency of implementation of evidence-based CVRM in PC, as well as knowledge of associated organizational factors that influence the efficiency.
This study aimed at determining the technical efficiency of PC practices delivering recommended CVRM for patients with established cardiovascular disease and especially aiming at identifying factors associated with the variation in efficiency. Here, technical efficiency is associated with the use of optimal procedures (i.e. optimal implementation of procedures) and was defined as the extent to which a PC practice delivers evidence-based CVRM in relation to its inputs in terms of medical labour. For this purpose, we used a two-stage data envelopment analysis (DEA) approach.
DEA originates from management science and operation research and has mostly been used for benchmarking efficiency of organizational units in, for example, banking. It is a nonparametric approach determining a piecewise linear efficiency frontier along the most efficient organizational or decision-making units (DMUs) by means of linear programming. Then, DEA compares all inefficient DMUs to the efficient frontier. The PC practices (DMUs) delivering recommended CVRM that lay on the efficient frontier can be regarded as best practices. As a benchmarking technique, the DEA approach has gained much attention since it was first proposed in the late 1970s by Charnes A, Cooper WW and Rhodes E [8]. Presently, there are several thousand recorded scientific contributions, some theoretical and some applied [9]. DEA has been widely applied within the health-care setting mostly with a focus on hospitals, nursing homes and physicians (see for example [10–15]). Other applications of DEA are within the context of dialysis centres [16] and health systems [17]. A recent systematic review by Pelone et al. [18] shows 39 DEA applications in PC. According to Pelone et al. [18], DEA has a number of features which make it an attractive tool for efficiency measurement of PC delivery: ‘it can handle effectively the existence of multiple primary care resources (inputs) and multiple health outcomes (outputs) in the transformation process. Furthermore, it does not require strong assumptions about the underlying technology linking the inputs to the outputs, and it measures efficiency in relative -instead of absolute- terms.’
Methods
Study population and data
This study was based on the follow-up measurements in a two-arm cluster randomized trial with a block design [NCT00791362], which was executed from September 2008 until September 2012 [19]. Patients were recruited from 41 general practices in the Netherlands, thereby involving 106 general practitioners. Only patients with established CVD, namely angina pectoris, acute myocardial infarction, transient ischemic attack (TIA), ischemic stroke, peripheral arterial disease, aortic aneurysm and other chronic ischemic heart diseases, were included from these PC practices. All these practices were enrolled in the national accreditation programme (NHG-Praktijkaccreditering) and used electronic medical records and International Classification of Primary Care codes (ICPC codes), a worldwide system to label conditions in primary care [19]. All measures used for the analysis were derived from patient records; for a detailed description of the variables, we refer to the published study protocol [19]. This study focused on the delivery of evidence-based prevention of cardiovascular disease in primary care, which covers monitoring, counselling and preventive medication. This study however did not show an effect on primary outcomes [20].
The present study uses data from the trial by Nouwens et al. [19], but it does not answer questions in the context of the initial trial. For instance, it lumps data from intervention and control groups for an observational study. So, the present study is a newly post hoc developed study.
Data-analysis
-
The DEA model runs under the assumption of constant returns to scale (CRS) as we adhere a policymaker perspective and assume that the underlying technology is in a ‘steady state’.
-
An output-oriented DEA model is performed to explore how each general practice could proportionally maximize its outputs given the inputs provided and move to a point on the efficient frontier.
-
DEA analysis does not require trade-offs between the various types of outputs or subjective judgments. Under the assumption that all general practices were homogeneous in their production technology (CVRM) and assigning the same importance to the inputs and outputs, no weight restrictions were given in the model.
-
DEA computes all the aforementioned values for each PC practice, taking into account its efficient peer PC practice(s) that use similar input–output ratios, but at a more efficient level.
Stage 1
Measures of the DEA: the production function
The input variables
Input is defined here as the average amount of labour time spent on a CVD patient within a period of 3 months, which was operationalized in terms of consultations. Labour in a PC practice can be performed by a GP, a practice nurse or a practice assistant.
-
Average number of consultations with a GP
-
Average number of consultations with a practice nurse
-
Average number of consults with a practice assistant
For establishing the input variables, all consultations per patient with each specific health-care worker were counted within a period of 3 months. Only consultations which were related to cardiovascular diseases, diabetes mellitus and chronic obstructive pulmonary disease were included. Also, the type of contact was taken into account, namely three different kinds were distinguished: telephone consultation, normal consultation and home visit. These were, respectively, counted and weighted with a factor .5, 1 and 1.5, respectively [22].
The output variables
-
Performance indicator, based on prescribing of recommended preventive medication
-
LDL cholesterol
-
Systolic blood pressure
The variables that were used to create the performance indicator are statin and anticoagulants prescription. Both medications need to be prescribed for nearly all included patients, according to the CVRM guideline [23]. The original variables (statin and anticoagulants prescription) are the basis of the performance indicator and have two categories: 0 ‘no’ and 1 ‘yes’. The scale of the composited measure (performance indicator) has two categories, namely 1 for ‘imperfect adherence to the guidelines’ (meaning none or one of the medications recommended by the guideline is prescribed) or 5 for ‘full adherence to the guidelines’ (meaning all of the medications recommended by the guideline are prescribed). The two categories (1 and 5) were chosen to reflect a 20 % relative reduction in the risk of cardiovascular events for full adherence to the guideline compared to imperfect adherence [24, 25].
Under the assumption of an output-oriented CRS DEA model that outputs should be increased, wrong conclusions will arise for LDL and SBP. Therefore, both variables were transformed. For both, a linear transformation was used of the form: y desirable = − y undesirable + v > 0 (−LDL plus 10; −SBP plus 300) [17, 26].
DEA measures efficiency relative to an estimate of the efficient frontier.
The difference between the DEA score and the efficient frontier is the relative (in)efficiency. This, however, neglects the fact that the sample of DMUs (PC practices) on which the DEA is performed underlays statistical noise and random error. In fact, the obtained efficiency scores are subject to uncertainty due to sampling variation. We therefore used a bootstrap DEA approach to estimate the bias-corrected efficiency scores applying 1000 bootstrap replications as proposed by Simar and Wilson [27]. We used the Benchmark package in R. This approach returns results directly as Farrell efficiency measures [9, 28].
Stage 2
Variables in the regression analyses
-
Practice size: The size of a PC practice has been found to influence efficiency [29].
-
Training practice: These practices educate future GPs, and therefore, their patients can benefit from more attention and a culture that is orientated on learning and reflection.
-
Task CVRM practice nurse: If a practice nurse performs CVRM clinical tasks instead of the GP, research shows that chronic care is similar or better although not necessarily less costly [30, 31].
-
Age: The older the patient, the more complicated treatments they have.
-
Gender: Men have a predisposition for cardiovascular disease.
-
Diabetes mellitus: Co-morbidity might influence the efficiency of the CVRM care [32, 33]. Diabetes is included because structured care programmes for patients with diabetes have been implemented nationwide in the Netherlands. These structured care programmes for diabetes have similarity with the implementation of CVRM. PC practices that have these structured diabetes programmes are more likely to implement CVRM successful.
-
Socio-economic status: Whether or not a practice is located in a disadvantaged neighbourhood (a neighbourhood with several infrastructural and societal shortcomings) can influence CVRM care.
The second stage DEA analysis was performed within the context of an assumed censoring data-generating process (CDGP) (data piled up at a censoring point (i.e. the efficient frontier)) and a fractional data-generating process (FDGP). Under the assumption of a CDGP, usually, the bias-corrected output-oriented DEA scores were regressed on explanatory variables using a Tobit regression. There is discussion that the efficiency scores are not generated by a censoring process but are fractional or proportional data [34]. Then, in the model considered by Simar and Wilson [27], a truncated regression provides for consistent estimation in the second stage [34]. Both regression models (Tobit and truncated) were performed using the statistical package STATA 13.1. The truncated regression was considered the base case, and these results are explicitly reported in the main text and Table 2.
Results
PC practice characteristics
Practice number | Practice size | Training practice | Tasks CVRM practice nurse | Mean age | Sex (% men) | Diabetes (% of patients with diabetes) | Socio-economic statusa | DEA scoreb |
---|---|---|---|---|---|---|---|---|
1 | 2473 | No | Yes | 66.30 | 63.0 | 10.9 | No | 1.105 |
2 | 2460 | Yes | Yes | 70.93 | 60.0 | 15.6 | No | 1 |
3 | 7500 | Yes | Yes | 66.55 | 74.5 | 32.7 | Yes | 1.084 |
4 | 3230 | No | Yes | 68.46 | 78.4 | 32.4 | No | 1 |
5 | 13,000 | Yes | Yes | 71.04 | 72.5 | 24.9 | No | 1.089 |
6 | 2600 | Yes | Yes | 67.68 | 52.5 | 17.5 | No | 1.004 |
7 | 4600 | Yes | Yes | 73.55 | 64.5 | 22.6 | Yes | 1.057 |
8 | 12,700 | Yes | Yes | 68.79 | 61.8 | 17.6 | No | 1.026 |
9 | 3400 | No | Yes | 66.76 | 69.6 | 28.3 | No | 1.092 |
10 | 3000 | No | Yes | 64.44 | 47.2 | 16.7 | No | 1.059 |
11 | 5300 | Yes | No | 66.49 | 79.1 | 25.6 | No | 1.010 |
12 | 4400 | Yes | Yes | 68.43 | 73.5 | 6.0 | No | 1.035 |
13 | 6800 | Yes | Yes | 67.57 | 62.9 | 0 | No | 1.042 |
14 | 3500 | Yes | Yes | 66.27 | 63.6 | 0 | No | 1 |
15 | 3200 | Yes | Yes | 65.53 | 70.6 | 14.7 | No | 1 |
16 | 2650 | Yes | Yes | 70.31 | 61.9 | 21.4 | No | 1.047 |
17 | 2500 | No | No | 69.90 | 68.8 | 37.5 | No | 1 |
18 | 4100 | Yes | Yes | 72.47 | 64.4 | 4.4 | No | 1.078 |
19 | 2998 | No | No | 66.90 | 59.0 | 2.6 | No | 1.049 |
20 | 2245 | Yes | Yes | 61.40 | 80.0 | 33.3 | Yes | 1.005 |
21 | 2820 | Yes | Yes | 69.71 | 60.5 | 26.3 | No | 1 |
22 | 2900 | Yes | No | 65.00 | 71.9 | 18.8 | No | 1.025 |
23 | 2286 | Yes | Yes | 72.40 | 68.0 | 12.0 | No | 1 |
24 | 2475 | Yes | Yes | 66.29 | 45.7 | 31.4 | No | 1.012 |
25 | 1900 | No | Yes | 65.13 | 37.5 | 50.0 | Yes | 1.014 |
26 | 2600 | Yes | No | 69.32 | 38.7 | 35.5 | No | 1 |
27 | 2650 | No | Yes | 57.00 | 75.0 | 25.0 | Yes | 1 |
28 | 2640 | No | Yes | 75.00 | 57.6 | 15.2 | No | 1.036 |
29 | 3600 | Yes | Yes | 68.82 | 78.9 | 5.3 | No | 1 |
30 | 4900 | Yes | No | 67.52 | 73.9 | 13.0 | No | 1.000 |
31 | 2620 | Yes | Yes | 68.25 | 67.9 | 17.9 | No | 1.004 |
32 | 8000 | Yes | Yes | 69.84 | 64.9 | 16.2 | No | 1 |
33 | 2758 | No | Yes | 65.87 | 82.6 | 15.2 | No | 1.009 |
34 | 3018 | Yes | Yes | 67.21 | 70.6 | 32.4 | No | 1 |
35 | 2648 | Yes | Yes | 71.94 | 62.9 | 17.1 | No | 1 |
36 | 3400 | Yes | Yes | 69.71 | 58.5 | 17.1 | No | 1 |
37 | 2600 | No | Yes | 67.94 | 57.6 | 6.1 | No | 1.120 |
38 | 6550 | Yes | Yes | 67.64 | 66.7 | 24.0 | Yes | 1.111 |
39 | 1850 | Yes | Yes | 72.33 | 61.5 | 30.8 | No | 1.045 |
40 | 7000 | Yes | Yes | 66.94 | 94.4 | 8.3 | No | 1 |
41 | 2600 | Yes | Yes | 72.50 | 70.6 | 14.7 | No | 1.039 |
Results of truncated regression analysis
Bootstrapped truncated regression (1000 replications) | ||||
---|---|---|---|---|
Coefficient | Bootstrapped | p value | ||
95 % confidence interval | ||||
Practice size | 6.86e-06 | −1.59e-07 | .0000139 | .055 |
CVRM task | .024 | −.0192 | .066 | .280 |
Training practice | −.053 | −.099 | −.006 | .026 |
Age | .004 | −.002 | .010 | .198 |
Gender | .078 | −.064 | .221 | .281 |
Diabetes | −.177 | −.351 | −.004 | .046 |
Socio-economic status | .056 | −.009 | .121 | .089 |
Discussion
This study provided insight into the technical efficiency of delivering evidence-based CVRM and illustrates the relevance of economic analysis of implementation in health-care organizations.
The analysis showed that several general practice organizations regarding their technical efficiency in providing recommended CVRM deviated from the efficient frontier. This implies that not all practices provide evidence-based CVRM at optimal efficiency; in fact, 15 of 41 practices provided evidence-based CVRM at optimal efficiency. Increased technical efficiency (better implementation of evidence-based CVRM for the same resources) is related to training practice status. We did not find evidence for the impact of nurse involvement and practice size on technical efficiency.
In efficiency analysis, there are two important competing methods, stochastic frontier analysis (SFA) and DEA [9]. According to Bogetoft and Otto, both methods have their own strengths and weaknesses and differ from each other with regard to (1) the usage, SFA is used more often in econometric efficiency analyses and DEA more often in operation research efficiency analyses; (2) the functional form of the efficient frontier, it is imposed a priori as in SFA or it is obtained a posteriori from the sample observations in an empirical way as in DEA and (3) whether they follow a deterministic or a stochastic approach (i.e. the distance of a DMU or PC practice from the efficient frontier is entirely attributed to its inefficiency as in DEA or partly due to statistical noise and random error, in addition to its inefficiency as in SFA) [9, 18]. Our method expands conventional DEA in a stochastic DEA which combines the nice features of conventional DEA (like no assumptions on functional form) and stochastic (SFA) methods. Stochastic DEA makes it possible to use in an unbiased way the DEA efficiency score in a multivariate regression analysis that provides insight in which factors influence efficiency of PC practices [27].
Some limitations of this study need to be discussed. The variation of efficiency across PC practices can partly be explained by the number of practices in the DEA. Including more practices would more accurately identify the efficient frontier; however, more practices can also cause some exogenous impacts of no interest or beyond control affecting the results.
Despite these limitations and the complexities of the methodology, we believe that the study has high relevance. Implementation science has largely focused on effectiveness of implementation strategies, but their efficiency is equally important. An effective yet costly implementation programme is often of little practical value, although it might contribute to scientific knowledge. DEA is a useful methodological tool for exploring this, which adds to the methods for efficiency research (see for example Pelone et al. [18]). For cardiovascular risk management, we found that primary care practices provide (‘produce’) evidence-based care at different efficiency levels. This information could guide both professional development and negotiations about reimbursement of cardiovascular care. Efficient practices have a number of characteristics, or lack organizational characteristics that are often assumed to be crucial, as discussed below.
The size of general practices (that is, number of registered patients) has been found to have a number of consequences. Higher practice size was associated with less physician time per 1000 patients, which suggested potential efficiency gains assuming equal quality [29]. The latter was not confirmed by this study, but the underlying mechanisms are complex. It might be that our study was underpowered (on PC level) with regard to practice size (p = .055). Moreover, larger practices provide more structured chronic disease care [35] and may therefore reach larger numbers of eligible patients, albeit at the cost of lowered technical efficiency in those who are diagnosed. This might have neutralized the impact of practice on efficiency in our analysis.
The positive impact of training practice status on technical efficiency was clear, but the practical relevance of this finding may be limited as not all practices can become training practices. There may be hidden resource use by GP trainees (that is, clinical work that is provided without a cost), which were not considered in the analysis. However, it is possible that training practices are truly more efficient, because these practices are regularly being assessed and stimulated to improve their performance as precondition of keeping their status as training practice. The effect may also be based on the selection of specific professionals into training practices, for instance professionals with higher interest in organizational topics.
Previous analysis on a different data set showed that more involvement of practice nurses was associated with improved cardiovascular risk management [30]. We found no evidence that this goes hand in hand with increased technical efficiency in delivering evidence-based CVRM. This is consistent with rigorous evaluations on the involvement of nurses in clinical activities in primary care, which found overall little impact of nurse involvement on cost or efficiency of healthcare [22]. Previous studies suggested that lower salary costs of nurses are offset by longer consultations and lower threshold for test ordering as compared to GPs.
Although it is generally assumed that co-morbidity is associated with decreased efficiency, this study found that the presence of diabetes actually enhanced efficiency of health-care delivery structured diabetes care has shown to be effective [32, 33] and is very well established care in the Netherlands. It overlaps partly with CVRM care which explains why ‘diabetes’ positively contributes to the efficiency of PC practice applying CVRM.
Conclusions
We are careful in providing strong recommendations for practice and policy on the basis of this study. Nevertheless, we showed that efficiency of providing evidence-based cardiovascular risk management can be quantified, which identified substantial differences between practices.
Considering the results as well as the emerging research evidence on PC practice organization, we conclude that practice size is associated with a range of outcomes. Larger practice size seems to have both positive and negative effects. Involving nurses in clinical and preventive activities has a number of advantages, but it seems to have little impact on efficiency. The positive impact of training status is consistent with other research: ideally, all practices should become training practices.
Declarations
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Authors’ Affiliations
References
- McKee M, Healy J. The changing role of hospitals in Europe: causes and consequences. Clin Med. 2001;1(4):299–304.View ArticleGoogle Scholar
- Culyer J, Newhouse JP, editors. “Handbook of health economics,” Handbook of Health Economics, Chapter 1 International comparisons of health expenditure: Theory, data and econometric analysis. Elsevier; 2000; 1(1): 1,00.Google Scholar
- Starfield B. Primary care: an increasingly important contributor to effectiveness, equity, and efficiency of health services. SESPAS report. Gac Sanit. 2012;26(1):20–6.View ArticlePubMedGoogle Scholar
- Hart – en vaatziekten in Nederland 2013. Cijfers over leeftijd, risicofactoren, ziekte en sterfte. Hartstichting 2013Google Scholar
- World Health Organization. Top ten causes of death. http://www.who.int/mediacentre/factsheets/fs310/en/
- Ludt S, Ose D, Rochon X, Van Lieshout J, Campbell S, Wensing M, Szecsenyi J. The challenge of cardiovascular prevention in primary care: implications of a European observational study in 8,928 patients at different risk levels. Eur J Prev Cardiol. 2014;21:203–13.View ArticlePubMedGoogle Scholar
- Delnoij DMJ, Van Merode G, Paulus A, Groenewegen PP. Does general practitioner gatekeeping curb health care expenditure? J Health Serv Res Pol. 2000;5:22–6.Google Scholar
- Charnes A, Cooper WW, Rhodes E. Measuring the efficiency of decision making units. Eur J Oper Res. 1978;2:429–44.View ArticleGoogle Scholar
- Bogetoft P, Otto L. Benchmarking with DEA, SFA, and R. New York: Springer; 2011.View ArticleGoogle Scholar
- Hollingsworth B, Street A. The market for efficiency analysis of health care organisations. Health Econ. 2006;15:1055–9.View ArticlePubMedGoogle Scholar
- Hollingsworth B, Dawson PJ, Maniadakis N. Efficiency measurement of health care: a review of non-parametric methods and applications. Health Care Manag Sci. 1999;2:161–72.View ArticlePubMedGoogle Scholar
- Hollingsworth B. Non-parametric and parametric applications measuring efficiency in health care. Health Care Manag Sci. 2003;6:203–18.View ArticlePubMedGoogle Scholar
- Hollingsworth B. The measurement of efficiency and productivity of health care delivery. Health Econ. 2008;17:1107–28.View ArticlePubMedGoogle Scholar
- Puig-Junoy J. Efficiency in primary health care: a critical review of frontier measures. Rev Esp Salud Publica. 2000;74:483–95.View ArticlePubMedGoogle Scholar
- Worthington AC. Frontier efficiency measurement in health care: a review of empirical techniques and selected applications. Medical Care Res Rev. 2004;61:135–70.View ArticleGoogle Scholar
- Shreay S, Ma M, McCluskey J, Mittelhammer RC, Gitlin M, Stephens JM. Efficiency of U.S. dialysis centers: an updated examination of facility characteristics that influence production of dialysis treatments. Health Serv Res. 2014;49(3):838–57.View ArticlePubMedPubMed CentralGoogle Scholar
- Adang EM, Borm GF. Is there an association between economic performance and public satisfaction in health care? Eur J Health Econ. 2007;8(3):279–85.View ArticlePubMedGoogle Scholar
- Pelone F, Kringos DS, Romaniello A, Archibugi M, Salsiri C, Ricciardi W. Primary care efficiency measurement using data envelopment analysis: a systematic review. J Med Syst. 2015;39:156.View ArticlePubMedGoogle Scholar
- Nouwens E, Van Lieshout J, Adang E, Bouma M, Braspenning J, Wensing M. Effectiveness and efficiency of a practice accreditation program on cardiovascular risk management in primary care: study protocol of a clustered randomized trial. Implement Sci. 2012;7:94.View ArticlePubMedPubMed CentralGoogle Scholar
- Nouwens E, van Lieshout J, Bouma M, Braspenning J, Wensing M. Effectiveness of improvement plans in primary care practice accreditation: a clustered randomized trial. PLoS One. 2014;9(12):e114045.View ArticlePubMedPubMed CentralGoogle Scholar
- Salinas-Jiménez J, Smith P. Data envelopment analysis applied to quality in primary health care. Ann Oper Res. 1996;67(1):141–61.View ArticleGoogle Scholar
- Oostenbrink JB, Koopmanschap en MA, Rutten FFH. Handleiding voor kostenonderzoek, Methoden en standaard kostprijzen voor economische evaluaties in de gezondheidszorg. Geactualiseerde versie 2010.Google Scholar
- NHG standard. Cardiovasculair risicomanagement. Huisarts Wet 2012;55(1):14–28.Google Scholar
- Gutierrez J et al. Statin therapy in the prevention of recurrent cardiovascular events: a sex-based meta-analysis. Arch Intern Med. 2012;172:909–19.View ArticlePubMedGoogle Scholar
- Taylor F et al. Statins work just as well in women as in men. Arch Intern Med. 2012;172:919–20.View ArticlePubMedGoogle Scholar
- Sarkis J. Preparing your data for DEA. In: Zhu J, Cook WD, editors. Modeling Data Irregularities and Structural Complexities in Data Envelopment Analysis. US: Springer; 2007.Google Scholar
- Simar L, Wilson PW. Sensitivity analysis of efficiency scores: how to bootstrap in nonparametric frontier models. Manage Sci. 2000;44:49–61.View ArticleGoogle Scholar
- Wilson PW. FEAR 1.0: a software package for frontier efficiency analysis with R. Socioecon Plan Sci. 2008;42:247–54.View ArticleGoogle Scholar
- Wensing M, van den Hombergh P, Akkermans R, van Doremalen J, Grol R. Physician workload in primary care: what is the optimal size of practices? A cross-sectional study. Health Policy. 2006;77(3):260–7.View ArticlePubMedGoogle Scholar
- Nouwens E, Van Lieshout J, Van den Hombergh P, Laurant M, Wensing M. Shifting care to nurses results in structured cardiovascular care management. Am J Manag Care. 2014;20:e278–284.PubMedGoogle Scholar
- Martínez-González NA, Djalali S, Tandjung R, Huber-Geismann F, Markun S, Wensing M, Rosemann T. Substitution of physicians by nurses in primary care: a systematic review and meta-analysis. BMC Health Serv Res. 2014;14:214.View ArticlePubMedPubMed CentralGoogle Scholar
- Tricco AC, Ivers NM, Grimshaw JM, Moher D, Turner L, Galipeau J, Halperin I, Vachon B, Ramsay T, Manns B, Tonelli M, Shojania K. Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet. 2012;379(9833):2252–61.View ArticlePubMedGoogle Scholar
- Nouwens E, van Lieshout J, Wensing M. Comorbidity complicates cardiovascular treatment: is diabetes the exception? Neth J Med. 2012;70(7):298–305.PubMedGoogle Scholar
- Simar L, Wilson PW. Two-stage DEA: caveat emptor. J Product Anal. 2011;36(2):205–18.View ArticleGoogle Scholar
- Wensing M, Van den Hombergh P, Doremalen V, Grol R, Szecsenyi J. General practitioners’ workload associated with practice size rather than chronic care organization. Health Policy. 2009;89:124–9.View ArticlePubMedGoogle Scholar